Server Room Darker

Global Bank delivers annual mainframe assured disaster recovery exercise using ICEFLO

Download PDF

About Assured Disaster Recovery

The Global Bank uses the IBM Mainframe as the bedrock technology infrastructure for the systems that underpin the Bank's critical data and services. IT resilience is a key element of the Bank's strategic priorities and is essential to retain customer trust and confidence. In this context, the Assured Disaster Recovery event demonstrates a crucial capability to Customers, staff, shareholders and the UK Regulator.

This "ADR" exercise involves a phased and controlled shutdown of the Bank's mainframe applications and infrastructure in one data centre, followed by a systematic restart of the systems and services in a secondary data centre. This process is then reversed to return all services to the originating data centre.

Get in Touch

The Problem

Performing any planned change of this magnitude represents a risk to the availability of service. The mandatory verification of IT resilience somewhat ironically places those business services at risk.

The problem is primarily one of planning and orchestrating a huge range of tasks, within a limited time-period, across Business and Technology teams that are geographically dispersed.

Co-ordination, collaboration and effective issue management are critical success factors.

The Goal

  • To perform a controlled and systematic shutdown of all in-scope banking services in the primary datacentre
  • To perform a controlled  and systematic start-up of all in-scope banking services in the secondary datacentre
  • To successfully operate these banking services for a period of time from the second datcentre
  • To reverse this process, reinstating the banking services to run from the primary datacentre
  • To perform all of the above while having no impact on Customers

The Challenge

An Assured Disaster Recovery rehearsal is a massive logistical, technical and collaboration challenge. The challenges faced are different before, during and after the event itself.

Before

  • Coordinating multiple teams across the globe and asserting runbook standards
  • Building integrated runbooks to deliver an overall DR cutover plan
  • Accurately identifying task duration and dependencies within each runbook
  • Providing evidence to secure sufficient confidence in the DR cutover plan 

During

  • Tightly orchestrating the array of people and activities during the DR cutover
  • Continuously tracking status and accurately forecasting the end time 
  • Identifying issues and managing them effectively, including their impact on the forecast end time
  • Prioritising scarce resources, typically people with multiple concurrent tasks
  • Providing accurate, timely and trusted status reports to senior executives
  • Avoiding any service impact due to an over-run of the ADR event itself

After

  • Providing a factual report of all that took place
  • Capturing lessons learned and delivering a proven DR cutover plan
  • Providing unequivocal evidence to the Regulator on the achievements made

The dedicated team of ADR specialists were familiar with all of these challenges and felt that ICEFLO helped them to better address each of them.

The Result

"The annual ADR Event was a huge success . The exercise completed in record time with no Production incidents associated with the Event. ICEFLO "did what is says on the tin" and helped deliver a controlled, safe execution of a complex technology cutover."- ICEFLO client. 

Growth
Growth
Growth
Growth
Orderly and collaborative progress
Time, effort and cost reductions achieved
Effective use of dress rehearsals
Total control with effective issue and risk management
Quotes
“By adopting ICEFLO to manage these critical events, the ADR team has enabled our global teams to adopt a standardised process of DR planning and execution. This has reduced the duration, effort and cost of these exercises which is great. The real benefit, however, is providing complete reassurance and comfort during those nerve jangling cutover periods.”
 
 

Anonymous, IT Operations

 

Share this story