As a tech chief, chances are you deal with ‘mini crisis’ situations regularly. But how do you respond when something truly terrible happens? How do show leadership while staying calm so you can think through how to best manage the situation?
Watch what happens during a crisis – some leaders rise to meet the issue head on while others freeze with uncertainty. These crisis events sometimes effect an individual operational system. Other times, they impact large-scale transformation programs, which can wreak havoc on your organisation’s reputation.
So being able to manage through a crisis is part and parcel of your transformation.
Let me illustrate this using a personal example. A number of years ago when I was chief information officer at CUA, we were going live with a major core, online and mobile banking system change. As this was being planned with the CEO, he suggested we go live in ‘crisis mode.’
Thus the entire management team and transformation team would meet four times a day for two weeks until everything had settled down. The crisis meetings included weekends, and I can assure you that having clear communications and the right people in the room made a huge difference in addressing the evolving situation and scenarios we faced. This is a great attribute to bring to your enterprise.
1. Plan for the worst
It is essential to start with a comprehensive plan. You will always find that there are some unforeseen elements that crop up in a crisis, but building this crisis management plan means that there is a common understanding and framework that can be applied.
You can make always adjustments to a plan when there is a new variety or emergence of a new threat, but not having a starting position is reckless.
The plan has to address common crisis situations and provide confidence to management, staff and customers that when an incident occurs, it can be invoked. It is usual to feel overwhelmed in a time of crisis; so take the time in advance to anticipate how to react and how to communicate to various stakeholders.
Then ‘test’ this plan to ensure that it actually works as expected. Recently, when I was working in Japan, we created a crisis simulation plan to model what happens if there’s a missile attack from North Korea.
This was not a scenario that was in the current crisis management plan for this large enterprise and the executives had to improvise in reacting to the situation. The plan also had provisions for what would need to be done following a cyber security attack.
2. Get the facts
The real reasons for a crisis are not always apparent. A crisis team has to be activated and the plan invoked to tackle the event. In the plan, the team has already identified the ideal candidates with business responsibility and profile to avoid any delays in decision making.
The crisis leader – who in some cases is you – will need get the broader team to investigate the specific issue and determine root cause. During a crisis, it’s often the case that symptoms are reported first that are not directly related to the actual issue.
This will usually require a cross-functional team that have the requisite knowledge and competencies. Be sure to reinforce that no communications both internal and especially external are to be made without your express approval. This is when you need to have the media trained staff to take the ‘stand-by’ statements and start to draft potential responses.
3. Establish a war room
This is a location that you have already established. When a crisis occurs, you will always want your team to be co-located at the same site. The war room will need to have appropriate technology (telephones, WiFi, printers, projectors) along with white boards and butcher paper.
Ideally, you have the crisis management plan projected onto a wall for all to see and a second projector that shows the log of events. This level of transparency will be critical to enable all the team to track progress and ensure that information is widely shared.
The war room should remain open around-the-clock during the crisis and you will need to ensure that there are no conflicting bookings for this room.
4. Set the tone
The team is looking to you at this time of crisis. Remaining calm but challenging is the right tone that you have to adopt. Now is not the time to start looking for scapegoats or for blame.
People working on the issue may also be the same staff who are responsible for the issue that occurred. To have them worried about repercussions is not going to create the appropriate mood. Remember that you set the tone with IT and with the business. The business is going to be sensitive and edgy; perhaps looking to blame others for the outage.
Be calm and confident but at the same time concerned and communicative. I’ve seen examples of outages at banks where multiple teams are involved in a major crisis that has brought down core and online banking systems.
In this cases, it was not certain where the issue began and there was ‘blame shifting’ occurring. Randomly laying blame is not going to help the situation and you need both teams to work together to resolve the issue.
5. Communicate often
You set the tone by communicating at all times – this is your key role. Create a checklist for notifying management. As soon as you become aware of a major crisis, notify management. There is usually a protocol as to how communications are made and through what medium.
The trick is that everybody wants to know root cause and more details than are available. Accordingly the CEO, the entire c-suite, and board members may all need to be notified. A major crisis has high profile and potential negative external impact so this requires your absolute attention.
6. Restore services
Ok, so the magic happens and your crisis is averted. In some cases, there are good self-healing mechanisms and in others, a reboot of all your systems might do the trick. The restoration of services is when the team and you, as CIO, can all breathe out.
But don’t forget to breathe in again because your work is not yet done. There is comprehensive checking and validation required to understand the real impacts, and what other unintended issues have been created by this crisis. The impact on IT operations is now over but business operations issues are mounting up and it’s time to turn your attention to finalising the ‘root cause’ analysis.
7. Do an ‘after action review’
The classic ‘after action review’ as used by the military is required to answer:
- What was supposed to happen?
- What actually happened?
- Why did this occur?
You should insist that within 10 days of the crisis that the team will convene to review any lessons learned. This is often a very painful exercise and needs to be completed and then reported back through various management governance processes.
It is often ideal to have a neutral third party to run this exercise, which can potentially be quite destructive. This last stage can involve regulators and external customers/partners so don’t relax yet.
In truth, it is impossible to avoid a major IT crisis. Like I said upfront, transformational activities will create opportunities for this to occur. We have to embrace and welcome these shifts as they can drive positive change into the enterprise.
Good luck.
Join the CIO Australia group on LinkedIn. The group is open to CIOs, IT Directors, COOs, CTOs and senior IT managers.