This special crisis management report from CIO India shares the experiences of 12 CIOs who recall unique crises — including government policy changes, industrial action, floods and power blowouts — that interrupted their businesses and tested their IT organizations. Even without contingency plans in place, these businesses had to find a way to grapple with these crises to keep their companies on course.
While many companies worldwide now hire IT talent from India, it's interesting to note that the prevalence of IT systems in Indian business is still relatively new. As an IT leader from the manufacturing sector notes: "[For a long time], situations in business in India were not so dependant on IT that it would lead to a shutdown or something of that sort." It's only in the past eight years that IT has revolutionized the way business functions - through thick and thin. A few CIOs believe that a distinction needs to be made between "crisis in the IT organization" and "business contingency that requires IT's support". Be that as it may, information unites business and the IT organization, and helps enable CIOs and business management to recover from crises.
Check out how 12 CIOs brought their house back to order when faced with difficult, unexpected circumstances.
An Electrical Reversal Wiped Out Your Data Centre - and You Didn't Know When It Could Be Fixed
Tamal Chakravorty, former head of IT infrastructure at PepsiCo, now CIO at Ericsson India
A while back, Tamal Chakravorty, then the head of IT infrastructure at PepsiCo, was headed for a bath when he got a call from his office. One of his servers had shut down, but that wasn't going to stop him from taking a shower.
"My first reaction wasn't of worry," says Chakravorty. "It's normal to have a server fail (in our line of work). I certainly didn't jump out of my seat. I just wanted my hardware vendor, HP, to fix it. If it was a software issue, then the software folks would fix it. I thought someone somewhere would," he recalls.
That's when the other servers started to drop like nine pins.
The wipe-out couldn't have come at a worse time. "It was half-yearly reporting. I remember because an England-India cricket match was on." Chakravorty knew it was time to start his one-hour commute to the office. He was getting phone updates: all that his staff could determine was that every rack in the server room had electricity running through it. "That's when I started to worry . . . The engineers could not even touch the electrically-charged racks, which meant that no one could assess the extent of the damage."
The situation fit one of his two definitions of a crisis. "A crisis is when business has shut down and there isn't a time-horizon to rectify it." Unfortunately, it also fit his second definition. "My other definition of a crisis is when business is dying to do something but because IT has faulted or systems don't work, they can't."
And business needed his systems urgently. Because it was half-yearly reporting time, the entire finance team, many of whom had been flown in for the meeting, was waiting. "Management was screaming, and I couldn't tell them when I could have the systems back up because I didn't know what the problem was."
Meanwhile, the electricity was frying his systems. "It started with my SAP server, then my e-mail server, my domain server, then my intranet and Web site servers. And, it went on until all 25 servers were out," he says.
It was four to five hours before electrical engineers could stop the electricity from destroying the servers. "But, my problems started only after they had done their job. The SAP server was destroyed, and my team had no idea about SAP," he explains. The IT team had to get HP and someone from Hyderabad on the phone to walk them through the repairs to PepsiCo's SAP system.
It took them nine hours, with 18 of the company's top suits waiting to get the systems back up. The losses came in two forms: delays in reporting and wasted time. More tangibly, the inability to create invoices and challans halted delivery of innumerable cases of Pepsi, "which meant that many more bottles of Coke were being sold". He estimates a loss of at least Rs 10 lakh.
"At that time, I couldn't have foreseen such a situation. But, now I can," he says wryly. "We learnt a lot about SAP, for sure. It also got the idea of starting a disaster recovery site going and we created a Bible to handle crises," he adds on a more serious note.
If the same were to happen to him today, Chakravorty says the damage would be more limited. Experience has given him eyes. "We have predictability now."
As told to Sunil Shah
Join the CIO Australia group on LinkedIn. The group is open to CIOs, IT Directors, COOs, CTOs and senior IT managers.