Risky Business
Experts will tell you that statistical risk analysis is as essential to real portfolio management as a processor is to a computer. Without it, portfolio management is simply a way to organise the view of projects that will almost certainly fail. CIOs who are serious about portfolio management need to be serious about statistical risk management. (For more on portfolio management, see "Portfolio Management: How to Do It Right", CIO June.)
"If you don't succeed with risk management, you won't succeed with project portfolio management," says Raytheon CIO Rebecca Rhoads, who credits risk management with lowering her project failure rate and helping Raytheon IT achieve its cost-performance targets. Rhoads is ahead of the curve, but despite her engineering background, she has yet to apply the kind of sophisticated statistical analysis that Aspinall uses for his volcano.
Robert Sanchez, senior vice president and CIO of Ryder, credits risk analysis with bringing order to his company's decision-making process for projects. He would welcome statistical analysis, but he's not there yet. "Have we really embraced it completely and understood it in all of its detail?" Sanchez asks rhetorically. "No, we haven't. But we will."
CIOs should become familiar with two statistical tools. They are the colourfully named workhorses of risk analysis: Monte Carlo simulation and decision tree analysis. Probabilities figure heavily into both, which means that risk has to be quantified. CIOs must draw their own line between the Exclusion Zone, where it's too risky to venture, and the beaches, rain forests and coconut groves, where the living is easy and the threats are manageable.
The Trap of Common Sense
Even a simple task like choosing to drive to work requires a risk assessment, although not a computational one; you can do shorthand probability in your head. Though the cost of being wrong is high, the risk is relatively low (a 5 per cent probability of being seriously hurt in a car accident) and easily mitigated by wearing a seat belt.
This sort of informal risk analysis can sometimes be useful. Steve Snodgrass, CIO of construction materials supplier Granite Rock, has the misfortune of managing IT for a company that literally straddles the San Andreas Fault. Snodgrass doesn't need statistics to tell him that it would be a bad idea to do nothing to mitigate the possibility that a quake will take out his critical applications. So he outsources his applications' backup far from the fault line.
However, CIOs often use this kind of common sense reasoning as a way to avoid doing real risk analysis, say Tom DeMarco and Timothy Lister, authors of Waltzing with Bears: Managing Risk on Software Projects, a primer on statistical risk analysis for IT. "It's been very frustrating to see a best practice like statistical analysis shunned in IT," says Lister. "It seems there's this enormously strong cultural pull in IT to avoid looking at the downside."
In lieu of choosing projects based on acceptable risk, Ryder's Sanchez says, IT often uses what he calls the moral argument, in which the greatest risk lies in not doing the project. Therefore, the risk is mitigated by doing the project. This reasoning was particularly valid during the boom years when there was a palpable fear of getting left behind technologically. But it was never called risk analysis. "I came into IT and was never really comfortable with the moral argument," says Sanchez, whose background is in engineering and finance. "I was looking at it thinking: We analyse the risk of building a new office, but we don't on an ERP system that costs the same amount."
Join the CIO Australia group on LinkedIn. The group is open to CIOs, IT Directors, COOs, CTOs and senior IT managers.