So Who Is Happy?
The big question is, if metrics exist and they are valid, then who is using them?
Let's be frank. Many of the CIOs we spoke to for this article had either never heard of satisfaction metrics, or were not even outsourcing at the moment, and therefore ruled themselves out of any comment. Even the vendors had trouble coming up with working case studies.
The CIO of a large state-based public sector agency said that during deliberations for renegotiated outsourcing contracts, satisfaction metrics were considered, with an eye to hopefully playing a part in supplier remuneration. However, they had trouble finding a working model, and the idea was ultimately scuttled.
EDS claims to have 1500 clients across the world using its dashboard as standard practice. But whatever the number, the question remains whether intangible "happiness" metrics are a decisive measure of outsourcing relationships.
Glenn Hickey, CIO of Zurich Financial Services, says "intangible measures are vital when looking at outsourcing". He says that protecting his company's intellectual capital is paramount to the organization, so trust and reliability are important measures of a potential outsourcing partner. But then he adds a twist. "We don't do a lot of outsourcing, primarily because we have difficulty finding a cost-effective solution . . . The cost benefit is hard to find."
Hickey suggests that outsourcers are finding the market very difficult at the moment. "Most IT departments are run as a very tight ship, and there's little room for profit for outsourcers."
Which might be why vendors emphasize the intangible benefits of outsourcing, the value-add, the relationship-builders. Gartner suggests that cost is not even in the top five reasons for outsourcing.
Hickey disagrees. "People are outsourcing based on cost," he says.
Getting a Handle on Intangibles
There is a view, often held by CFOs and others used to dealing with numbers and calculations on a regular basis, that if it cannot be measured it is not worth considering. And if it is intangible, then by nature it cannot be measured. This means that whatever the system and its heritage, any measure that smacks of warm and fuzzy qualities is akin to guesswork, and should be ignored.
Jack Keen, founder and president of IT consultancy The Deciding Factor, in a column published in this magazine last November ("Wanted: Metric Sceptics") criticizes this very attitude and "the misuse of numbers . . . when management refuses to accept guesstimates (informed estimates) as legitimate metrics.
"This problem occurs whenever someone says: 'If you can't prove it, we won't use it.' In this mistaken world view, only hard metrics from factual situations are valid. The clear message is that guesses don't count.
"The reality is that informed guesses are what makes the world work. If business investment strategies were 100 percent fact-based, we wouldn't need high-price executives to guide the company. Computers could probably do the job via fancy calculations. The whole discipline of risk management is ultimately grounded in probabilities, which themselves are estimates. The medical profession begins and ends with the reliability of doctor diagnoses, none of which are 100 percent correct. But they get enough right that we don't ban their profession. A similar situation exists with our economy's reliance on meteorologists. They don't hit the mark 100 percent of the time. But their track record is good enough for planes to fly safely through troubling storms, not to mention properly attiring our families for school and work.
"The key to accepting informed guesses as valid metrics is to make sure they: (a) come from knowledgeable, experienced and dependable people; and (b) are accompanied by clear explanations of the guesstimate's premises, assumptions and logic."
In an article published last year on the Technology Evaluation Web site ("If software is a commodity . . . then what?"), Olin Thompson, a principal of US-based Process ERP Partners, talks about the criteria involved in selecting a software partner when most software is commoditized, and his views apply as much to outsourcers in an often commoditized environment as they do to software.
"[In a mature market and less so in a newer market] we find that most products yield the same results and the differences are in the 'How' they do it instead of the 'What' they do. The relatively small percent of features that are different (usually less than 20 percent) are often in the 'nice to have' category, or are not features that most end users need."
Join the CIO Australia group on LinkedIn. The group is open to CIOs, IT Directors, COOs, CTOs and senior IT managers.