Why Benchmarking Cloud vs. Current IT Costs is So Hard

A couple of weeks ago I was asked to moderate an HP-sponsored meeting on the subject of virtualization. Predictably, most of the discussion (attended by press and vendors including Citrix, Microsoft, Red Hat, and VMware) focused on cloud computing. It was a pretty lively session, but what I want to address here is an HP product portfolio called "IT Financial Management" that was discussed, along with its implications for cloud computing. As you might guess, the product focuses on financial analysis of IT operations, which is extremely relevant to the adoption of cloud computing.

Naturally, one of the topics raised during the panel was the potential cost benefits of cloud computing.

And, as one might expect, there was a lot of ambiguity about the putative financial benefits of cloud computing. While there was a lot of nodding about the undoubted benefits, no one offered concrete examples. I will say, however, that we've seen significant TCO savings in the projects we've done, but, as moderator, it wasn't appropriate to discuss them.

The paucity of examples is troubling. Most of the organizations that have implemented cloud solutions have done so with an intuitive belief that inexpensive, pay-by-the-use computing must be less expensive than the current asset-heavy, low-automation environment characteristic of most IT organizations. But they've forged ahead, mostly, without real proof.

The reason for this, however, lies not with the cloud providers. Their pricing is, on the whole, quite transparent. Amazon, for example, displays its pricing on its website and even features a nifty online calculator that may be used to figure out how much a given application topology and usage will cost.

The problem lies, instead, on the other side of the comparison ledger-the cost of the incumbent solution placed within an internal data center. Most IT organizations cost assessment is extremely coarse-grained. Typical is a cost assignment that imposes a per-server charge and then assigns a general overhead cost for IT headcount and general costs based on the using organization's overall proportion of the corporate cost structure. Put another way, there is little ability to assess the actual marginal cost of an individual server; instead, a server comes with a fee plus the knowledge that a rather large surcharge will be delivered sometime in the future.

One can debate the reasons why this opaque cost methodology exists. My view is that IT used to be a very small part of an overall corporate budget and it didn't seem worth trying to perform fine-grained assessment on what was, in effect, rounding error. In the past 10 or 15 years IT expenditures have gone up precipitously, turning a small issue into a much larger one-but one still without adequate tools in place to address it.

At this event HP discussed a new (or pretty new, anyway) product called "IT Financial Management" (certainly not the most compelling of names, one must admit). It is designed to enable fine-grained cost assignment through an examination of existing expenses and assignment of them to a structured chart of accounts which can then be applied to individual applications. In short, the product (and service, since shoveling through existing costs and assigning them to appropriate categories and ultimately to individual applications is labor-intensive) helps show the "current cost" side of the today-vs-cloud TCO comparison that is such a hot topic.

Intrigued, I asked about the product after the event. I got the impression that the product is new enough that there are not a lot of concrete results from customers, but that there is a lot of interest in it. The HP executive who described it, Ramin Sayar, used system admin span as an example of cost comparison. He noted that cloud providers are able to obtain spans of one admin to 10,000 machines, perhaps, while internal IT groups run ratios of around one admin to 70 machines.

I believe that the data a product like this turns up will be in high demand over the next couple of years, as the battleground around cloud computing shifts from "what is it? I don't trust it" to "how much benefit will I realize if I use cloud computing?" A key metric for this latter question will be TCO comparison-and a discussion that leaves out half the comparison because the data is hard to get won't be acceptable in the future.

And what might be turned up if such a comparison were performed?

A year ago Bechtel, the giant construction firm, benchmarked its costs for a number of common IT resources against cloud providers. Remember, this was Bechtel, a very large and financially well-off company which one would assume would be a very capable IT user, with as efficient an operation as could be obtained.

In fact, Bechtel found that its cost structure was wildly higher than the best-of-breed cloud providers. Here are some of the numbers from Bechtel's benchmarking exercise:

1.Internet traffic cost Bechtel $US500/MB; in the cloud that cost $US10-$US15/MB (YouTube.)

2.With regards to sys admins Bechtel had one admin per 100 servers; in the cloud, one admin per 17,000 servers (Google.)

3.Storage cost Bechtel $US3.75/GB/month; in the cloud that cost $US.10/GB/month (Amazon).

As a result of this benchmark exercise, Bechtel re-engineered its IT. It built three super-efficient data centers with high utilization rates. It raised storage utilization rates from single digits to 70 per cent. Likewise, its server utilization rates grew to 70 per cent. And it stopped using public Internet providers and installed its own fiber optic connectivity to Internet peering points.

The investment Bechtel was able to make, however, won't be possible for most organizations. They will be stuck with their old, high-cost business processes, unable to fund new data centers and bespoke fiber optic connectivity. For these organizations, transparent cost comparisons, once available, will force a discussion of how to begin the shift to lower-cost methods of IT.

Offhand dismissal of cloud computing providers as "must be more expensive, after all, they're trying to make a profit" will be a thing of the past. Despite each of the comparison providers in the Bechtel example being profit-seeking organizations, they were able to operate far less expensively because they've been built from the ground up to be efficient.

I expect products/services like IT Financial Management will be in high demand over the next 18 - 24 months as organizations seek clarity about their IT cost structures and begin serious work on squeezing out IT costs. We'll see restructuring, not trimming. And we'll see a lot more use of cloud computing.

Bernard Golden is CEO of consulting firm HyperStratus, which specializes in virtualization, cloud computing and related issues. He is also the author of "Virtualization for Dummies," the best-selling book on virtualization to date.