ROIowa

By centralising its state IT budget process, establishing a scoring method and instituting value metrics, Iowa has become a national model for gauging project ROI.

In Iowa state government, as in most organisations, there was a time when few outside of the IT department gave technology much thought. If an agency needed money for an IT project, officials slipped it into the budget. If a legislator asked what the money was for, a good sales pitch would close the deal and a poor one might mean no funding at all.

But along came Y2K, and officials in Iowa government started to think about the $US22.5 million needed to fix the bug, and how best to spend it. To ensure taxpayers got their money’s worth, Iowa’s Year 2000 Project Office audited agencies’ progress toward compliance and reviewed every expenditure before reimbursing them from a central fund. At the end of the project, an estimated $US3 million was still left, unspent, in the Y2K fund. Meanwhile, state leaders decided to examine the value they were getting from other IT expenditures. “I was a legislator for six years, and we had no clue,” says Governor Tom Vilsack, a Democrat who was first elected in 1998.

And so the idea for the Iowa Return on Investment Program was born. Today, three years after it was launched, the program has become a national model for documenting value and prioritising IT investments in the public sector. Last year, the National Association of State Chief Information Officers named the program the winner in its competition for the best state IT management initiative. The Iowa program helps state leaders identify the most pressing new IT investments at a time when competition for funds is fierce. In addition, it has saved taxpayers more than $US5 million since its inception by enabling officials to combine duplicative projects and flag unnecessary expenditures.

Value Model

As one of his early initiatives, Vilsack, now in his second term, began promoting performance-based government, an approach to public sector management that focuses on measuring the results of government programs. He realised that the principles used to validate the Y2K expenditures could be applied to any IT project. “I think [Vilsack] saw there was a lot of value in independently verifying and validating,” says Paul Carlson, director of the IT department’s Enterprise Quality Assurance Office, which is in charge of the ROI Program.

When Carlson and then-CIO Richard Varn devised the new IT value methodology in 1999 using various ROI research sources, they incorporated the two main features of Y2K management. First, new investments are paid for primarily from a pot of money called the Pooled Technology Account, which is appropriated by the legislature and controlled by the IT department. Pooling the funds makes budget oversight easier and fights duplication. Second, the IT department reimburses agencies for expenses from this fund only after verifying that they’re necessary. If an agency’s expenditures aren’t in line with the project schedule, it’s a red flag for auditors that the project could be in trouble. The Enterprise Quality Assurance Office will consider conducting post-implementation audits next year. But Carlson says the details concerning how projects are chosen for audits and how those audits will be conducted are up in the air, pending decisions by John Gillispie, the executive director of the Iowa Communications Network, who is expected to manage the IT department after a proposed agency reorganisation is complete.

To support spending decisions, agency managers have to document the expected costs and benefits according to a standard set of factors. The maximum score for each factor ranges from five to 15 points, for a maximum total score of 100 points. In addition, they must specify metrics related to those factors to determine the project’s success (see “How Iowa Scores Investments”, 64).

The ROI Program saved taxpayers more than $US2.5 million a year during 2001 and 2002, about 16 per cent of what the state spent overall on new IT projects during that period. Because the scoring system is weighted toward projects that comply with standards and directly benefit citizens, the program promotes initiatives like information security that are critical but don’t have vocal political constituencies. By encouraging cooperation between agencies, it has also transcended decades-old bureaucratic fiefdoms. Agency managers “started to see that joint projects and collaborative projects were the ones that were getting done”, observes Varn.

The program hasn’t ended political wrangling over the IT budget, however. Varn was asked to resign in December (along with several other cabinet directors), and his resignation was accepted partly because he and Vilsack did not agree on how big the technology budget should be (Varn wanted more money and a more integrated IT organisation). Agencies still battle with the legislature for approval of specific projects. But it’s easier for officials to make informed decisions. “We’re able to make decisions about what should be funded based on data, rather than based on politics,” says Debbie O’Leary, the acting director of the IT department.

Everyone Knows the Score

Like many processes in government, IT investments start with a form. A Return on Investment Program Funding Application must be completed for any expenditure over $US100,000, Pooled Technology Account requests, any request for money from a source outside of an agency’s annual appropriation, and any expenditure that’s not routine. Waivers that let agencies skip the ROI process are allowed for some large, routine expenses but are quietly discouraged. During the fiscal year 2003 budget approval process last year, agencies asked for 17 waivers and were granted only six. They got the message and requested only four waivers in their fiscal year 2004 proposals.

To complete the form, agencies describe the project according to 10 criteria that are used to define value. “In the private sector, what you decide to invest in is substantially based on a cost-benefit analysis,” says Carlson, who culled the criteria from value methodologies used by both public- and private-sector IT organisations. “Because we’re in the public sector, we have to take into account more than just a dollar.”

An information technology council — including the Iowa CIO, the CIOs from major state agencies, representatives from the state judicial and legislative branches, and private citizens — scores each application and ranks them. Projects can get up to 15 points for fulfilling a legal mandate or complying with enterprise technology standards. Improving customer service may garner another 15 points. The bulk of the remaining score takes into account such factors as ROI and risk, whether the project will streamline business processes, whether it affects multiple agencies and, for multiyear projects, the success of earlier phases. The application includes a methodology for assessing benefits that — as is the case with many public-sector initiatives — aren’t easily quantifiable.

Projects that protect citizens, like security, and initiatives that improve access to government services, like electronic tax administration, score consistently high.

Save a Penny, Save Millions

Iowa’s statewide budget for new IT projects has recently averaged about $US15.7 million annually. The available funds typically cover about 24 per cent of the total money Iowa agencies request for new investments — about $US76 million next year. O’Leary has to find a way to stretch every dollar. One way she’ll do it is to roll similar projects into a single implementation. Another will be to shave costs once projects are under way.

The ROI Program requires agencies to detail their technology requirements and functional needs, which enforces standards, but it also helps officials identify duplicative expenditures. During a review of funding requests for 2001, Varn and his staff noticed that several agencies were proposing to build pieces of an ERP system, such as electronic procurement and human resources management. Varn suggested the state could deploy a single ERP system that agencies could share for less money. The project, which has an estimated cost of $US9.6 million, could easily cost many times more, says Varn, if agencies were allowed to go it alone. Meanwhile, agencies are finding ways to collaborate on their own. Some requests from agencies for money to fund compliance with the federal Health Insurance Portability and Accountability Act, which took effect last month, came already bundled.

Once a project is funded, the state saves money by scrutinising agencies’ expenses. Agencies have to submit their purchase orders and invoices to the Enterprise Quality Assurance Office for approval before they can be reimbursed. As officials discovered during the Y2K project, purchases may cost less than agencies estimated they would when they were first proposed.

New Rigour

After four years, agencies have adopted a more rigorous approach to assessing value that has brought agency technologists, finance officials and business line managers together. Business analysts in IT departments typically don’t understand finance well enough to complete the ROI analysis, so agencies need a cross-functional team to write a successful application.

“It gave us structure to say to our customers — because it’s not an IT responsibility — that you must provide answers to these questions,” says Rich Jacobs, administrator of technology and information management in the Iowa Department of Revenue and Finance. At first, agency executives were all too willing to identify the projects that they wanted and let IT handle the paperwork. That changed once officials realised that legislators were responding to the list of priorities presented to them from the ROI Program. “We felt like we didn’t just have to ask for it; we had to demonstrate that we could accomplish it,” says Jacobs.

Still, many agencies resented having to cough up so much information because they didn’t understand why all of it was relevant. Agency officials now know how the data is used, but the ill-feelings haven’t completely dissipated. The absence, to date, of full audits is one sore point, according to Jacobs. “If you say you’re going to audit, you have to be able to honour that commitment, or [providing the necessary data] becomes a waste of people’s time. No one has excess staff to start up an audit program,” he says, but the long-term credibility of the ROI program depends on it.

Getting funding through the ROI Program is, however, better than doing business the old-fashioned way, when getting a project funded had less to do with its merits than “networking and your ability to monitor budget requests”, Jacobs says. Lawmakers still choose some projects for political reasons. Carlson says that in such cases, the methodology helps improve projects by encouraging officials to comply with state IT standards. Varn thinks the program also positions agencies better for success once they get funded because they have to document technical and business risks. Before the ROI Program, he says, “a lot of people didn’t do project management, and they didn’t know what the technology was going to be. [They said] just give me money, and I’ll do good work with it.”

Jacobs confirms the benefit. The revenue and finance agency had started several small electronic tax administration initiatives in the mid-90s that were largely funded through its operating budget. But as the state tightened its belt during the past few years, IT wasn’t a priority with the legislature. To move forward with online filing — already a reality in other states — and some related initiatives in 2001, the department applied for a grant from the Pooled Technology Fund. The process forced officials to look at the project holistically; they identified six phases that would be completed over several years.

Not long after the ROI Program was established, the legislature enshrined it in law by enacting the Accountable Government Act, which requires the Iowa Department of Management to establish methodologies for evaluating all state investments, not just IT projects. “One of the concerns I have is that people don’t see the relevance of government in their lives day to day,” says Vilsack. “As I went out on the stump, I wanted to be able to tell people that for your investment in X number of tax dollars, this is what you get.”

ORGANISATION: Iowa Information Technology Department

Principal Business: State government

Headquarters: Des Moines, Iowa

URL: www.state.ia.us/ government/its

VALUE PROPOSITION: To establish a centralised way to better track and measure the ROI of IT projects, enabling the state to rein in project costs and eliminate duplication.

THE PLAYERS

Tom Vilsack - Governor

Paul Carlson - Director of the IT Department’s Enterprise Quality Assurance Office

Richard Varn - Former CIO

Debbie O’Leary - Acting Director of the IT Department

Rich Jacobs - Administrator of Technology and Information Management, Iowa Department of Revenue and Finance

CASE ANALYST Douglas Hubbard - President Hubbard Decision Research

Expert Analysis

An Effective Measure

BY DOUGLAS HUBBARD

Iowa’s mission to implement a formal approach to portfolio management has eliminated redundancies, justified projects without strong political constituencies, and provided active project and budget oversight. Contrary to some stereotypes, the private sector can learn a lot from state governments. Iowa deserves the National Association of State Chief Information Officers’ accolades.

However, I am sceptical that the scoring system is helping Iowa choose the best projects. Research over the years has shown that subjective scoring methods are not as effective as other valuation methods. The Enterprise Quality Assurance Office should consider methods that can assign dollar values to the criteria it uses to measure a project’s worthiness. While it does calculate financial impacts for such things as transaction savings, it could represent any other value this way as well. I routinely quantify the value of intangibles in IT organisations, including those in government agencies.

I don’t believe that scoring is an effective method for quantifying risk or the value of customer service. If the Quality Assurance Office officials were to replace the language of scores with the language of actuaries, CFOs and economists, I believe they would see yet another significant improvement in IT decision making.

How Iowa Scores Investments

Iowa scores proposed projects based on 10 criteria used to determine value. The highest scoring projects get priority for funding. Below is an explanation of the criteria, the maximum points each one can receive and examples of how the top-ranked project proposed for funding next year was scored.

Criteria and Point System

Statutory requirement (15 points total): Project is required by law or regulation, or it is needed to comply with state IT standards. No points unless the project fulfils a mandate.

Improves customer service (15 points): The bigger the improvement, the more points.

Impacts citizens (10 points): More points for greater impact.

Re-engineers government processes (10 points): The most points for “significant” use of IT to revamp business processes.

Project participants (10 points): The most points for projects needed by every state agency.

Risk (10 points): More points for projects with low technical and business risk and high chance of success.

Experience and past performance (5 points): More points to agencies that are better at achieving project objectives.

Funding requirements (10 points): More points to projects at advanced stages of implementation.

Additional funding sources (5 points): More points to projects for which agencies will share costs with the IT department.

Financial ROI (10 points): The higher the ROI, the more points.

A Project Scorecard

This is the budgeting request that received the highest score for FY04. The Electronic Tax Administration filled out the necessary Return on Investment Program Funding Application, and it was scored and ranked by an information technology council, a group that includes the Iowa CIO, the CIOs from major state agencies, representatives from the state judicial and legislative branches, and private citizens.

Electronic Tax Administration

FY04 request: $US521,000

CRITERIA POINTS

Statutory requirement 11.8

Improves customer service 13.5

Impacts citizens 9.4

Re-engineers government processes 9.0

Project participants 7.1

Risk 9.3

Experience and past performance 4.7

Funding requirements 8.6

Additional funding sources 4.9

Financial ROI 9.8

TOTAL 88.1