How many lines of code did you write this quarter? By how many hours (or days or weeks) were you behind in delivering your latest project? Was it over or under budget, and if so, by how much? How quickly were you able to get your last program to testing?
For many years, those were the metrics by which IT employees were judged when their performance reviews rolled around.
These days, IT managers acknowledge that writing fewer lines of code can be more efficient, that longer timelines and bigger budgets can be acceptable if that means getting a project done right the first time, and that moving a program to testing quickly isn't necessarily a sign of a job well done.
What's more, as IT shifts from being a service provider to a strategic business partner, managers are looking for people with different skills than IT workers were expected to have in the past.
"What we value is high performance of our services and strong execution on our projects," says Joel Jacobs, vice president and CIO at The Mitre Corp., a not-for-profit science and engineering organization.
Jacobs is clear about his expectations, and, like other IT leaders, he wants to make sure his IT staffers are meeting those objectives. But what's the best way to determine if that's the case? Gone are the days when technologists were measured on straightforward, quantifiable metrics that reflected a heads-down development focus. But what should replace those metrics?
"We have to decide what we measure, and that's true in any part of the organization, not just IT," Jacobs says.
Mitre is overhauling the way it evaluates its employees, including those in IT. The organization now evaluates the impact of what people produce alongside more traditional measures such as meeting deadlines and staying under budget. "What we're trying to measure is performance against our plans for existing services and new capabilities," Jacobs explains.
Wanted: Better IT Assessment
Throughout IT, managers like Jacobs are wrestling with the question of how to best evaluate tech workers in the face of evolving responsibilities.
Consider the findings of an October 2012 survey of 3,500 IT professionals and leaders by TEKsystems, an IT staffing and consulting firm: Only about half of the respondents in either group said that their supervisors are great at performance management.
Performance management "is definitely challenging, and that's why you see it not happening," says TEKsystems marketing director Rachel Russell.
But any organization that wants to maximize the value of its workers will want to make sure assessments do, indeed, get done, Russell says. Performance management, of which evaluation is an important part, is "basically how an organization leverages people to achieve what it's trying to achieve," she says.
"If you have a set of goals and if you only ever talk about them formally once a year, as many companies do, you're not going to be set up for success," Russell explains. "The more you talk about them, the more ingrained they are with your staff and managers, and the more likely you are to achieve those goals."
In the TEKsystems survey, 83% of the IT professionals polled said that formal feedback is either extremely important or important to their success. About half of them said they receive formal feedback once or twice a year, and 37% of them reported that they get formal feedback on at least a quarterly basis.
Respondents value informal feedback even more: 93% of the IT professionals said that regular, high-quality informal feedback is important to their success.
Yet, only 14% of the IT professionals and just 12% of the IT leaders said that informal feedback is given when performance deviates from expectations. Moreover, 15% of the IT professionals and 8% of the IT leaders said that informal feedback isn't given at all.
Assessing Value to the Business
The employee evaluation process should start with clearly defined goals and an understanding how each IT worker helps reach those goals, says Dan Roberts, president of Ouellette & Associates Consulting. Those goals should shape the metrics that are used to evaluate technologists, he says.
"The skill sets and competencies are changing, so we have to assess people in new ways, provide new frameworks, to see where they are with these new competencies," Roberts says. "More and more, they're being measured around relationships [and] how they're perceived by the business."
One of Roberts' clients, the IT department in a large pharmaceutical company, has a report card that takes into account business-unit satisfaction levels -- a metric is then used to assess how well IT teams are delivering on stated goals. "The metrics are changing from the heads-down technology metrics," he says. "Now it's on business results and the business value they're bringing to the organization."
Stephen Olive, CIO at Philips Healthcare, says his 500-member IT department sees itself as a strategic partner, so he's judging success on what IT is doing for the business and what value it delivers.
While Olive still considers budgets and schedules when thinking about employee performance, determining whether IT workers meet those standards now plays a smaller role in the way he evaluates people. Like other managers, he now looks for and measures more esoteric qualities, such as teamwork and cooperative behavior, as well as the employee's understanding of business goals and how to go about achieving them.
By the Numbers
Importance of Informal Feedback
How important is regular, high-quality, informal feedback to your success in your role?
Extremely important: 49%
Important: 44%
Neutral: 6%
Unimportant/extremely unimportant: 1%
Source: TEKsystems survey of 3,500 North American IT professionals, October 2012
Olive points to his company's rollout of Salesforce.com, which recently went live to 750 users. The success of that IT project is being measured by managers' and users' testimonials on whether the application helps them do their jobs better or more efficiently. "You share that with your IT employees, so they know we're making an impact on the business," he says.
While Philips Healthcare does have a formal performance development process -- which includes an annual goal-setting session and a formal midyear review where managers meet with their workers and provide written feedback -- the company's ongoing, informal performance-review mechanisms are more likely to capture the business value Olive is after.
Managers gather feedback on their employees from "assessors" -- key people surrounding individual IT workers. With that feedback, managers can better evaluate an employee's strengths and weaknesses, says Jean Scire, senior director of information systems at Philips, who reports to Olive.
The ongoing, informal evaluation process allows managers to identify staffers who aren't performing up to snuff, and either offer them coaching or move them into positions that better fit their strengths. It also allows management to recognize outstanding work through both feedback and formal recognition programs -- both of which are great for morale, Olive says.
Above all, the process ensures that IT as a department is serving the business. "With this business alignment, how you evaluate [performance] is really about what you're doing for the business and what value do you add," he says.
Walking the Walk
CIOs and IT management consultants stress that this evolving approach to performance monitoring and evaluation requires a lot from managers. They must interact with employees and their business unit colleagues often enough and openly enough to hear and see when systems are making the desired improvements, when collaborations are happening as they should, and when projects are progressing as intended -- and when they're not.
That means taking the time to talk to people about a job well done, soliciting nominations for formal recognition programs, or working one-on-one with an employee who's missing milestones, Scire says. "You have to know your employees well enough to have those open communications," she observes.
Jacobs agrees. At Mitre, "we've been increasingly pushing our [managers] toward 'meaningful conversations' rather than compliance to a checklist," he says. "But even though it's a conversation, it's not just chitchat. There's a method to it -- a method that aligns to what our company is trying to achieve."
Pratt is a Computerworld contributing writer in Waltham, Mass. You can contact her at marykpratt@verizon.net.
This version of this story was originally published in Computerworld's print edition. It was adapted from an article that appeared earlier on Computerworld.com.
Join the CIO Australia group on LinkedIn. The group is open to CIOs, IT Directors, COOs, CTOs and senior IT managers.