CIOs who accept favours in return for saying good things about vendors are putting themselves and their careers in jeopardy. And so are the CIOs who listen to them.
- Learn how vendors court and capitalise on customer references.
- Understand why vendor references are not as reliable or informative as you might think.
- Read how becoming a reference can jeopardise your career.
Like many CIOs, Steve Pickett is happy to serve as a reference for his favourite vendors. If he spends a couple of hours talking to a prospective customer, the vendor might not bill his company the next time he needs a service visit not covered in his contract. He doesn't benefit personally from the deal; the perks go to his company, which approves of the practice, so why not?
But when the time comes for Pickett to make a decision about buying his own software, he doesn't place much stake in what his colleagues have to say - and he doesn't always think that the CIOs who call him should either.
"Some of the companies that have called me have taken [the reference process] much too lightly," says Pickett, vice president and CIO for Penske, a Detroit-based transportation services company. "If they're calling me to find out whether a particular supplier is doing a good job, without knowing the complexities within my company, I'm not sure that they're learning anything. I may be giving them misleading information because their business is different from mine."
Pickett says he's up front about this. Nevertheless, he admits there is a natural tendency to avoid the negatives if things are going well. "If you have a good relationship with the supplier, you're not going to bad-mouth them," he says, though he's careful to point out that if he's receiving bad service from a vendor, he'll say so.
All of which is to say, customer references come with baggage - big, unwieldy baggage. Vendors' long-held practice of offering enticements to customers to speak to potential new customers is widespread and widely accepted by CIOs. Yet despite the fact that reference checks are possibly the most important step in the software selection process, CIOs generally know little about the references they call and even less about the terms of the arrangement between the reference and the vendor.
Companies that serve as references are likely to get things that average customers never get: preferential treatment, influence on the development cycle - even cash rebates. These customers aren't average in any way and might not reflect the reality of your situation unless you become a reference too. And even if the reference isn't getting special treatment, most software projects have become so big and complex that no two companies are likely to have the same experience anyway. All this means that customer references - especially in the realm of big enterprise software projects like ERP and CRM - have ceased to have much real meaning.
Worse, serving as a reference yourself can have tragic consequences for your company and your career. Accepting personal gifts or money for being a reference is unethical and can, of course, get you fired. But even if the gifts and special treatment benefit only your enterprise, not you, they can still violate your company's code of ethics or jeopardise your company's fortunes. For example in the US, a CIO saying good things about bad or problematic software can send up a red flag with Wall Street analysts, who now track software projects more closely and downgrade company stocks at the slightest whiff of trouble. The reference game is so much of an ethical quagmire that the US federal government has banned its IT executives from making endorsements of any kind .
The government may be on to something. The benefits that companies gain from acting as references for vendors may no longer justify the risks - both personal and companywide - that are part of the package. In fact, the only viable reason for agreeing to be a reference for a supplier may turn out to be the simplest one of all: because you think it's good.
The Care and Feeding of You
According to one survey by the IT Services Marketing Association, 52 per cent of IT buyers make purchase decisions based on referrals from colleagues. If you don't have a buddy you can call for a reference, vendors are ready to help. They enlist and groom armies of happy reference customers to take calls about their experiences. "There's a science to the care and feeding of these references," says analyst Jim Shepherd of Boston-based AMR Research. That's because references are the cornerstone of the software sales cycle.
Being a reference can be informal and simple - such as agreeing to be listed on the vendor's Web site or doing the occasional favour for the vendor's sales team. Or it can be quite complex, with specific agreements written into the software contract between the reference and the vendor. The contract might stipulate taking a certain number of calls from prospective customers, the press or analysts, for example, or speaking at a user conference, or approving a certain number of press releases and case studies about the implementation.
At large software companies like PeopleSoft, managing these references is a full-time job. In fact, it's five full-time jobs, overseen by Wendi Wolfgram, program manager for the user reference program, for which more than 700 companies have signed up. Wolfgram and her staff keep a database of information about those customers, including details about each installation, multiple contacts in different parts of the business, and the customer's level of commitment to and history of serving as a reference. When a salesperson needs a reference for a prospective customer, the reference team looks for a good match - a company in a similar industry that's not a direct competitor, that has a similar shop and that hasn't been overused as a reference. According to Wolfgram, the program is so valuable that it affects more than 78 per cent of deals closed each quarter. And it's so valuable that PeopleSoft puts a great deal of time, effort and money into keeping it running smoothly.
"We do have rewards, and it's kind of like a frequent flier program," says Heather Loisel, PeopleSoft's vice president of global sales productivity. "It's completely voluntary, and it kicks in after a certain level of volunteer activity. The reference earns points, and those points can be used for things like a pass to one of our member conferences."
Givers and Takers
Free conference passes are only the beginning. "You usually get something out of it," says Eastman Kodak CIO Mark Gulling, who has got free consulting and, from SAP, a rebate for serving as a reference. He won't give specifics but says the rebate is minimal and benefits his employer, not him. "[Vendors] understand that it takes time for you to do this, but that's not why you do it," he says. "It gives me contacts that I can go back and talk to later."
The vendors CIO spoke with for this story - i2 Technologies, Manugistics, Oracle and PeopleSoft - all downplayed these incentives. (Other competing vendors declined to be interviewed for this article.) "We don't want to have a program in place where the reward is so large that the customer says: I'm going to go after that award," says Katrina Roche, chief marketing officer for i2 Technologies. "We want people to know we appreciate them, but we don't want that appreciation to sway that decision."
But it would be naive to assume that shady reference deals aren't cut in the software industry. Roche remembers working for one small software vendor that offered a customer a $US7000 discount to serve as a "marketing centre" - participating in ad campaigns and the like.
Dave Munn, CEO of IT Services Marketing Association, says typical reference deals involve credit toward training classes or a future purchase. And a cheque in the mail? "Clearly that happens," he says.
Patti Morrison, CIO of Office Depot, and former CIO of GE Industrial Systems and Quaker Oats, does not count herself among the takers. But she does feel pressure from vendors that want her to put a reference agreement in the software contract - especially smaller start-ups trying to solidify their position in the marketplace with references so that they can get financing. "They say, If you do a press release by the end of our first quarter, we'll give you X per cent discount," Morrison says. "They might use it as a negotiating tactic for price, but that [discount] is not a good decision on which to make the choice of that vendor." Morrison, whose company has a policy of not putting reference agreements into contracts, tells them that she'll serve as an informal reference - only after the project is complete and has been successful. But even in this reduced role she still benefits, albeit in a roundabout way.
"It's an additional incentive for the vendor to support you," she says, noting that serving as a reference sometimes gets her better customer service, including access to early software releases, and better response from the vendor's senior managers and software developers. "It's a great way to build a relationship with the supplier [and get] their skin in the game to make you successful."
Blinded by the Lighthouse
Among companies that agree to link their name to a vendor, some have more value as references than others. These are the giants, the companies people instantly recognise, the brands they trust. A short jaunt through any vendor's annual report, conference line-up or Web site makes it clear which customers are so-called lighthouse accounts: Procter & Gamble and Nestlé for SAP; Charles Schwab and Deutsche Telekom for Siebel; Dell and Home Depot for i2 Technologies.
These lighthouses can help vendors gain legitimacy, sell products and boost shareholder confidence by proving that the vendor has a steady paycheque from a company with deep, well-constructed pockets. But they can also break their vendors, by talking up a project and then pulling back on it. Last February, for example, TheStreet.com reported that Manugistics' stock price fell 18.5 per cent reportedly after a research analyst said that one of Manugistics' lighthouse customers, Ford Motor, had put some IT projects on hold. Manugistics' CFO Raj Rajaji had to assure nervous investors that Lloyd Hansen, Ford's vice president of revenue management, was speaking at an upcoming user conference - proof that the partnership was still strong.
How Happy Talk Works
A reference may sound good, but it's rarely an accurate representation of reality. Companies that agree to serve as references are self-selected, happy customers, with a vested interest in the vendor leading a long, productive life. "You want your vendor to be successful and gain resources and clout so that they can improve their software," says Wendy Close, an analyst for Gartner (US). This is one incentive for a CIO to become a reference - but it's also an incentive for a CIO to be a positive reference.
References also use their position to influence the direction of the technology. "If I do things for [vendors], I end up having some influence on them in terms of the future direction I'd like them to take," says the CIO of a retail company who spoke on condition of anonymity. When asked if that makes him less valuable as a reference, there's a long pause. "An average customer would have to think that customers like me have their best interest in mind," he says finally, noting that most prospects don't ask about his relationship with the vendor.
It's worth taking the time to understand that relationship. In some cases, a company may form such a strong bond with a vendor that the company becomes something more than a customer if something less than a partner. But you have to dig to find that out for yourself - vendors won't spell it out for you.
Consider, for example, Charles Schwab's affiliation with Siebel. The brokerage has long been prominent in Siebel's marketing materials. And a casual visitor to its Web site would assume that Schwab is nothing more than a good customer. But the CEO of Schwab has been on Siebel's board of directors since 1994, and back in 1996 the brokerage accounted for fully 23 per cent of Siebel's revenue, according to an SEC filing. In other words, assuming that Schwab's experiences with Siebel will predict yours would require an imaginative leap.
What Is Your Word Worth
Of course, there's nothing inherently wrong with forming a strategic partnership with a trusted vendor. And there's not necessarily anything wrong with accepting something in return for providing a reference either. But companies had better be willing to talk about it, ethics experts say.
"If a customer is unwilling to be transparent [about the rewards], then the conclusion that a disinterested party might draw is that the person has been bought," says Daryl Koehn, director of the Centre for Business Ethics at the University of St Thomas in Houston.
So what can CIOs ethically accept in return for serving as a reference? Ideally, nothing, says Koehn. "If the vendor's products are good, the customer should be willing to make a recommendation for that vendor free of charge, right?" she asks. "The question is: why does the customer feel entitled to demand some kind of recompense, or why does the vendor feel the need to offer some kind of recompense?"
If the reference process doesn't take much time, the reference shouldn't take anything in return, Koehn suggests. But "if you're talking about the company's employees being tied up for days, it's not necessarily inappropriate for the customer to get something in return, like a small discount" or free training that takes up roughly the same time as the site visit, she says.
Regardless, the CIO had better believe in the product without reservation. That's why some companies set strict parameters around referencing. For instance, The Hartford Financial Services Group has a policy forbidding its IT group from negotiating better prices or accepting free services in return for acting as a reference. "We pay for the technology and the services that we use in dollars, not in references, not in endorsements," says a spokesperson for the $US15 billion insurance company.
"If we've had a good experience with a vendor, we will occasionally share what the results were, but we try to make sure that nothing could get misinterpreted," says Hartford CIO David Annis. That means making sure that comments made during the reference process could not be interpreted as an endorsement.
The US federal government has barred its employees from endorsing vendors and accepting any gifts unless they go through an elaborate legal approval process. "A government employee who holds the decision-making authority cannot personally benefit from that authority," says Mark Forman, associate director for IT and e-government for the US Office of Management and Budget. What's more, he says, the department as a whole cannot benefit from that kind of arrangement either. "I would consider that inappropriate," says Forman, the federal government's de facto CIO.
Getting at the Truth
If the reference process is so troublesome, then how can CIOs make sure they're not getting hoodwinked? You can protect yourself; first, put the software through its paces as much as possible. Then, look for a trusted peer to talk to you about the software before you go to the vendor. That is why it's so important to stay in touch with your peers, either by attending conferences or joining a professional organisation. "It's your personal relationship with the CIO that you have to leverage to find out what really happened. You won't get that out of a formal reference call," says the anonymous CIO.
When he does talk to vendor references, this CIO asks for contacts at specific companies on the customer list rather than accepting the ones the vendors try to give him. When you call the phone numbers, ask the companies what they did to get on the list. "It's entirely fair to ask: what have you received from the vendor in exchange for allowing me to sit here in your conference room?" says Andrew McAfee, an assistant professor in technology and operations management at the Harvard Business School, who is doing research on the CIO decision-making process. "I would imagine it's a question that gets asked less often than it should."
Merely knowing there are references is not enough. Rob Cohen, vice president and CIO at AstraZeneca, the London-based pharmaceutical company, says that two-thirds of the time, when he agrees to talk to a prospect, the company never calls. This suggests that some companies skimp on the homework - and that's dangerous. "I can't tell you the number of times where [vendors have] trotted out some very big name, but the reference is only using part of the product in one department," says Jim Sutter, senior partner at The Peer Consulting Group, and former CIO of Xerox and Rockwell International.
Companies that decide to do a site visit should send in people who will be on the front lines of the implementation, not just the CIO, Sutter recommends. "Particularly to a peer, [the CIO reference] will be reluctant to say: I really messed up when I made this decision," he says. "If you send some technical people in, they will uncover what worked and what hasn't worked."
Then, during conversations, ask lots of questions - carefully, to extract facts, rather than opinions or generalities. For instance, you might ask: how long does it take for them to get back to you? Not just: how's their service? AstraZeneca's Cohen says. And remember Pickett, the reference who wouldn't listen to himself if he were making a purchase decision? That's not to say he finds references useless. He just won't use them to make a decision. Pickett has found that they're really only helpful once he's made up his mind.
"If it's a highly valued customer, I'm going to find out from them what extra mile [the vendor was] willing to go to get [them]," Pickett says. He'll probe to find out, for instance, what the reference was able to get over the standard vanilla contract. Then, he'll ask for tips on how to actually implement a product - not pick it. "It's free advice if you use it correctly," he says.
And if you don't? Well, you probably won't be worse off for following your own mind. After all, that's what Pickett does.
Five Reasons Not to Trust a Customer Reference 1 The vendor will offer only the references who have had good experiences.
2 References get extra attention and support that you won't - unless you become a reference.
3 The customer might be receiving services - or even cash rebates - in return for serving as a reference.
4 References don't like to admit multimillion-dollar mistakes or bad-mouth crucial vendors.
5 No two big software projects are the same. The reference's success offers no reason to believe that youwill be successful.
Five Ways to Get Value from a Customer ReferenceAccording to the IT Services Marketing Association, 52 per cent of IT buyers make purchase decisions based on referrals from colleagues. Which makes sense. Who knows your problems better than your peers do? But, as you've read, the customer referral process is fraught with conflicts of interest, and the information you derive needs to be taken with a few pounds of salt. Here's some advice on how to season the data you need with wisdom.
1 Put the software through its paces as much as possible. (You've heard the expression "buying a pig in a poke"? It's never a good idea).
2 Before taking the word of a CIO you don't know, try networking with CIOs you do know. (Another reason you should make a practice of regularly getting out of your office to hobnob with your fellow wizards).
3 When a vendor lists a company as a reference, you can presume that that company had a good experience with said vendor. But that presumption should not dissuade you from calling and finding out for yourself. (In other words, don't be lazy; do your homework).
4 When you call a reference from a vendor's list, it's wise to ask what exactly the company got in return for agreeing to be a reference in the first place. (It's also fair. But the question doesn't get asked as often as it should).
5 Be specific. Don't ask, How's their service? Ask, How long does it take them to get back to you? (What's good for one CIO may be wholly inadequate for you).
Different Strokes for Different GovernmentsIn Australia, no mandate from above prohibits government IT execs acting as vendor refereesPublic service codes of ethics are seen as the best bastion against potential trouble for Australian public servants wondering whether to agree to allow their agencies to act as customer reference sites.
Steve Alford, general manager government online infrastructure with the National Office for the Information Economy (NOIE), says he is not aware of any firm federal government stance on the issue. But he says a number of general principles outlined by the Australian Public Service Commission relating to appropriate behaviour by departmental officers, and embracing issues like conflict of interest and probity, are available to guide public servants' personal behaviour.
Alford says Commonwealth officers should not agree to accept inducements of any kind from vendors, whether for personal or corporate gain. However, as frequent early adopters - particularly in the early stages of development of a product - governments are often asked permission by vendors to be listed as a reference site. "That means that basically you say: Yes, we've installed the product, these are the business issues that we were dealing with, and this product is installed to help us deal with those issues.'"Generally you'll find that when Commonwealth officers speak - as they often do at conferences and those sorts of things - the things they mainly talk about are the business drivers that were leading them to develop a certain strategy and the outcomes of the project. Any mention of the software that assisted in that process is purely incidental, really."
On the other hand, Alford says, those charged with the purchase of services on behalf of the Commonwealth recognise the value of being able to consult with fellow Commonwealth officers listed as vendor referees about their experience with products.
And he says there is another dimension to the issue, given that government officials are on occasion called upon to support trade missions organised by the Department of Foreign Affairs and Trade to promote Australian ICT companies. But Alford says public servants need to be constantly aware of the limits of such situations. He says he was approached recently by a company he does business with to sit down with the investors. He declined, conscious that he risked providing advice these investors might rely on which would turn out to be unwise.
The other issue Commonwealth agencies need to be aware of is the emerging trend for agencies to decide to take on a "sophisticated, often expensive" solution to a business problem, with the business case predicated on them getting other agencies to work with them on a cost-sharing basis. That puts the Commonwealth officer potentially in a conflict of interest, since the business case internally is predicated on sharing expenses, and convincing people to share those expenses is predicated on getting other agencies to accept that this is a worthwhile product or worthwhile solution to the problem.
"My preference in those cases is to scope the project early and identify if this is an issue before you go to tender, and identify your potential partners in the process ahead of time, rather than inviting them in as a potential sales person after the event," Alford says.
The situation is similar at state level. Randall Straw, executive director, Multimedia Victoria says the Victorian Government does not have a central policy on departments or agencies acting as customer reference sites, but leaves it to the determination of individual agencies.
Victorian Government agencies have often acted as customer reference sites, he says. "As a buyer, the Victorian Government usually seeks out other government customer reference sites because the exchange between government to government is often more frank.
"The public service codes of conduct and employment conditions prohibit individuals from benefiting from purchasing decisions," Straw says.
The attitude is much the same in Queensland, where John Parker, director Government Technology Queensland Government, says there is no policy on customer reference sites since the government has an industry development program, which would fly in the face of not being a reference site. "We have a case-by-case basis and the agencies make up their own minds in terms of whether or not they think it's appropriate. And therefore by default I, and this agency, would assist them where possible - particularly in the case of Queensland businesses, if we've done business with them and we feel that it's a reasonable product," he says.
The policy is different for multinationals, he says, with agencies more reluctant to agree to act as a reference site in those cases.
Join the CIO Australia group on LinkedIn. The group is open to CIOs, IT Directors, COOs, CTOs and senior IT managers.