. . . the Bad
That brings us to the bad news: the fact that many experts now believe few companies will ever be able to rely purely on technology to resolve their data quality issues.
"The reality is that real data, as in data that people actually use, as in real-life data, is noisy: it's got errors in it," says Curtin's professor Pervan.
"And I'm not sure that there is going to be a technological solution at all. To some extent you can, by picking up outlying items, sort of correct for it but data is always going to have errors in it. It's getting worse of course, because we've got more data, and the possibility of having bad data is increasing."
The lack of a ready technological solution presents many challenges for companies wishing to move ahead with strategic IT projects. For instance, data quality is the cornerstone of any successful customer relationship management implementation. Incorrect information - for example, duplicate contacts, outdated or erroneous contact information, contacts incorrectly categorised - erodes the system's credibility with users, erodes an organisation's credibility with their clients, and limits professionals' ability to make sound business decisions and produce effective business development programs.
"Avoiding the entry of incorrect information and identifying and correcting that which seeps in are essential to ensuring the effective use of relationship intelligence," says Rick Klau, vice president of vertical markets with US company Interface Software.
A provider of CRM software and services, Interface understands how to achieve data quality and is keen on innovation. Early in its development (circa 1991), the company struggled with the issue as much as any other organisation. But Klau says more than 300 implementations brought perspective on the barriers to data quality and enabled the company and its customers to overcome them. Now Interface uses its data quality features and process capabilities as key selling points.
"Many companies have failed to realise that the real problems in data quality lie as much with process as with technology," Klau says. "In a typical organisation, project, contact and relationship data exists in application and practice group silos. It's sliced, diced, expanded and updated for many different purposes." Klau says the problem is compounded because insights from one silo rarely get shared with any kind of centralised database due to time constraints and the limits of database reconciliation technology.
"I think this is an ongoing problem for virtually every financial institution, because they all have multiple channels," agrees Zurich Financial Services Australia head of investment management and life (and former CIO) Peter Delprado. "Some business will come in through an intermediary such as a financial planner or an insurance broker, and then they'll have other channels where it comes direct, and everywhere I've worked they've had this issue."
Delprado points out that the quest for absolute data quality can be damaging to corporate intent. For instance, while good quality data is essential to CRM, overzealous efforts to maintain that quality can occasionally prove counterproductive by marring the organisation's relationship with some customers, potentially neutralising such well-intentioned efforts. While there is some fairly sophisticated software available to do the de-duping, scrubbing and cleansing, the only way to ensure that the data really is up to speed is to check the accuracy and currency of data every time you touch the customer.
But Delprado says organisations must always remain sensitive to customer sensibilities, and avoid unnecessarily annoying customers in their efforts to keep the data up to date. He says organisations should also remain sensitive to the privacy and other concerns of customers.
"Take the channel conflict issue, where you've got a customer coming in through a planner and taking out a certain policy and then coming in through a general insurance broker, and it's the same person. They may specifically not want the two (sets of information) to come together, and we certainly honour the customer's privacy in that circumstance. We don't even attempt to match that kind of data.
"I know that other institutions want to get that database [match] because they have a view that if a customer has got a bank loan with them over here and a superannuation product over there, then the bank thinks that they should be telling them one story at all times. But there are certainly circumstances where customers don't want that at all. We certainly would prefer to keep the customers happy," Delprado says.
Join the CIO Australia group on LinkedIn. The group is open to CIOs, IT Directors, COOs, CTOs and senior IT managers.