Sears Roebuck knows who its customers are. It just doesn't know which of its customers are the same people. "Sears is a series of silo businesses," explains Mark Sherman, senior project manager at the giant Hoffman Estates, Ill., retailer. Under the Sears umbrella, customers can get a credit card and use it to buy clothing from a catalogue. They can dye their hair red, then have their picture taken. Sears will go to their houses to kill termites and install fences. But each purchase record appears in a different Sears database; as far as the company knows, that's six separate transactions from six different people. Sherman's job is to tie together the records that make up a customer's profile. But first, he had to figure out which records belonged to the same person.
His job got even more complicated because these data sources don't have to use the same underlying structure. The hair salon may index records by hairdresser's name, while the credit card database identifies customers by Social Security number, and the appliance-repair arm tracks service calls by the customer's home phone number. Furthermore, some of those businesses are licensees; Sears eventually gets the billing information but doesn't necessarily own the customer data.
So Sherman needed some help in order to first get the data out of the databases and then standardise it. He's using Vality Technology's Integrity tool to clean the data and organise it into a dynamic and unified data view. The unified data view allows Sears to build a picture of each customer. "Ultimately, when we're done, we've created a powerful marketing tool," Sherman says.
When Is a Warehouse Not a Warehouse?
Any experienced CIO will say, "But that's a data warehouse." True, Sears and others are using software like Vality's Integrity (and others from InfoRay and MatrixOne ) as a tool for creating a warehouse. But the warehouse doesn't have to be the company's ultimate goal.
This type of software produces unified data views in real-time from data still stored in its original source database. The data is current, as opposed to the data in a data warehouse, which is periodically updated.
Scott Lundstrom, vice president of technology research at the Boston-based analyst firm AMR Research, sees this "next-generation philosophy of an executive information system" becoming more common within IT departments. What makes unified data view software different from traditional data warehouses, Lundstrom says, is that it has been heavily influenced by the Internet. "The warehousing paradigm was that if you could structure the data right, then the answer would pop out. But the Internet said that the gold was in the content," he explains. "If we bring [the warehousing and Internet philosophies] together, we can build a business view, not just a transaction view." The focus of this new technology is on creating the view, Lundstrom says, not on managing the data.
There are dozens of vendors working on unified data view software, says Lundstrom. The enterprise-application vendors are particularly interested in supplying this type of software to their sales, relationship-management or supply-chain software customers. However, Lundstrom points out, they tend to produce solutions focused on the data sources and reporting needs of their market segment. Other vendors concentrate on creating unified views of data from particular databases or Lotus Development 's Notes. Lundstrom describes another set of players as "born-again OLAP [online analytical processing] and reporting vendors," that already have expertise in data analysis and data mining, trying to extend their franchises into the unified data view market.
The new players, Lundstrom says, can work with unstructured data, such as that found in sources like Web pages and free-form database fields. Their horizontal applications can work with a variety of data sources and answer a wide range of business questions. Vality, InfoRay in Cambridge and MatrixOne in Chelmsford, Mass., belong in this category.
Vality has tools for identifying and correcting errors in data. It also uses fuzzy logic to make data matches, allowing users to set a level of certainty at which Vality declares that the data is related. This capability has allowed Sherman to build a system that can efficiently deal with Sears' large volume of records-236 million for the pilot project alone. "We try to find the happy medium between speed and quality," he says.
InfoRay's eponymous software and MatrixOne's eMatrix are similar. Both allow users to specify the translation rules for the data in the original sources-legacy, client/server and Web-based applications-and have a separate component that contains the company's business rules for handling that information.
A Custom Job
It was this ability to preserve his company's business rules that attracted David Boscia, manager of engineering systems for NCR 's IT Global Applications Development Department, to MatrixOne. He is in the process of linking financial offices in Dundee, Scotland, Waterloo, Ontario, and retail offices in Atlanta (with San Diego and Columbia, S.C., in the planning stages) that provide engineering design services for the Dayton, Ohio, company's retail sales systems.
Boscia is replacing a home-grown system that relies on batch files to copy data from one site to another, and has all the problems associated with manually replicating data and synchronizing information databases. Now each site will be able to access the same information using MatrixOne's Web-based interface and have confidence that they're all using identical, current data.
What was even more attractive about MatrixOne was that Boscia is able to preserve each office's particular business practices. "NCR's weakness is not being able to find a truly common global process," says Boscia. With MatrixOne, each site will be able to configure the business rules a bit differently, even though they're using the same process.
Those customisations are the most challenging part of his project, Boscia says. eMatrix comes out of the box as a generic application, which means that he has to spend time putting in NCR-specific extensions. But he doesn't think that having eMatrix preconfigured for his business would be a better option. "That would be a monster because each company does things a little differently," he says.
To help with the customisation, Boscia turned to the MatrixOne consulting services. "Inevitably, you're going to need professional services," he says. These products are too new to have a base of independent consultants, and they are too complicated to use without some training and practice, he says.
Premera Blue Cross, a health-care organisation based in Mountlake Terrace, Wash., also had to hire the vendor's consulting organisation-in this case, Vality-to get its unified data views project off the ground. The company needed to create common member IDs from insurance eligibility records stored in two reporting data marts that run on IBM 's DB2 and SQL/DS databases. According to Wayne Paradis, Premera's information warehouse architect, the common member IDs are used to support the development of reports that help Premera perform the analyses (known as HEDIS-the Health Employer Data Information Set) that corporations are starting to require when they solicit bids for health-care contracts.
At Fortis , a Utrecht, Netherlands-based banking and insurance company, the payback for using InfoRay was better, faster internal processes and the ability to quickly create new products and services. According to Wim Vonk, managing director of Fortis's insurance group, the initial goal was to improve the company's revenue analysis: where revenue came from, which clients were the most profitable and where the business's growth was. He picked InfoRay to build a unified data view of insurance administration, risk assessment and mortgage information stored in Microsoft 's Access and Computer Associates International 's IDMS databases. The payback came when InfoRay's unified data views allowed Fortis to quickly analyze the effects of changing the rates on its mortgage products. What previously took two to three weeks, says Vonk, now takes one or two days. Fortis also made use of InfoRay's ability to combine financial and nonfinancial data within one view. That allowed the insurance company to not only answer questions about quantifiable data like costs but to bring less rigid measures like customer satisfaction into the analysis.
Should You Unify?
Two types of organisations can benefit from unified data view software, says AMR's Lundstrom. The first is companies with complex internal structures, multiple sources of supply, diverse product lines and a large number of customers. The idea is to strip down the data to its significant elements for analysis. The second type: a company with a complex IT environment, with best-of-breed applications and no unifying application architecture, or one with multiple businesses units that need to coordinate but run on different systems. Instead of trying to rationalise the data at the transaction level, these companies are better off creating a unified view of their data, says Lundstrom.
The key to success, says Lundstrom, is to build a dictionary of common meta-data and common terms so that data can be matched and compared. "If all I'm doing is displaying data from a bunch of different apps concurrently, I haven't helped the user as much as I could," he says. "We haven't created a unified understanding of the data, just a unified view."
Join the CIO Australia group on LinkedIn. The group is open to CIOs, IT Directors, COOs, CTOs and senior IT managers.