Whatever enterprise application project you're facing, several major - and some hidden - factors will affect its cost.
A great number of Australian companies have integrated an awful lot of data in the quest for new operational efficiencies and improved business processes. Along the way most of those organisations have forked out bucket loads of money. But for all their efforts, integration remains a murky area where pitfalls lie in wait to trap the unwary and even the best prepared organisations can find their efforts thwarted by unanticipated problems.
The need for integration is being increasingly widely recognised. According to a survey published last US spring by Morgan Stanley Dean Witter & Company, 35 per cent of all Fortune 500 companies list integration as their top objective. More telling still is that the next several objectives on the list - e-business, CRM, and supply-chain management, respectively - all require integration. But experience shows most integration projects are subject to cost overruns in unexpected areas. Many of those bitten in recent times can testify: integration projects are a recipe for bloat.
Just how much of an issue the cost of integration has become was made a little clearer late last year when Mercator Software (a provider of business integration solutions) completed its round of confidential internal customer audits with the CIOs of Australian top 100 companies on integration and legacy technology in Australia. The audits found four out of five top 100 Australian companies have integration issues, and for most of them the major issue is cost. Capping development costs with respect to integration has become a major issue for almost all of those top companies, says Mercator vice president, professional services Asia Pacific Peter Zeglis.
"Ninety per cent of the top 100 enterprises in Australia still retain their core legacy systems and less than five per cent replace or override them. Most of them have all good intent to replace them in the future, but less than five per cent are replacing or overriding them at this stage," Zeglis says.
"For most of the CIOs there are core business drivers such as mergers and acquisitions, the need to replace legacy systems, as well as e-commerce and Web initiatives and enablement of ERP and CRM. There are also a lot of shared services initiatives in place. In order for those initiatives to happen, organisations clearly face integration issues; and one of them - the predominant one - is capping development costs."
Other issues the audits highlighted are: problems with architecture, Web-enablement, ensuring data integrity, decoupling legacy applications, changing business needs and growth, portability of standard architecture functions, containing operational costs within the execution environment, best-of-breed components and lack of developers.
Problems with Architecture Organisations need to know they can assure information delivery not only within the enterprise but also beyond the enterprise. Ensuring data gets delivered isn't easy where organisations face data integrity issues (see below), where systems aren't getting updated overnight because of reliability issues, or where there's no appropriate messaging infrastructure.
While the need is critical, Zeglis says, and some products for ensuring information delivery are available, organisations are struggling to develop the supporting architecture, particularly where there's a shortfall of skilled architects capable of designing, developing and implementing an infrastructure for assured delivery. "What they're finding is that there is a real shortage of really highly skilled architects who have the vision or leadership to actually implement a solution that is scalable and can provide that delivery," he says.
"The best companies are paying the top dollar to get these people in. The companies that don't basically won't adopt a messaging-based architecture and will use a traditional copy-based approach. Will that cause them problems down the track? It certainly will, both in terms of consistency, security and in ensuring data gets from point A to point B."
Web-enablement The second issue plaguing most organisations is the need to enable new e-commerce initiatives. Organisations are eager to start delivering information across the Internet because of its low cost. They also want to open up new trading partner relationships and enable e-business over the Internet.
"The issue here is not only do they have existing integration issues within the enterprise, they also have them beyond the enterprise," Zeglis says. "How do they provide an infrastructure, or put something in place so that they can enable the plug-and-play of any system, whether external to the enterprise or internal to the enterprise? It's really about scalability and having a scalable architecture in place or something in infrastructure in place, which sits on top of the network to enable new business initiatives.
To address the issue, he says leading companies are deploying the Big Five, and laying the foundations for enterprise application integration (EAI) architecture. But banking on partnerships can be risky. If any of the relationships between partnered vendors and firms sour later, companies that bought the package deal are stuck: they must either replace the outcast software or service, or risk paying higher fees to continue doing business with the estranged partner.
Ensuring Data Integrity Assured delivery means nothing unless the data delivered is pure. In some cases where organisations are using traditional approaches to delivering information from one system to another, the data gets delivered perfectly, but it hasn't been scrubbed and is therefore unreliable.
Companies that have processes and tools in place to ensure data is clean won't have ongoing problems, Zeglis says. But as part of the integration exercise, some companies assume that the data they're receiving is okay. It rarely is, as more and more of these companies are now discovering to their great cost. Those companies that are doing it well need to start introducing those good habits into the organisations they're partnering and trading with, he says.
Decoupling Legacy Applications Many Australian organisations over the past five years have moved from a traditional product-centric environment to becoming much more service-based, customer-centric organisations. As a consequence the need has been building for them to replace their core legacy application, but with most core legacy applications tightly bound to existing applications decoupling can prove to be extremely difficult.
"The difficulty is that there's a deep-rooted range of interfaces that have been in place and been in operation for many, many years," Zeglis says. "The system works, and to go away and try to replace that legacy application is quite a mammoth task. In the case of one particular system that we looked at there were up to 300 individual interfaces that would have to be replaced just to simply swap that system out.
"Clearly these systems are very tightly coupled and tightly bound. For example, you could have one department asking for an account enquiry, which is an interface that has been written from one area to another. Then in another area you have another department basically issuing the same request, which is another separate interface. What we're seeing is this is very tightly bound and there's a lot of transactions. If we try to normalise those transaction types, we see a lot of duplication of those interfaces. To simply undo that is quite a complex task."
Zeglis claims those companies that are mastering the task have normalised the transaction type under their EAI architecture, then distributed functions or business objects for use by subscription.
Changing Business Needs and Growth Business conditions change and organisations are constantly shifting their focus. Having a robust scalable technical architecture in place makes it easier for organisations to accommodate business needs and growth. Zeglis says it's a matter of organisations having the necessary bandwidth, pipes and infrastructure to support integration demands going forward.
Portability of Standard Architecture Functions It's one thing to develop applications and provide integration with legacy systems, and to build architecture standards and integration standards around that work. The question becomes how can the organisation define and provide portability of standards across different environments, including Web services (see "The Essential Guide to Web Services", page 98).
Containing Operational Costs within the Execution Environment It's bad enough forking out for high development costs, but there's also a challenge in ensuring unit transaction costs are kept to a minimum.
EAI vendors claim that deploying EAI and the products to support it allows organisations to obtain efficiencies that minimise operational costs. Business processes are streamlined, alleviating the need for labour-intensive overheads - like manual reconciliation - via automated functions.
Best-of-Breed Components Zeglis says that best of breed applications are becoming increasingly important, even in the financial services space where organisations used to prefer to develop everything themselves. He says every CIO he's spoken to has some best of breed systems in place, and is looking to find the right balance between these and the systems they develop in-house to maintain their competitive edge.
Supporting that changing mindset are a host of pre-built business solution adapters to enable best-of-breed applications in financial services, manufacturing, healthcare and even government. "You'll find that most of the products out in the marketplace have pre-built business solution adapters which basically plug in to these business solutions," Zeglis says. "There are SAP adapters. There are CRM adapters."
Lack of Developers As well as the shortage of architectural skills the country is also facing a lack of integration skills. Companies needing "floor level" integration have a huge need for developers and are forever contracting systems integration partners to provide all the delivery components. That too can be a hugely expensive exercise.
Behind the EAI Ball
With so many issues plaguing so many organisations Zeglis claims Australia is a long way behind its US and UK counterparts in the integration game. There many companies adopted the EAI concept four or five years ago. Here some early EAI adopters are seeing a ROI, but the rest are barely dipping their toes in the water with respect to this architecture or integration approach.
"My advice to our clients is that integration itself, and this is one of the major problems, used to be thought of as a development cost, as an issue, as a point-to-point solution. The issue is that integration needs to be thought of as an architecture in its own right.
"A purpose-built architecture needs to be developed, and it should be part of the overall scheme that sits on top of the networks, the pipes, the layer of infrastructure that needs to be carefully thought and planned. Some of the larger organisations in this country are doing exactly that. And it's really refreshing to see," Zeglis says. National Australia Group, Coca-Cola Amatil, Colonial First State and The Australian Postal Service are leading the Australian field, he says. VPackaged Tools Still a Challenge to UseBy Michael Meehan For some large companies, the need to tie together incompatible systems has grown to the point that the main mission of their IT departments is shifting from application development and support to integration.
In the US, users such as Corporate Express, Best Buy and Union Pacific say they no longer have any choice about braving the rigours of building an enterprise application integration (EAI) infrastructure. EAI tools still have a reputation as being difficult to use, but IT managers at those companies say trying to cope with "application spaghetti" is worse.
"We really need to make integration a central part of every IT initiative," says Marty Malley, director of information systems at Union Pacific. For example, he says, the railroad has to find a way to deliver real-time pricing information from its systems to freight customers in the oil and gas industries.
"That price data touches many internal systems, and now we've got to find a way to tie it all together," Malley says. In November, Union Pacific began working with EAI tools developed by Tibco Software.
Once in place, the integration framework will also be used to help migrate components of the railroad's core transportation control system off its mainframes. But Malley says the railroad hasn't set a full road map for the EAI work. He also wouldn't comment on the project's expected cost.
Monty Sooter, CIO at Corporate Express, says EAI technology will play a role in all four of the major IT projects the office supplies distributor plans to take on this year. "We made a business decision that the [integration] infrastructure has to be correct," Sooter says. "If it's not, we're not going to be able to move forward."
In a project that began 18 months ago, Corporate Express initially used EAI tools developed by webMethods to integrate its systems with the business-to-business applications of 120 large customers.
Now, Sooter says, the company plans to use the tools in rollouts of i2 Technologies' online procurement and warehouse management applications. They will also be used as part of projects aimed at improving information feeds to Corporate Express's suppliers and for developing an online system for paying its sales force. The EAI layer will funnel data among various databases and front-end system interfaces, Sooter says.
Tyler McDaniel, an analyst at Hurwitz Group (US), says early versions of EAI tools were difficult to use. But improved business object, repository and graphical user interface capabilities have lessened their arcane nature, he says.
However, EAI projects can still be long and hard.
For example, Best Buy plans to use webMethods' tools to patch its electronic data interchange (EDI) trading partners into a new Windows 2000 server backbone that's being installed to run its inventory and financial systems.
The electronics retailer has such a heavy volume of EDI transactions that it's running short of the nightly downtime needed to complete its batch processing, says Patricia Vessey, e-business communications manager at Best Buy.
The webMethods tools will translate data among various systems for real-time transaction processing of purchase orders and invoices, Vessey says. But she estimated that it will take two years to do the required data-conversion work.
Join the CIO Australia group on LinkedIn. The group is open to CIOs, IT Directors, COOs, CTOs and senior IT managers.