ESB Persists As Application Integration Tool

A technology with a knack for reinvention, the enterprise service bus (ESB) has been toiling away in IT shops, largely unnoticed, for about a decade.

ESB products swept onto the scene as the latest generation of enterprise application integration technology and a replacement for older message-oriented middleware. The ESB's initial role, going back to the early 2000s, was to translate different data formats, let applications communicate with each another through a messaging layer and make sure the messages passed among applications reach the intended target. By 2005, ESBs had become the foundation for service-oriented architecture (SOA) deployments. The technology took on the task of coordinating Web services.

Over the years, though, SOA has lost some of its appeal as a term of art. "Services" of late have been recast as application programming interfaces, and API management tools have emerged as a product category. In April, CA Technologies inked a definitive agreement to purchase Layer 7 Technologies, an API management vendor, underscoring the growing interest in the field. The rise of cloud computing, meanwhile, has generated new types of integration challenges and technology remedies.

Against that backdrop, ESB has entered yet another transitional phase. Jason Bloomberg, president of ZapThink, a Dovel Technologies company, says there's a lot of noise in the market for and against ESBs. ZapThink, based in McLean, Va., offers training in certification in SOA and cloud computing.

"It's a little hard to cut through and find out what's really going on," Bloomberg says. "Customers are generally confused about the whole thing."

ESB vendors, however, contend that their products will continue to have a role in both traditional integration and as a core component of new applications. Along the way, the technology must navigate overlapping API management capabilities and cultivate niches such as Software-as-a-Service integration.

ESBs at Work: Integrating Disparate Systems

Enterprise customers, for their part, continue to carry on with ESB. That group includes some recent adopters such as NCR Corp. The company deployed MuleSoft ESB software about 18 months ago, notes Eli Rosner, vice president of global software engineering at NCR.

Rosner says ESB offers the ability to handle complex integration while providing speedier time to market. The technology lets developers integrate applications quickly, as it offers built-in connectivity and shields them from having to deal with individual connectors. "ESB as an integration engine is very important to us," Rosner says.

ESB helps NCR bring together three key domains: The company's data center, a customer's data center and the public cloud. The integration technology can unite a SaaS solution hosted in an NCR facility with an enterprise resource planning system at a customer's location and link both into public cloud technology, Rosner explains.

In one example, NCR uses ESB to integrate a loyalty program system it's building for a retailer. The system will involve an NCR-developed Facebook app hosted on the Microsoft Azure cloud platform and NCR's own data center resources. It will let customers participating in the loyalty program opt into retailer offers via Facebook. ESB, in this case, will tie the different elements together.

In a different scenario, ESB may be deployed to help organizations reconcile legacy point of sale gear with a new payment system. A company could have a mix of legacy and new POS technology, but the data generated by the older POS devices exists in a format incompatible with the backend system.

POS transactions can be routed to NCR's ESB, which then determines whether it came from a legacy or new POS based on the device ID, Rosner says. "Based on a rule, [the ESB] then triggers a transformation of the data to the one accepted by the system."

Rosner estimates that NCR's ESB handles hundreds of concurrent connections but adds that the number of connections will grow exponentially once the company works through the initial challenges of the deployment.

ESB 'Agnostic To the Technology Platform'

Another ESB user, Social Interest Solutions, a Sacramento, Calif. nonprofit that builds health and social services systems, has been deploying the technology for several years.

Social Interest's ESB is built on Microsoft BizTalk Server and Windows Communication Foundation. The ESB is part of the organization's One-e-App system, which lets people submit one application for multiple health and social services programs. The ESB includes a rules engine that contains the eligibility requirements for various programs; organizations in Arizona, California, Florida, Indiana and Maryland use One-e-App.

Ashok Rout, chief technology officer at Social Interest, sees a continuing role for ESB technology. The organization is now expanding its Arizona system to help the state meet its Affordable Care Act mandates. The system, slated to go live October 1, will use the ESB to integrate with federal and state systems. The connections will be used to verify income and other eligibility criteria. ESB will also provide a mechanism that other states can use to leverage features or functions of the Arizona system.

Rout counts support for different integration methods among ESB's enduring qualities. "The beauty of ESB is that is allows you to be agnostic to the technology platform [and]...the integration you are doing," he says.

Rout notes that his organization's ESB allows Web services, APIs or flat files to be integrated. An ESB designed to be agnostic can accommodate new integration approaches as they surface. "ESB is going to be critical in the future," he says.

ESB, API Management Coming Together

While customers see a stable role for ESB, the technology itself is far from static.

"I've seen a lot of evolution in the ESB marketplace," notes Paul Fremantle, co-founder and CTO at WS02, which provides an ESB among other enterprise middleware products. His background includes building one of the first XML Web service gateways, which found use in 2003 as an early ESB at Charles Schwab.

News: Software AG Goes All Out for In-Memory ESB

Freemantle cites the convergence of ESB and API management functions among the current technology trends of note. He says ESBs have taken on the quality of service, usage throttling and access control roles that are typical of API gateways. On the other hand, API management vendors have effectively incorporated a number of ESB capabilities into their products, he adds.

WS02 offers both technologies, having launched an API management product in September 2012. Freemantle says newer customers are adopting both products and using ESB and API management components where they see a fit.

Ross Mason, founder and vice president of product strategy at MuleSoft, agrees that ESB and API management are coming together. "API publishing and management have become a very big piece of what enterprise integration is today, he says.

Mason says APIs have become the mechanism for connecting endpoints that exist outside of the firewall-cloud platforms and SaaS systems, for instance. He adds that mobile devices also need APIs to connect back into enterprise systems.

With that in mind, MuleSoft in April debuted its Anypoint Platform, which includes Mulesoft ESB along with API-related components such as APIhub, a public API respository; Anypoint Service Registry, a cloud-based registry for API governance, and Anypoint API Manager. The latter module is in beta now and slated for availability in mid-2013, according to Ken Yagen, vice president of products at MuleSoft.

Future developments may overtake both ESB and API management. ZapThink's Bloomberg believes Integration as a Service, which traces its roots to B2B integration products, will emerge over time as the next generation of ESB and API management. In the meantime, enterprises will turn to ESB and API management to pull together in-house applications and a growing population of external endpoints.

Related: Alcatel-Lucent Launches Open Source Platform for API Management

"ESB is a good foundation bedrock to enable that," Mason says. "Layering on API capabilities on top of that is extremely important to reach all the requirements of the enterprise."

John Moore has written on business and technology topics for more than 20 years. His areas of focus include mobile app development, health IT, cloud computing, government IT and distribution channels. Follow everything from CIO.com on Twitter @CIOonline, Facebook, Google + and LinkedIn.

Read more about service-oriented architecture (soa) in CIO's Service-Oriented Architecture (SOA) Drilldown.