SOA: A Governance Nightmare
- 03 April, 2007 15:04
As a graduate student years ago, Layer 7 CTO Dr Toufic Boubez used to have a poster on his wall that read: "A distributed system is one in which the failure of a computer you didn't even know existed can render your own computer unusable."
That about sums it up, really, doesn't it? In a few neat words, the poster suggests both how long flexibility has been a dominant theme of software engineering, and — given the vehement response the slogan still gets in seminars and presentations — just what an elusive goal flexibility remains even today. Indeed whenever Boubez repeats the slogan, people all around the room nod their heads in agreement, suggesting brittleness remains one of software implementers' most dominant preoccupations.
Whatever the promises of SOA, the reality is brittle interconnections with coding of each endpoint
Yet as anyone who has headed down the SOA path knows, things are very different in the real world, where real applications live. Whatever the promises of SOA, the reality is brittle interconnections with coding of each endpoint. In fact, Boubez says, to date the promise of loose coupling has only ever proved real for the simplest, most "vanilla" Web services, like those with no security requirements. Constraints and capabilities for services have to be hard-coded, while any changes in these preferences will render your own computer unusable.
"The original goal of the service-oriented architecture concept, and its implementation technologies such as Web services, was to build flexible, loosely coupled systems. But any two components in a system that communicate with each other are by definition coupled to a certain extent," he says.
"The fact is that currently the way we build service-oriented architectures using Web services is pretty tightly coupled for anything that is not just plain vanilla-type Web services. [SOA] works well in the lab: that whole trilogy of Publish, Find and Bind using SOAP [simple object access protocol] and WSDL [Web services description language] and UDDI [universal description, discovery and integration] and so on. It works pretty well under very controlled conditions, but as soon as you start deploying that stuff in the real world — where you need to deal with issues like security models, like identity issues, like access control, encryption, confidentiality, integrity, even routing and encapsulation, all that kind of stuff — there is absolutely no way to deal with it properly right now using any of those mechanisms," Boubez says.
"If you have pilot projects that don't have to deal with these things then loose coupling works pretty well. But any time you have to deal with the real world it breaks down unless you start thinking pretty hard about a policy layer/policy abstraction layer," he says.
So in pursuit of real flexibility, the aim of the game should be to lessen that coupling or, at least, to loosely couple components in systems by removing or lessening the run-time dependencies between them. According to Boubez, the best mechanism to achieve that is to delegate as far as possible the run-time tasks to the infrastructure.
If this is to work, the organization needs to define and automate contracts, requirement and capabilities through a declarative, configurable and manageable mechanism. But while WSDL is the contract language for Web services, Boubez says WSDL is far from being adequate as a contract language for SOA.
Page Break
To make up for those inadequacies, Boubez is urging organizations to adopt the concept of "policy driven" SOA as the new contract abstraction for SOA. "Policy is that extra abstraction layer that decouples services from their implementation, and that decouples not just service providers but service requesters: applications that request services from the same kind of infrastructure and security model and all that extra stuff that you need in the real world," he says.
The only way to architect and implement secure and flexible SOA is to find ways to mediate interactions between services based on a declarative and remediable policy language. Coding interaction policies between services results in brittle SOA infrastructures that cannot accommodate changing security and business needs.
Done right, Boubez says, a policy layer helps ensure SOA can finally start addressing business goals.
There are two main business drivers for SOA: growth and increased productivity. Inside the organization, enterprise application integration (EAI) has played an essential part in increasing productivity by allowing systems to integrate so they work better and faster and empowering users to do more and better things. Externally, integration with partners has been essential for growth: the more people and companies you partner with, and the more services you can offer your customers, the better.
However, as business continues to pursue the goal of making IT more of a strategic tool and less of a necessary evil, SOA also plays into major corporate or strategic goals like the push for flexibility and agility, which requires organizations to excel at integrating with internal and external resources as needed. Flexibility means being able to easily add and change system integrators and the IT platform, while avoiding vendor lock-in. And it means establishing what Boubez calls "one-time configurable ITN policies" that allow changes in partnership agreements, for instance, to be made simply and quickly.
As organizations pursue easier, cheaper and more logical ways to build applications and unite the silos of functionality, the concept of just-in-time integration, a technique for combining new functionalities as quickly and cheaply as required, whether internal or external to the organization, has risen in importance.
"From the architectural perspective, just-in-time integration is a cornerstone of service-oriented architecture," Boubez wrote recently. "Under SOA, applications consist of aggregations of calls to services. Services are simply coarsely-grained functions that are made available to invoking applications using a consistent semantic. They might encapsulate a well-defined unit of business logic, a legacy application or an interface to a data gathering system. What a service does is not a concern of SOA — how it is made available is. One of the fundamental principles of SOA is the idea of loose coupling. Loosely-coupled systems exhibit a flexibility such that a change effected in one component of the system does not break the rest of the system.
"To achieve this, SOAs typically provide a mechanism to publish services and a means for consumers to discover and invoke them dynamically. Web services is an SOA enabling — composed of technologies like SOAP, WSDL and UDDI — that has the unique characteristic of being based on open standards and being independent of the deployment platform. This is in contrast to other SOAs, such as Sun's Jini, and alternative distributed application technologies, such as OMG's CORBA or Microsoft's COM+. But despite the media triumph of Web services, we still have a long way to go from this basic set of technologies to the end goal of just-in-time integration," Boubez says.
Page Break
That is because in the real world CIOs face multiple hurdles during deployment of SOA, starting from when they first "get their toes wet" and continuing through to total SOA immersion. The hurdles begin with the need to define a security model, as Boubez relates.
"As a CIO, let's say I have over the last five, 10 or 15 years been implementing and provisioning identity access control mechanisms into my organization. I also may in a lot of cases have implemented PKI. In some instances I have regular directory services such as LDAP or Active Directory. Maybe I have portals also that I have integrated somehow into those secure infrastructures. Now Web services comes along and everything gets routed through Port 80, right? So how do I deal with that new model, and much, much more importantly, how do I deal with that new model without having to rip anything out and start fresh? How do I leverage my existing security infrastructure into the service-oriented architecture that my architects — and I believe them, I have to trust my architects — are telling me is the way to go?
"The question is: can we do that using Web services technology today?"
Probably not. Consider the way Web services are supposed to work in the area of owned and managed inventory. In theory, Boubez says, when an organization wants to enable suppliers to manage their own inventory it gives them an entry into its inventory system so they can check product availability. Under Web services that means creating an inventory Web service and then creating and publishing the service description of that service to a registry like UDDI under the Publish, Find and Bind mechanism. The vendor of the procurement application queries that UDDI registry using the Find mechanism, gets the API and then uses that API to produce a request (the Bind mechanism) to send to an inventory where it gets the information it needs.
That is an approach that works brilliantly in the lab, but stumbles at the first hurdle as it hits the real world and the issues that organizations face in that real world, Boubez reiterates. "For instance, why would I expose my back-end systems on the Web? What about security, privacy, reliability?"
It is these issues, Boubez says, which make the potential of Web services a governance nightmare and make that policy abstraction layer so essential. "Typically, if you don't use that policy abstraction, that policy layer, you end up having your developers write the access controls into the code of the services. So you start with a particular project, a lot of times you have pilot projects, and you say: 'All right, I am going to make my inventory control or my inventory management system available to other applications through Web services'.
"So somebody builds that Web service and then they say: 'Well, wait a second, this can't just be open — we need to secure it, right?' So they go into the code of that service and they build some kind of access control. And if that access control mechanism changes again they will have to go back into the code of the service again.
"So now you have developers whose main task really is to resolve functionality or business logic, to actually start to build security and access controls into their applications. But much, much worse I believe is the fact that you are forcing developers on the other side of the requesting application — let's say one of your partners — to write code."
For example, Boubez says, an organization implementing vendor-managed inventory may want to give vendors access to its inventory controls. Suddenly, their developers are writing their security model into their own code, and if the organization wants to change its access control, those vendors will also have to make changes.
Page Break
"Let's say at the beginning you impose basic authentication type of credentials for them to present credentials to access your services, and you say basic authentication is not good enough any more. Now the Web service spec is out I want to use Web service security in the header. Now you go and change your code and then you force every one of your vendors to change the code. And that is only for one kind of application. Imagine if you actually rolled out a large number of services . . . if you rolled out a full service-oriented architecture into your organization. Now you've got developers writing code all over the place and changing it all the time and then breaking every one of your partners every time they make a change into that particular service. To me that is an absolute nightmare, and if I am a CIO that is the last thing that I want," Boubez says.
"The other much more important question in my mind is whatever happened to loose coupling."
Fortunately, Boubez says, a policy control infrastructure for SOA can be used to engineer loosely coupled SOA architectures capable of spanning security domains.
Under Web services the WSDL document is the interface or contract that defines the syntax of a service, describes an interface, defines the data types the consumer must provide when calling the service, and outlines what a consumer can expect to receive in return. The contract may incorporate some service semantics in comments embedded in the description or through the logical grouping of functionality — such as methods or operations — into a common service unit.
However, Boubez says although WSDL is a powerful technology, CIOs should recognize its limitations. WSDL simply is not adequate to convey these concepts, he says. WSDL is essentially an interface definition language (IDL) conveying API — necessary but insufficient.
"It could be argued that the 'D' in WSDL is not quite complete, since it only goes so far in describing access to a service." That is where policy, policy enforcement and the whole field of what is variously known as extensible markup language (XML) firewalls, XML gateways or Web services gateways come in, providing an enforcement point to handle all Web services traffic and leverage the existing infrastructure from access control to EDI management to PKI into the SOA, he says.
A policy is simply a set of constraints and capabilities that governs how a Web service and its consumers interact. According to Boubez, simple policies typically include rules describing:
• Who can access that service
• What kind of credentials are acceptable
• Whether encryption or signatures are required
• How messages get routed to the service
• What endpoint to use for a particular request
• What service level agreements (SLAs) are in place
• If there are any necessary transformations to be applied.
While the policy concept is a very prevalent and common requirement in every aspect of IT (for example, server configuration files), he says, it is so far non-existent in Web services.
Page Break
Boubez says policies are not the sole domain of service providers but should exist both for providers and consumers. As SOA implementations become more dynamic, policy agents for both the Web service provider and the consumer will be able to negotiate terms. This negotiation between policies should lead to the creation of a contract between two entities.
However, he also warns that while policy enforcement points can play a very important part in SOA, that is only half of the equation.
"Let's say your architects and developers are moving towards an SOA and implementing vendor managed inventory and you want to give access to your IT inventory management systems to your partners or to your vendors. If they provide a policy abstraction layer and maybe they buy an XML gateway or an XML firewall they can, using that XML firewall or XML gateway, go and create policies for their Web services, which is great. So now they are doing it right and they've taken all of that stuff out of the hands of developers and put it in that declarative, administrable layer.
"But that's only half of the equation and only solves half the problem, which to my mind is the least interesting half of the problem. There is still the issue for everybody who is writing applications that are going to require those services."
So even more important than policy enforcement is a concept known as "policy application points", Boubez says. CIOs should only go with vendors that offer such policy enforcement points, whether they call them XML firewalls, XML gateways or Web services gateways.
When the policy of a Web service is refactored into a formalized policy document, the WSDL can more flexibly bind to a policy at run-time. The policy and decision enforcement point works in conjunction with a policy registry, which takes care of sending the WSDL document to the client, he says, while leaving the client still coupled to the policy details.
The policy application point acts as a proxy for policy details and allows the client to focus on programming to the WSDL body. The flexibility of policy allows for more dynamic Web services based on the run-time needs of the client.
Boubez says the policy application point at the client side decouples the requester from the security policy requirements, in the same way that the policy enforcement point decouples the Web service itself. The policy application and enforcement points can exchange policy documents and coordinate at run-time to make the whole security mechanism truly loosely coupled.