We're at a major inflection point in the development of Web services; one that can be paralleled against the evolution of previous infrastructure technologies, including databases and programming environments. The next year will see certain vendors step forward to offer value-added components that will help fill in many of the gaps that currently exist with Web services, generating even more corporate interest than is seen today.
While some may be disappointed in the development of Web services to date, particularly when viewed against initial expectations, it's important to view the progress within the context of previous infrastructure technologies. In that context, Web services appear to be right on track.
Evolution
Web service-based integration is actually the evolution of an idea that's been around for 40 years - service orientation. As early as the 1960s, elementary integrations took place through the use of subroutines that called upon reusable processes. As I wrote in this magazine several months ago (WSJ, Vol. 2, issue 2), this concept has evolved over the years through structured programming, client/ server development, object orientation, components, and on to Web services today. Web services are essentially a technique for Web-based application-to-application integration, much the same way that client/server was for non-Web-based distributed computing.
Applications for the Web can be split into two camps - those for human-to-computer interactions and those for computer-to-computer applications. Web service-based integration is all about moving the second group of applications forward. It can do for system-to-system communication what HTTP and HTML did for the browser-based Web.
So, where are we today? All signs point to our being on the cusp of a significant turning point in the development of Web services. There is a standard pattern to the development of technology solutions. It begins with a problem being encountered in multiple situations, and solutions being developed on a one-off basis to address the specific challenge. Over time, entrepreneurs deduce that a generic solution can be developed to address the problem, and they create powerful, proprietary vendor platforms to solve the challenge. As the problem domain matures, standards develop, at which point a shake-out begins and the companies that survive are those that successfully do two things: first, they embrace the developing standards, and indeed demonstrate leadership in their development. Second, they continue to innovate at the top of the stack; in other words, in related areas where standards have not yet matured.
We saw this pattern develop in databases, where Oracle took the leadership position; more recently, we saw BEA take a leadership role in the J2EE application development space. The same pattern now appears to be emerging in the integration space. As with the other technologies, the first development phase of integration platforms began with companies experiencing a problem: they increasingly needed to integrate their standalone enterprise applications. Invoicing applications needed to talk to accounts receivable applications, and CRM applications needed to plug into existing customer data warehouses. Over time, powerful integration platforms sprang up to address heterogeneous application interoperability and integration. As the space matured and became better understood, we started to see the development of Web services standards around integration in a Web-based world.
As a result, we now have agreements on a set of Web services standards that address a few key interoperability issues - standards like Simple Object Access Protocol (SOAP) for sending Web service messages, and Web Services Description Language (WSDL) for providing a simple way to describe how to invoke a Web service on the data being exchanged.
The companies leading in the integration space at the moment are the companies that recognize the importance of these standards, are leading their development, and are incorporating them into their platforms. At the same time, these leaders are plugging the gaps in the standards by providing a complete stack of integration capability, and continuing to innovate at the "high end" of this stack.
Plugging the Gaps
While the Web services standards that have been accepted to date are an important step towards simplifying application connectivity, they address only a small portion of what is involved in true application integration. Using the image of an iceberg, Web services standards address a small number of issues above the water line (see Figure 1). The lower portion of the iceberg, hidden below the water, is far larger and ultimately more important.
Some of the issues not yet addressed by Web services standards include data transformation, adapters to other platforms, transactionality, business-process management, service monitoring, and security. Each of these issues merits a brief examination.
Data Transformation
While Web services standards make it easier to communicate with applications, they simply pass parameters back and forth in the same format and form. Integration platforms, on the other hand, have capabilities that automatically transform data to application- and machine-specific formats.
Adapters
Adapters are currently used to expose applications to the outside world. Web services are designed to do the same thing, and as standards are incorporated into major applications, adapters will become less necessary. But not all of the systems a company owns will be Web service enabled. In fact, the vast majority of organizations are populated with old mainframe systems, custom applications, and legacy databases that are not yet Web service compatible.
Transactionality
A company may create a Web service that allows its customers to change a "Ship To" address, but what happens if at the moment that information is being sent, a system failure occurs - a phone line goes down or a plug gets pulled. The customer has no way of knowing the data was not transferred. With integration platform technology, transaction controls guarantee the integrity of a transaction. There is no such provision in Web services standards.
Business Process Management
While work is certainly progressing in this area with initiatives such as BPEL4WS, provisions have not yet been built into Web services standards to manage business processes that span multiple applications and business partners, both behind the firewall and across the Internet.
Security
For communication inside or outside your company using a Web service, there is nothing built into the technology to protect against that communication being intercepted. Again, work is taking place in this area, but for now Web services standards do not address that area of security.
Service Monitoring
A company looking to use a Web service to run its business can't afford to be caught with an unreliable service. The way to protect against this today is by using sophisticated management technology to fix problems when they occur, and predict problems before they happen so issues can be resolved before they cause damage. At the moment, this is not addressed by Web services standards.
It's easy to see how all of these issues create the larger part of the iceberg, and can ultimately sink the ship. Consider the challenges involved in creating a Web service that leverages customer information. In most Global 2000 companies, customer information is stored in a wide range of systems, from mainframe and legacy systems and databases, to newer CRM and ERP systems. A company looking to build a Web service that accesses customer information might have to build up to a dozen or more connectors in order to tap into the complete customer record. As outlined earlier, there are a number of reasons why that can't be done with Web services standards today.
Vendors and members of the industry recognize that standards are just beginning to mature and are starting to address some of these issues, but we're not there yet. What that brings us to is the need for Web services-based integration platforms that support Web services fully, but also provide all the other elements that are not yet available through open standards mechanisms.
Extending Value
How do we get beyond mere interoperability, which is essentially what is provided by SOAP and WSDL, to true integration? Can we achieve an "Application Net," or the equivalent of a dial tone, where it might be possible to simply plug in and conduct business, just as we plug in for e-mail today? What are the barriers that need to be overcome to achieve true, ubiquitous integration? I see three fundamental areas to consider:
Technical infrastructure
For the most part, this is composed of the issues described above, including security, business process orchestration, and service monitoring. Excellent progress has been made in this area, including work on such standards as WS-Security, BPEL4WS, and OMI. What doesn't exist yet is a standard above these standards that is application centric at its core - in other words, a standard that defines the standards needed to have an enterprise class, trustworthy Web services-based infrastructure.
Application semantics
Every application, be it a custom application or a commercial application like SAP or Siebel, has a different way of representing the world. Interoperability does not imply semantic equality. In other words, if we solve the first barrier, technical infrastructure, we can get a dial tone. We know it's going to stay up, and we know other people cannot listen in on transmissions. But one person might still be speaking Russian and another Japanese. Numerous efforts have been undertaken over the years to achieve an overall standardized data model, but it hasn't happened yet, and it doesn't look like it will happen anytime soon.
Data models are the assets of the software vendors, and they understandably go out of their way to protect those assets. The future may very well lie in point-to-point collaborations between application vendors. But these collaborations are going to have to be standards based, or they'll simply be one-off collaborations.
Implementation-specific differences
Even if we solve the first two barriers, no packaged solution will ever be 100% complete. Customers will need to modify or customize the products they implement. Customization standards are needed that allow customers to modify or upgrade their platforms without having to rewrite collaborations that are built on the integration infrastructure. Over time, as applications are changed, the integrations and application collaborations will still work.
Until all three of these areas are addressed, complex one-off solutions will prevail.
The Task at Hand
The question, of course, is what can information technology managers do today to prepare for the future?
The first and most critical action is to get a service-oriented architecture into place. Web services are a means to integration, but the benefits cannot be realized if a services-based architecture has not been implemented in the first place. It sounds obvious, but in the rush to solve a specific business need, future technology considerations are often pushed to the side.
The second thing managers can do is start developing Web services-based integration components now, but at the same time remain pragmatic. Not everything can be, or should be, a Web service, particularly today when the standards are immature. On the other hand, it's clear that Web services are going to be the basis of integration in the future, and it will pay to begin laying the foundation.
The third action is to select vendors that lead and follow standards. The age of proprietary integration is over. As standards develop to address each of the underlying business issues, make sure your vendors offer support.
Companies are just beginning to experiment with Web services, developing prototypes and testing the technology in isolated corners of their business. The activity illustrates the general enthusiasm for Web services and the benefits it can afford corporations - from faster and less expensive integrations to the ability to reuse applications and the promise of widespread automation of business processes.
Rather than being viewed as a replacement for enterprise integration technologies, Web services are now correctly seen as a natural extension of integration technology. Companies will be able to achieve true enterprise class integration by using these new standards, while at the same time taking advantage of the strength, reliability, maturity, and scaleability that have already been incorporated into proven integration platforms.
It's true that Web services have not yet delivered to the degree that some have predicted or perhaps hoped, but that has to be considered within the context. The technology is simply following a similar development pattern to that taken by other important innovations. The fact is, the market is converging around the standards that have already been adopted, and work is proceeding on a number of fronts to address the barriers that exist today.
The next year could see important breakthroughs on a number of fronts, making this an exciting time indeed for the Web services community.
The new widgetry features multi-cluster suppo...
Subscribe to SOA World Magazine Email News Alerts
Subscribe via RSS
