In the IT world, integration became an issue as soon as the second computer with the second application came online. Many different approaches to solving the complex problems associated with integration have been tried since that time, some of them more successful than others. At this point it's safe to say that integration is still an expensive, usually difficult, aspect of every major IT infrastructure. The need to collaborate across multiple businesses as well as large geographical and cultural divides has only added to the list of issues.

Ten years ago, integration within an enterprise was typically built in one of two ways: by synchronizing data elements around a database or data warehouse, or by installing a complex, expensive ERP system and modifying every application to conform to that system's data model and interfaces. These types of integrations are commonly called "intelligent bridges." Communicating outside the enterprise with business partners was accomplished via EDI, again requiring expensive infrastructure, as well as a difficult mapping and synchronization exercise. The majority of companies still use one or both of these methods today.

Over time, middleware became available, usually based around a messaging backbone that used its own proprietary interface and protocols for transporting data. The data itself was typically in a format that was defined by the company or the departments that needed to exchange it, and again the data went through complex mappings and transformations to and from the databases and applications that shared it.

Standards
Eventually, standards initiatives came along. Groups such as OAG and OMG worked on standards for representing and transporting data. Huge steps were made in simplifying information access, mapping, and transformation. However, these early standards were rigid and tightly coupled, and didn't allow for the constant changes that are always present in business environments. This made implementations difficult and maintenance costly.

We are now in an era where more interoperable, flexible standards have started to emerge and mature: XML has become the prevalent way of representing shared data, XSLT and XPATH for translation and business logic, WSDL and SOAP for application definition and access to services, BPEL and BPML for business process management. Of course, some of these standards have a long way to go, and there are numerous competing standards in many areas.

Despite their presence, currently no suite of standards addresses the complete set of integration and interoperability problems that exist. However, the great news about standards is that they allow these problems to be broken down and addressed piece by piece. It is now possible to separate application logic from a canonical data model, and both of these from the business process workflow. Applications can have a consistent interface, allowing them to be versioned, or swapped out wholesale for other applications. Data models can be enhanced or modified without changes to the applications that interact with them. Business workflows can be enhanced or configured without changes to the applications that they depend on for their extensive, complex business logic.

A few commercial integration solutions have become available during the last few years that leverage these standards. Through these solutions, an important evolution has started to emerge. The economic disadvantage of building and maintaining an ever increasing number of point-to-point integration solutions within an enterprise became obvious, and the move to building an integration backbone became a siren call for forward-thinking CIOs. Having an integration backbone in turn allowed corporations to achieve many of the benefits that have been promised by believers in packaged integration solutions and the standards that they leverage.

But there is an even more significant outcome from this maturity of integration standards and solutions. The evolution described here allowed the industry to shift its focus from gluing a few applications together and syncing them with a database to focusing instead on the business processes that drive financial operations, supply chains, health care practices, retail transactions, and so on.

The first software vendors to grasp this tectonic shift in focus were the infrastructure players. New companies emerged that focused on the business process management layer. Integration vendors either enhanced their products to include business process and workflow support, or they acquired business process-focused solutions.

Then a consolidation took place among these groups. When the dust settled, all serious integration vendors had solutions that empowered business users to design business processes and their logical flows, independent of the application components that supported these details. Further, they were able to tie these processes into the user interaction, deploy them in a distributed fashion, and manage them in a consistent manner.

Meanwhile, standards have continued to evolve and mature to support this new way of thinking. BPEL4WS has been touted with great fanfare by its founders - IBM, Microsoft, and BEA - as the language that should be used to describe the execution of business processes. Another serious contender, BPML, is a less commercial effort that has been well thought out and endorsed by many. In fact, BPEL is currently mired in intellectual property rights issues that, if not resolved soon, will virtually ensure BPML's growth and further acceptance.

Integration
All of this focus on the business process has led to a change in how commercial vendors approach the problem of integration. Instead of touting the ability of customers to join applications together for data sharing and synchronization, the new mantra is the ability to provide business processes on top of a service-oriented architecture, usually broken down by industry verticals. Vendors provide a huge head start in deployment because the large, common piece of the business process has been predeveloped. All that remains to be done is to extend or customize the process for the business practices specific to the deployment.

Systems integrators were the first to jump on this idea. The expertise they gained over the years from implementations of business processes using their own methodologies, combined with their expertise in bringing together best-of-breed composite applications, led to a compelling value proposition. In fact, some of the major firms now have architectural frameworks to leverage business processes that harness heterogeneous components. Examples are the CSC e3 framework, and BearingPoint's Solution Integration Architecture.

Of course, integration software vendors have also seized this opportunity. Early attempts (such as CrossWorld's work in business process modeling) failed because they were too far ahead of the maturity curve of the supporting standards, and lacked the credibility of a larger player or a more mature market. In the last year webMethods, Tibco, and IBM have all gained footholds in the business process-centric approach.

Perhaps the largest group to latch onto business process integration is enterprise system vendors. The first was J.D. Edwards, with its XBP/XPI approach. SAP developed its version, known as xApps, on top of its NetWeaver platform.. PeopleSoft has AppConnect, and the most recent, and probably most publicized, is Siebel's UAN initiative.

Each vendor approaches the solution in a slightly different way. SAP and PeopleSoft, for the most part, have built all of the underlying technology themselves as an extension of the latest generation of their core infrastructure and middleware. At this point, they both still outsource adapter support. JD Edwards, on the other hand, embeds webMethods technology as their XPI infrastructure, then builds their XBPs, or Cross Business Processes, using webMethods middleware. Siebel has decided that, to achieve true openness and interoperability they need to define platform-neutral business processes using standards, and then work with multiple vendors, including, webMethods and Tibco, to create packaged solutions to deploy these processes on their respective platforms.

Whether employing business processes through the offerings of integration vendors, system integrators, or enterprise software vendors, a key set of requirements must exist in order to be successful:

Finally, a service-oriented integration infrastructure for the design, development, extension, deployment, maintenance and management of the processes must be in place.

This integration infrastructure needs to leverage the decoupling that was discussed earlier. It needs to encourage and leverage the separation of the process flow from the human interaction from the data documents and their associated representation and translation. It also must be able to instantiate business processes that were defined without the end implementation and deployment in mind. And last, it must facilitate the management of multiple business processes executing simultaneously across a distributed environment.

Summary
Management of the business process has become the next frontier. Vendors are rushing to make the real-time enterprise a reality. The end goal is to be able to identify a set of critical metrics associated with a business process, and to track and measure performance against these metrics as part of the day-to-day execution of the process. This should happen as near to real time as possible, allowing an end user to react to any anomalies as soon as they occur. It requires the ability to capture information as it flows through business processes, analyze it against historical information, and propagate any red-flag issues through the proper channels.

What's next for business processes? In a word, maturity. As is typical in our industry, the hype runs well ahead of the reality. Implementation of business processes at this point is still a very professional services-oriented effort, and very few canned solutions (beyond simple data synchronization) from the enterprise vendors have been more than 70% repeatable. Infrastructure and middleware will continue to mature and embrace open standards as the standards themselves mature. More and more applications will expose a service layer. And the marriage of components required for the real-time enterprise will continue.

The notion of the business process leveraging a set of underlying heterogeneous application services is here to stay. The focus has moved to this top-down model, rather than the bottom-up approach of wiring applications and databases together point to point. Integration initiatives as we knew them have, in fact, been turned upside down.

Published Apr. 22, 2003 — Reads 15,536
Copyright © 2008 SYS-CON Media. All Rights Reserved.

About Jim Mackay
Jim Mackay serves as senior vice president of webMethods' Business Technology Group, a group within webMethods Global Alliances that serves the technical requirements of the ever-growing number of strategic partnerships signed by the company. Previously, Jim served as CTO of i2 Technologies, where he was responsible for guiding their assessment and creation of technologies and standards to continually improve their market-leading value chain management solutions.