Web Services Orchestration

Sep. 23, 2002 12:00 AM

In the past decade "workflow" has become one of the most overloaded terms in the software industry. Almost every application is tagged as "based on workflow." While this doesn't always mean a lot, there is good reason for it; it involves recognition among software architects that the business process is the application.

With the advent of Web services, workflow vendors and enterprise application integration (EAI) vendors are aligning themselves and often reinventing themselves to make full use of Web services and the inherent strengths of the asynchronous, loosely coupled software model. While Web services are powerful in and of themselves, the combination of Web services with a process-based approach is even stronger. This marriage of workflow with Web services is often termed Web services orchestration.

Orchestration is a relatively new term, but it's already being used in differing contexts. While Web service orchestration is usually used consistently at the 30,000-ft level, when you look at the implementation details, you find that there are meaningful differences in terms of product capabilities and even in philosophy. At this point, there's plenty of confusion surrounding the terminology and we're seeing the emergence of an acronym soup. This is demonstrated by commonly raised questions, such as, "Is EAI the same as workflow?"

This article provides a broad market survey and introduction to the major categories being described as Web service orchestration systems. The taxonomy we present will range from categories that are loosely related to Web services to those that depend heavily on Web services.

Workflow Systems
Many systems are based on process/workflow engines. It is readily accepted that workflow management is an important ingredient in flexible integration. The strength of workflow systems is that they abstract the essence of the application flows into a set of processes that can be easily modified to account for differences in business processes. Because these flows usually touch many systems, the management of the workflow has always been closely tied to the integration of the systems involved in the business process (see "XML Glue," XML-Journal, Vol. 3, issue 3).

Workflow is not a new concept. It has been around for over 20 years and has been well formalized by the Workflow Management Coalition (WfMC). The following definitions appear in the WfMC glossary:

Business process: A set of one or more linked procedures or activities that collectively realize a business objective or policy goal, normally within the context of an organizational structure defining functional roles and relationships

Workflow: The automation of a business process, in whole or part, during which documents, information, or tasks are passed from one participant to another for action according to a set of procedural rules

Workflow management system: A system that defines, creates, and manages the execution of workflows through the use of software running on one or more workflow engines, and that is able to interpret the process definition, interact with workflow participants, and, where required, invoke appropriate IT tools and applications

A workflow management system needs to support three types of features:

Build: A workflow management system needs to provide tools with which a business analyst can design and assemble a process definition comprising the activities that need to be performed, the roles that participate, and the flow between the activities based on a set of business rules. Figure 1 shows the process editor in use when deploying systems built upon ViryaNet's Workflow Engine.

Process engine: A workflow management system must include a runtime engine that manages the processes and walks each process through the set of activities specified by the process definition. The engine needs to perform the actions and evaluate the business rules to determine how the flow proceeds.

Runtime user interface: A workflow management system must include user interfaces through which people interact with the system. The workflows modeling the business processes almost always include a set of activities that require human intervention; the system must therefore manage a set of screens to allow this. In addition, the process engine should have a user interface component to allow users to monitor and control the engine and its processes. While workflow systems are not directly related to Web services, they are receiving a facelift due to the promise of Web services. The marriage between workflow engines and Web services is made by introducing a new type of step into a process (a step in which a Web service is called), and by allowing each process to be created or updated through a Web service. The orchestration capability is inherently provided by the workflow engine that manages the transitions within the state machine defined by a process.

Enterprise Application Integration (EAI)
EAI has developed from an evolution of approaches over the past two decades to deliver on the promise of enabling data sharing across applications and processes. Initially, database vendors pushed the enterprise data model and federated databases as a way to pull together data from diverse applications. Later on, ERP vendors focused on using a monolithic homogeneous system for integrating back-office functionality, such as manufacturing processes, human resources, and financial processes.

Recently, message-oriented middleware (MOM) has emerged as a means to route messages among diverse systems. MOM has now evolved into integration broker product suites (AKA EAI) to unify heterogeneous applications. Gartner estimates that EAI can reduce integration costs associated with implementing a packaged application by one third, and the cost of maintaining and making changes to such applications by two thirds.

EAI software suites are still evolving to address the complex requirements of application integration. These software suites include:

An integration broker: A set of services for message transformation and smart (e.g., content-based) routing

Application adapters: Off-the-shelf adapters for common applications (e.g., PeopleSoft or SAP R/3)

Development tools: A set of tools for specifying the transformation and routing rules to be executed by the broker

Adapter tools: A set of tools for building adapters into legacy applications

Administration tools: A set of tools for monitoring, security, etc.

Most EAI product suites are vertically integrated on top of MOM infrastructures and in recent years have added products to their suites, which focus on higher value-add functionality. These products include business process management (BPM), enterprise portals, B2B connectivity, and even e-commerce solutions (e.g., management of trading partners).

The advent of Web services hasn't been lost on EAI vendors, who see Web services as an opportunity to increase interoperability with other EAI solutions and streamline the complex and expensive task of creating custom connectors to legacy systems. Increasingly, these vendors are adding Web services support as well as business activity monitoring capabilities.

Web services play a particularly important role in extending the integration capabilities of the BPM tool within the EAI suite. They enable the EAI solution to expose business processes as Web services and to create new business processes from Web services. The BPM tool has the added capability of orchestrating business processes that incorporate Web services and coordinating Web services along with legacy systems and human-based workflow. In that view, orchestrating Web services within the context of a BPM tool as part of an EAI suite is not materially different from such orchestration within the context of workflow systems.

BizTalk Orchestration and XLANG
While BizTalk orchestration may be thought to fall into the same category as EAI-centric orchestration, Microsoft has made BizTalk orchestration, as well as Web services, so central to their architecture that this solution warrants its own category. BizTalk provides two core functions: EAI and orchestration services. Orchestration is not viewed as a subelement within the EAI facilities, and it supports advanced workflow features. In fact, we believe BizTalk orchestration is one of the best workflow engines out there today.

A BizTalk Server orchestration is a process created in the BizTalk Orchestration Designer, which is based on Visio, as shown in Figure 2. The process is serialized into an XLANG document - XLANG being an XML language that describes the process flow. Implementation services orchestrated by such a process can be any .NET component. Because the use of Web services is so prominent in VS.NET, ASP.NET, and in the .NET Framework, the use of BizTalk orchestration to compose Web service calls is becoming a dominant theme within the BizTalk community. Becoming - but not quite there yet. BizTalk orchestration provides a number of implementation shapes that are composed into a process, including COM components, script components, message queuing, and BizTalk messaging. While Web service calls may be wrapped within one of these components, there is no native Web services shape in BizTalk orchestration yet.

The next version of XLANG, called XLANG+, will have Web services as a native shape. Until then, the simplest alternative is to implement a COM or .NET component that invokes a Web service using SOAP, something that is very easy to do within the Microsoft tools.

IBM Web Services Flow Language
The Web Services Flow Language (WSFL) is an XML-based language used for describing the composition of Web services. A flow composition (also called orchestration or choreography of Web services) defines the execution sequence of the functionality provided by the composed Web services. WSFL fits neatly into the Web services stack above the Web Services Description Language (WSDL) and supports a recursive model by which orchestrations defined using WSFL can themselves be externalized as new Web services to support business processes hierarchies.

WSFL is an IBM specification, in many ways very similar to Microsoft's XLANG. In fact, a Gartner research note in May 2001 predicted that IBM and Microsoft would jointly submit a proposal to the W3C to combine XLANG and WSFL by year-end 2001. While this has not yet come to pass, WSFL is receiving a lot of industry backing. However, it's not getting the same backing from IBM that XLANG is getting from Microsoft. As an example, IBM offers the Web Services Process Management Toolkit at www.alphaworks.ibm.com/tech/wspmt. This is a toolkit for composing Web services, including them in a business process, and implementing them as business processes. It is based on MQSeries Workflow and uses the Flow Definition Language (FDL) instead.

Web Service Orchestration Server
Web services provide a number of benefits and additional capabilities to EAI tools that address the high-end segment of the business process integration problem. However, the adoption of Web services carries a bigger promise, which is to fundamentally change the economics of integration. The promise of Web services is that companies will be able to implement business processes cheaper and faster, align their integration efforts with internal application development - thereby utilizing development skills more effectively, and avoid duplication and operational overhead of their computing infrastructure.

The clean-slate approach to Web services is based on the realization that making Web services work is a two-step process. First you publish them, i.e., make the services available, and then you orchestrate them, i.e., coordinate the execution of multiple Web services into business processes. Orchestrating synchronous and asynchronous Web services into a collaborative or transactional business process is complex. The emergence of services as building blocks for delivering process-centric applications introduces new challenges throughout the development life cycle.

In particular, the synchronous request-reply programming model using fine-grained components is now giving way to a more flexible and reliable model called orchestration, based on asynchronous and nonlinear multistep interactions across loosely coupled service components. Orchestration entails a consistent set of infrastructure-level requirements that can be grouped into three major categories:

Coordination: Asynchronous conversations, parallel processing, event handling, transaction management, clustering, and scalability

Management: Administration, cancellation and change management, exception and timeout handling, version control

Activity monitoring: Business reporting, audit trailing, nonrepudiation

A Web Service Orchestration Server (WSOS) is a built-from-scratch piece of infrastructure software that reduces complexity by encapsulating the facilities needed for executing the orchestration's technical requirements.

Orchestration within the context of a WSOS differs from that within the context of EAI. WSOS is built from the ground up to extend the computing architecture defined by J2EE and XML Web services standards, while EAI takes the tools approach.

Put together, these innovations change the economics of integration by dramatically reducing the complexity, skill requirements, and total cost of integration relative to traditional EAI solutions. Similar to how J2EE and application servers emerged to deliver and manage fine-grained, tightly coupled Web applications, Web services and orchestration servers are now emerging to address the challenge of delivering and managing loosely coupled service-oriented business processes.

Summary
There is evidence that Web services impact the various approaches to integration and promote a continuing shift toward delivery of process-centric applications built from loosely coupled service components. This new paradigm induces a shift toward a service-oriented architecture in which orchestration becomes the primary concept for coordinating the execution of services for delivery of collaborative and transactional business processes. Web services promises to fundamentally change the economics of integration and reduce its perplexing complexity. The extent to which orchestration systems succeed in delivering on that promise and grab the mind share of the developer community, will ultimately determine the leaders in the rapidly evolving integration space.

Resources
Gartner. (2001). "Microsoft Continues Web Service Leadership With New XML Specs." www3.gartner.com/resources/98300/98366/98366.pdf

Related Links

▪ Figure 1

▪ Figure 2

About Ron Ben-Natan
Ron Ben-Natan is Chief Technology Officer at ViryaNet Inc. Prior to that he worked for companies such as Intel, AT&T Bell Laboratories, Merrill Lynch and as a consultant at J.P. Morgan. He has a Ph.D. in Computer Science in the field of distributed computing and has been architecting and developing distributed applications for over 15 years. His hobby is writing about how technology is used to solve real problems and he has authored numerous books including “IBM WebSphere Application Server: The Complete Reference” published by Osborne/McGraw. He can be reached at

About Doron Sherman
Doron Sherman is the CTO of Collaxa, Inc., a Web Service Orchestration Server vendor and a BEA Partner located in Redwood Shores, California. He has been involved with Java since its early days and pioneered application server technology while being a founder and chief scientist at NetDynamics.