Welcome!

Microservices Expo Authors: Liz McMillan, Pat Romanski, Carmen Gonzalez, Elizabeth White, Jason Bloomberg

Related Topics: Microservices Expo

Microservices Expo: Article

Beyond Point to Point

WS-Eventing and content-based routing for integrating multiple applications via Web services

Web services have emerged as an excellent method of integrating pairs of applications. Free and cheap Web services development tools from many different vendors make it easy to expose one application's capabilities to other applications that wish to invoke them. But, given recent trends and innovations in Web service standards for more complex integrations of multiple applications from many parties, integrating applications two-by-two with tightly coupled simple Web services may not be the best approach.

In this article, I'll take a look at a reasonably complex integration scenario performed by integrating with many different applications' Web services. I'll show you how the architecture of the integration is simplified by using XML content-based routing with a publish/subscribe approach. In the remainder of the article we'll assume that we are using a content-based routing service based on the recent WS-Eventing specification from Microsoft, Tibco, and BEA. The XML document message flow in a WS-Eventing-based approach to the problem is shown. Finally we'll show the code necessary from one of the endpoint applications to generate and process these messages.

One Scenario
A consumer electronics retailer decides to carry a new consumer electronics product. The "product master catalog" might invoke a Web service exposed by the inventory system to create an entry to track the inventory of the product, then call another Web service that performs integration with one supplier of the product to request some product for delivery. A service on the accounts payable system is called to create a new account for the vendor to ensure that the supplier is paid when the invoice is received.

After deciding that they want to carry the product on their direct Web site, the Web site product catalog is updated with the new product's presence, also via a Web service. They then need to invoke Web services to communicate with each of their stores about the impending arrival of the new product, so that the stores can plan stocking and shelf space for it, and to let them know about the initial allocations of the product.

The product marketing group uses the product's category to make Web services calls to the company's cross-marketing software that determines correlated products that are likely to be bought with it. Then they call services on other suppliers to get quotes on those products for consideration of starting the new product stocking process once again.

Figure 1 attempts to show this integration as a diagram. Each edge is a Web service of a system that the "master catalog" application needs to integrate with. Notice that they each have their own names, indicating a different Web service that must be integrated with. There are quite a few independent Web services integration efforts required just around this one business event - the addition of a new product.

A Better Way: XML Content-Based Routing
With the emergence of standards focused on publish-subscribe messaging such as WS-Eventing and WS-Notification, and a coincidental trend toward more coarse-grained, document-style payloads of Web services, I think there is a better way to do this integration and still have the advantages of simplicity, standards support, and ease of development that are the hallmarks of Web services projects.

Publish-subscribe approaches to integration are, of course, nothing new. Before Web services came about, an integration consultant presented with the scenario above might well have architected the integration as publishing the "new product" event out to a messaging bus to a "new product" topic. The other systems that needed to know about new products would subscribe to that topic. Many projects based on software from Tibco, IBM's MQSeries, and Java Messaging Service implementations from vendors such as Sonic Software and Fiorano Systems were targeted at similar scenarios.

However, there is an opportunity to perform such integration using pure Web services standards and nothing but XML content. In the scenario described here, the master product catalog application generates a NewProduct XML message. The company's accounts payable application and inventory applications are subscribed to all instances of the NewProduct document. However, other subscription rules vary between these applications. There is content in the NewProduct message that indicates that the product will be sold on the company's Web site; hence, the Web site catalog application subscribes only to that content. Other content in the document indicates which stores will carry the product. Each retail store thus subscribes only to the NewProduct documents that are relevant for it.

Suppliers subscribe to NewProduct documents for products that they carry. Note that the decision to use a publish-subscribe paradigm for some parts of the Web service message exchange (such as informing all of these applications about the NewProduct) doesn't imply that everything has to be done that way on a given project or message choreography. For example, suppliers may call back to the master product catalog application by invoking a ProvideQuote Web service operation exposed by that application. Other suppliers of accessory goods can call the ProvideQuote Web service on related products to the original product.

WS-Eventing Message Exchange
WS-Eventing is a specification proposed in January 2004 by Microsoft, Tibco, and BEA (you can see the spec itself at ws-eventing.asp). It is a general-purpose mechanism for Web services to send messages to register interest in certain events. Those Web services then expect to receive messages about those events. In the terminology of WS-Eventing, "event sinks" create "subscriptions" by sending messages to "event sources." Those event sources then send "notifications," all documents that match the subscription criteria, back to the event sinks.

WS-Eventing is not particularly targeted to content-based routing per se. It can be used to express interest in system events such as servers being down or printers being out of paper. Nevertheless, it has just about all of the messages that we would want for XML content-based routing in a relatively simple specification that makes intelligent use of existing Web services standards. Because of the need for content-based routing for scenarios like the one above, and the historical availability of other non-Web services-based publish-subscribe solutions, WS-Eventing-based, content-based routing systems will probably emerge quickly. In this article I'll show you some code samples against an implementation of WS-Eventing that my company has released for demonstration purposes only.

To make the scenario more concrete I'll show you how a WS-Eventing-based, content-based routing service can make that integration much simpler. Specifically, we'll look at how the publishing master catalog application and various subscribing enterprise applications in the example just described can use WS-Eventing's Subscribe, SubscribeReponse, and notification messages for more efficient and direct integration.

Subscribe Messages
Each of the applications that need to be informed about the addition of new products subscribes to the NewProduct document. For example, the inventory application may be interested in all NewProduct documents. To express that, the application would send the message shown in Listing 1 to the content-based routing service.

This message is sent by the inventory application to the Electronics Retailer content-based router service. The address of the CBR service is http://electronicsretailer.com/cbrservice as expressed in the WS-Addressing To header. Although the WS-Eventing specification implies that the To header is required, SOAP stacks with WS-Addressing support are few and far between. So the WS-Eventing-based CBR service that we use in this code example (an alpha implementation from Systinet) can be connected to via basic SOAP over http and does not require that the wsa:To header be present.

The WS-Addressing ReplyTo header is used to indicate the location for the CBR service to deliver SubscribeResponse messages to. The contents of the WS-Addressing MessageID header sent in the subscription request are placed by the CBR service (or any WS-Eventing-compliant event source) in the subscription response RelatesTo header.

Looking at the elements in the body of the Subscribe request, the NotifyTo element contains the address where the inventory service wants to receive notifications, or more specifically, the NewProduct documents specified in the Filter element. The address is http://electronicsretailer.com/inventory/newproducts.asp, which presumably will perform the processing necessary on any NewProduct documents received.

Following the WS-Addressing specification, the contents of ReferenceProperties element will be sent as a header in any notifications that are sent in response to this subscription. In this example, <eri:LocalSubscriptionID>1234</eri: LocalSubscriptionID> will be placed in the headers of any NewProduct documents that are sent to the inventory application. Depending on the application that is handed the document, the identity of the subscription may be useful in the processing of received notifications.

The Filter element allows the subscriber to identify which notifications they want to receive. If not specified, the default dialect for expressing the filter is XPath (www.w3.org/TR/1999/REC-xpath-19991116). The dialect for the Filter can also be specified as an attribute to another one that the event source supports (e.g., XQuery perhaps). The example above uses the following XPath expression:

/s12:Envelope/s12:Body/er:NewProduct

to specify that the subscriber wants to receive all NewProduct documents. The expressions can also be used by subscribers to restrict message delivery to only those that match certain criteria on the content of the document. For example the expression below would result in the subscriber receiving all NewProduct documents that have a Price subelement with a value greater than 100.

/s12:Envelope/s12:Body/er:NewProduct/er:Price > 100

SubscribeResponse Messages
The content-based routing service will send SubscribeResponse messages to each subscribing application. The SubscribeResponse message that the CBR service sends is shown in Listing 2.

According to the WS-Addressing specification, the WS-Addressing To header has the same contents as the WS-Addressing ReplyTo header in the original request. The RelatesTo header contains the contents of the Subscribe request MessageID header. In the body, the ID element contains a unique ID for the subscription generated by the event source, in this case the content-based routing service. The ID element can be used by the subscriber, the inventory application in this case, to maintain the subscription for example, for example, unsubscribing or renewing a subscription.

Notifications
Listing 3 is an example of NewProduct messages as sent by the CBR service to the subscribing inventory application.

Subscription Ends, Renews, and Unsubscribes
Several other messages may be used to optimize the subscription process. If a subscription expires, the CBR service will send a SubscriptionEnd header block in either one of the notification messages (the NewProduct document in this example) or attached to an otherwise empty message.

Although subscriptions will eventually expire based on the requested timeframe of the subscription or the event source's default expiration policy, if an application decides that it is no longer interested in receiving notifications, it should send an Unsubscribe message to the content-based router service or other event source. It should receive an UnsubscribeResponse in reply. If an application knows that its subscription is about to expire then it can send a request to renew the subscription, via a Renew message, and should receive a RenewResponse message in reply.

The CBR service alpha used in this example does not yet support these maintenance messages, so we won't discuss them here. Refer to the WS-Eventing specification for more details.

Code Sample
Listing 4 shows a functional C# program that sends WS-Eventing Subscribe messages to a content-based routing service (it is also downloadable along with the entire C# project at http://adamblum.com/CBRSubscribe.zip). You can simply invoke this program with arguments of:

  • The endpoint to send notifications to
  • The XPath expression of the documents to subscribe to
  • The URL of the content-based router service

After the subscription is established, you can create Web services with any Web service development tool. If they send documents to the content-based routing service address (e.g., http://localhost:6060/cbr) that matches the expressions described in the CBRSubscribe program invocation, those XML documents will be forwarded by the CBR service to the notification endpoint identified in the CBRSubcribe invocation. If you want easy generation of these Web services clients, you may wish to start with an existing WSDL file. Just change the port in the WSDL to use the address of the content-based routing service. Your Web service client will then send XML messages corresponding to your WSDL to the CBR service for possible routing. Refer to the tutorials of your favorite Web services development tool for more information how to create generic Web services client using the WSDL of your choice.

Working More with Content-Based Routing
A functional WS-Eventing-based content-based routing service is posted and downloadable at www.systinet.com/contentbasedrouting. Addresses to a hosted and operational content-based routing service may also be posted there as well. Also, more full-fledged tutorials expanding on this scenario are available there.

Summary
XML content-based routing provides a very efficient way to integrate multiple applications around individual business events. While integration platforms that offer publishing and subscription to topics have enabled this for some time, the advent of WS-Addressing and WS-Eventing brings such efficiencies to the world of Web services. Companies that wish to integrate many applications simultaneously should consider using WS-Eventing-based integration capabilities to perform centralized yet Web services-based integration.

References

  • WS-Addressing by Adam Bosworth et al: WS-Addressing by Adam Bosworth et al: March 2003.
  • WS-Eventing by Cabrera et al: WS-Eventing by Cabrera et al:, January 2004.
  • More Stories By Adam Blum

    Adam Blum is CEO of Rhomobile. He came from Good Technology and while spending millions on enterprise mobile application development he realized there was a need for a framework for enterprises to build mobile applications easily and cost-effectively empower their workforce without training their programmers to learn different programming languages and building apps from scratch. He has spoken at Interop in Las Vegas and at Ruby events all over the world.

    Comments (0)

    Share your thoughts on this story.

    Add your comment
    You must be signed in to add a comment. Sign-in | Register

    In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.


    Microservices Articles
    Modern software design has fundamentally changed how we manage applications, causing many to turn to containers as the new virtual machine for resource management. As container adoption grows beyond stateless applications to stateful workloads, the need for persistent storage is foundational - something customers routinely cite as a top pain point. In his session at @DevOpsSummit at 21st Cloud Expo, Bill Borsari, Head of Systems Engineering at Datera, explored how organizations can reap the bene...
    "NetApp's vision is how we help organizations manage data - delivering the right data in the right place, in the right time, to the people who need it, and doing it agnostic to what the platform is," explained Josh Atwell, Developer Advocate for NetApp, in this SYS-CON.tv interview at 20th Cloud Expo, held June 6-8, 2017, at the Javits Center in New York City, NY.
    The Jevons Paradox suggests that when technological advances increase efficiency of a resource, it results in an overall increase in consumption. Writing on the increased use of coal as a result of technological improvements, 19th-century economist William Stanley Jevons found that these improvements led to the development of new ways to utilize coal. In his session at 19th Cloud Expo, Mark Thiele, Chief Strategy Officer for Apcera, compared the Jevons Paradox to modern-day enterprise IT, examin...
    In his session at 20th Cloud Expo, Mike Johnston, an infrastructure engineer at Supergiant.io, discussed how to use Kubernetes to set up a SaaS infrastructure for your business. Mike Johnston is an infrastructure engineer at Supergiant.io with over 12 years of experience designing, deploying, and maintaining server and workstation infrastructure at all scales. He has experience with brick and mortar data centers as well as cloud providers like Digital Ocean, Amazon Web Services, and Rackspace. H...
    Skeuomorphism usually means retaining existing design cues in something new that doesn’t actually need them. However, the concept of skeuomorphism can be thought of as relating more broadly to applying existing patterns to new technologies that, in fact, cry out for new approaches. In his session at DevOps Summit, Gordon Haff, Senior Cloud Strategy Marketing and Evangelism Manager at Red Hat, will discuss why containers should be paired with new architectural practices such as microservices ra...
    In his session at 20th Cloud Expo, Scott Davis, CTO of Embotics, discussed how automation can provide the dynamic management required to cost-effectively deliver microservices and container solutions at scale. He also discussed how flexible automation is the key to effectively bridging and seamlessly coordinating both IT and developer needs for component orchestration across disparate clouds – an increasingly important requirement at today’s multi-cloud enterprise.
    The Software Defined Data Center (SDDC), which enables organizations to seamlessly run in a hybrid cloud model (public + private cloud), is here to stay. IDC estimates that the software-defined networking market will be valued at $3.7 billion by 2016. Security is a key component and benefit of the SDDC, and offers an opportunity to build security 'from the ground up' and weave it into the environment from day one. In his session at 16th Cloud Expo, Reuven Harrison, CTO and Co-Founder of Tufin, ...
    DevOps is often described as a combination of technology and culture. Without both, DevOps isn't complete. However, applying the culture to outdated technology is a recipe for disaster; as response times grow and connections between teams are delayed by technology, the culture will die. A Nutanix Enterprise Cloud has many benefits that provide the needed base for a true DevOps paradigm. In their Day 3 Keynote at 20th Cloud Expo, Chris Brown, a Solutions Marketing Manager at Nutanix, and Mark Lav...
    Many organizations are now looking to DevOps maturity models to gauge their DevOps adoption and compare their maturity to their peers. However, as enterprise organizations rush to adopt DevOps, moving past experimentation to embrace it at scale, they are in danger of falling into the trap that they have fallen into time and time again. Unfortunately, we've seen this movie before, and we know how it ends: badly.
    TCP (Transmission Control Protocol) is a common and reliable transmission protocol on the Internet. TCP was introduced in the 70s by Stanford University for US Defense to establish connectivity between distributed systems to maintain a backup of defense information. At the time, TCP was introduced to communicate amongst a selected set of devices for a smaller dataset over shorter distances. As the Internet evolved, however, the number of applications and users, and the types of data accessed and...