Microservices Expo Authors: Liz McMillan, Pat Romanski, Carmen Gonzalez, Elizabeth White, Jason Bloomberg

Related Topics: Microservices Expo

Microservices Expo: Article

SOA and the Art of Riding the Enterprise Service Bus

The need for the ESB and other infrastructure technologies

In this article (part 1 of two), I will discuss the role of the enterprise service bus (ESB) and other technologies in providing an infrastructure for service-oriented architecture (SOA) in the enterprise. This month, I'll discuss some of the middleware requirements necessary to support a full-function SOA. These requirements are driven both by the concepts of SOA, and by the need to apply them in enterprise situations. Some of the important middleware capabilities that fulfil these requirements are now characterized as an "enterprise service bus." By examining the requirements, it is possible to identify some key mandatory and optional capabilities for an ESB. These include functions such as data format or protocol translation, support for open standards such as Web services and XML, and the provision of qualities of service such as performance, security, and transactionality.

Requirements for the Enterprise Service Bus
The need for an ESB results from both the concepts of SOA and the practicalities of applying it in an enterprise. In this section, we briefly discuss some of those factors.

Decoupling the Consumer View of Services from Their Implementation
Service-oriented architecture at its broadest describes:

  • Viewing business as the delivery of services to consumers
  • Decoupling the packaging and delivery of those services from their implementation
  • Implementing those services as business processes that use a highly flexible combination of other business and technical services
  • Applying this view to the supporting IT systems both by directly modelling and mapping services and processes to them, and by implementing interactions between systems as services
An example may help to make these points clear. Imagine the fictitious "Wildcat" luxury car manufacturer that offers a "roadside assistance" service to its customers. The package includes roadside and home assistance, vehicle recovery, the provision of replacement cars, and covers the cost of temporary accommodation. However, Wildcat is not expert in any of these areas, so they subcontract delivery:
  • Roadside assistance: To a roadside assistance specialist
  • Vehicle recovery: To local garages with towing facilities
  • Replacement cars: To a car rental company
  • Temporary accommodation: To a motel chain.
Wildcat therefore offers a comprehensive service to their customers without delivering any elements of that service themselves. They can subcontract to the lowest-cost or highest-standard service provider in each area. If a new provider becomes available, offering lower cost or higher standards, Wildcat can switch to them. If Wildcat needs to offer a new package of slightly different services to some of their customers, it is easy for them to do so.

The principles of SOA apply to this example, but could also extend it in several ways:

  • Automating the provision of business services and the aggregation of service suppliers. As described, the sourcing and aggregation of services could take place through face-to-face negotiation, telephone calls, the exchange of faxes and e-mails, etc. SOA describes the automation of these activities so that services are increasingly located, selected, and bound through interactions between systems rather than people.
  • Applying the same principles farther down the technology stack to achieve flexibility in the way technology is applied to support the business. The ability of any business to change its services in response to marketplace demands is constrained by the agility of the IT systems that support those services. By treating IT capabilities as services and providing similar decoupling and flexibility, SOA seeks to improve the overall agility of a business.
  • Delivering outsourcing or resourcing of business services at various levels of granularity: In the Wildcat example, we describe the outsourcing of relatively large-grained business services such as replacement cars. By extending this approach to the technology stack, SOA enables more flexible outsourcing. This might be in outsourcing finer-grained services, such as payments, credit checks etc.; or in more dynamically outsourcing large-grained services - for example, different hotel chains could be used to provide emergency accommodation depending on current discount programmes, geographical area, etc.
In this way, we see that the decoupling of the consumer view of services from their implementation is of critical importance to an agile business; SOA enables consumers to exploit services through a specific delivery channel and brand without binding directly to the eventual provider of the service.

To fulfill this decoupling, we need to introduce the concept of intermediaries. Intermediaries publish services to consumers as what we will call "facade services" - the consumer binds to a facade service through an intermediary, with no direct coupling to the eventual provider of the service. It is the role of the intermediary to map the facade service to an implementation of the service, which may be another facade presented by another intermediary, or may be the actual implementation of the service. Figure 1 illustrates the provision of facade services by an intermediary, this time in a banking scenario.

One of the roles of the ESB is to provide this intermediary function, particularly within an organization. In this way, it increases the internal agility of an organisation to combine its capabilities into services to offer to its customers. However, as we will see later, the same ESB capabilities can be applied in different scenarios, such as the "ESB Gateway," to provide this flexibility in offering services to consumers.

Decoupling Technical Aspects of Service Interactions
While most definitions of SOA describe services as "loosely coupled," in order to implement real systems we need to be more specific: we can't "loosely couple" a service (or any other interaction between systems); rather, we have to think carefully about how we couple or decouple aspects of service interactions, such as those shown in Figure 2, and including:

  • The platform and language in which services are implemented, or from which they are requested
  • The communication protocols used to invoke services
  • The data formats used to exchange input and output data between service consumers and providers
We also want to handle some of the more difficult technical aspects of transactions outside application code; at the same time, applications can't ignore them - ideally we'd like to declaratively specify them, similar to the J2EE model of deployment descriptors, and have the infrastructure take care of them. This might apply to such aspects of interactions as:
  • How service interactions are secured
  • How the integrity of business transactions and data is maintained through reliable messaging or the use of transaction monitors or compensation techniques
  • The invocation of alternative service providers in the event that the default provider is unavailable
In this way, it increases the internal ability of an organization to combine its capabilities into services, and allow those services to be invoked by consumers within the organiaation. However, as we will see later, the same ESB capabilities can be applied in different scenarios, such as the "ESB Gateway," to provide this flexibility in offering services to external consumers. Therefore, some of the capabilities that are required in an ESB are to:
  • Map service requests from one protocol and address to another
  • Transform data formats
  • Support a variety of security and transactional models between service consumers and service providers, recognizing that consumers and providers may support or require different models
  • Aggregate or disaggregate messages
  • Support communication protocols between multiple platforms with appropriate qualities of service
  • Provide messaging capabilities such as message correlation and publish/subscribe to support different messaging models within an SOA, such as events, asynchronous request/response, etc.
Integrating and Managing Services in the Enterprise
The use of basic communication technologies or Web services may solve individual problems, but if unchecked, it will result in a point-to-point integration style that will quickly become unmanageable. Consider Figure 3, which depicts the basic use of SOAP/HTTP or other technologies appropriate to SOA to perform integration. We quickly enter a complex, uncontrolled scenario with multiple security and transaction models, routing control distributed throughout the infrastructure, and probably no consistent approach to logging, monitoring etc.

The addition of a simple routing capability to this picture provides a first overall point of control. Additional capabilities (such as security, store and forward capability to provide delivery assurance, etc.) may be required in some scenarios. Note that this simple routing capability, while it appears to be a hub-and-spoke approach, in fact is consistent with a distributed bus model because we have provided a consistent management and administration model to an infrastructure that may be physically distributed (see Figure 4).

Another consideration in a realistic enterprise scenario is the need to integrate a potentially large number of heterogeneous service providers (such as CRM, ERP, legacy systems, or modern distributed applications using application servers like IBM WebSphere Application Server or Microsoft's.NET platform). It is likely that each of these will have their own integration techniques, protocols, security models, etc. Ideally, we don't want to expose this complexity to service consumers - we need to offer them a simpler model. To achieve this, the ESB is required to mediate between the multiple interaction models understood by service providers and the simplified view provided to consumers.

In my next article I will go on to describe some specific situations in which an ESB might be deployed and discuss the ESB capabilities in relation to other architectural components that support SOA. Finally, I'll briefly consider some of the technology options for implementing the ESB.

More Stories By Rick Robinson

Rick Robinson is an IT architect in IBM Software Services for WebSphere, based at the Hursley Laboratory in the United Kingdom. He has seven years of experience in the IT industry, and his roles have included solution architecture, design, and development. Rick holds a PhD in the physics of superconducting devices from the University of Birmingham. His areas of expertise include distributed systems design, the WebSphere platform, and service-oriented architecture.

Comments (0)

Share your thoughts on this story.

Add your comment
You must be signed in to add a comment. Sign-in | Register

In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.

Microservices Articles
Modern software design has fundamentally changed how we manage applications, causing many to turn to containers as the new virtual machine for resource management. As container adoption grows beyond stateless applications to stateful workloads, the need for persistent storage is foundational - something customers routinely cite as a top pain point. In his session at @DevOpsSummit at 21st Cloud Expo, Bill Borsari, Head of Systems Engineering at Datera, explored how organizations can reap the bene...
"NetApp's vision is how we help organizations manage data - delivering the right data in the right place, in the right time, to the people who need it, and doing it agnostic to what the platform is," explained Josh Atwell, Developer Advocate for NetApp, in this SYS-CON.tv interview at 20th Cloud Expo, held June 6-8, 2017, at the Javits Center in New York City, NY.
The Jevons Paradox suggests that when technological advances increase efficiency of a resource, it results in an overall increase in consumption. Writing on the increased use of coal as a result of technological improvements, 19th-century economist William Stanley Jevons found that these improvements led to the development of new ways to utilize coal. In his session at 19th Cloud Expo, Mark Thiele, Chief Strategy Officer for Apcera, compared the Jevons Paradox to modern-day enterprise IT, examin...
In his session at 20th Cloud Expo, Mike Johnston, an infrastructure engineer at Supergiant.io, discussed how to use Kubernetes to set up a SaaS infrastructure for your business. Mike Johnston is an infrastructure engineer at Supergiant.io with over 12 years of experience designing, deploying, and maintaining server and workstation infrastructure at all scales. He has experience with brick and mortar data centers as well as cloud providers like Digital Ocean, Amazon Web Services, and Rackspace. H...
Skeuomorphism usually means retaining existing design cues in something new that doesn’t actually need them. However, the concept of skeuomorphism can be thought of as relating more broadly to applying existing patterns to new technologies that, in fact, cry out for new approaches. In his session at DevOps Summit, Gordon Haff, Senior Cloud Strategy Marketing and Evangelism Manager at Red Hat, will discuss why containers should be paired with new architectural practices such as microservices ra...
In his session at 20th Cloud Expo, Scott Davis, CTO of Embotics, discussed how automation can provide the dynamic management required to cost-effectively deliver microservices and container solutions at scale. He also discussed how flexible automation is the key to effectively bridging and seamlessly coordinating both IT and developer needs for component orchestration across disparate clouds – an increasingly important requirement at today’s multi-cloud enterprise.
The Software Defined Data Center (SDDC), which enables organizations to seamlessly run in a hybrid cloud model (public + private cloud), is here to stay. IDC estimates that the software-defined networking market will be valued at $3.7 billion by 2016. Security is a key component and benefit of the SDDC, and offers an opportunity to build security 'from the ground up' and weave it into the environment from day one. In his session at 16th Cloud Expo, Reuven Harrison, CTO and Co-Founder of Tufin, ...
DevOps is often described as a combination of technology and culture. Without both, DevOps isn't complete. However, applying the culture to outdated technology is a recipe for disaster; as response times grow and connections between teams are delayed by technology, the culture will die. A Nutanix Enterprise Cloud has many benefits that provide the needed base for a true DevOps paradigm. In their Day 3 Keynote at 20th Cloud Expo, Chris Brown, a Solutions Marketing Manager at Nutanix, and Mark Lav...
Many organizations are now looking to DevOps maturity models to gauge their DevOps adoption and compare their maturity to their peers. However, as enterprise organizations rush to adopt DevOps, moving past experimentation to embrace it at scale, they are in danger of falling into the trap that they have fallen into time and time again. Unfortunately, we've seen this movie before, and we know how it ends: badly.
TCP (Transmission Control Protocol) is a common and reliable transmission protocol on the Internet. TCP was introduced in the 70s by Stanford University for US Defense to establish connectivity between distributed systems to maintain a backup of defense information. At the time, TCP was introduced to communicate amongst a selected set of devices for a smaller dataset over shorter distances. As the Internet evolved, however, the number of applications and users, and the types of data accessed and...