Welcome!

Microservices Expo Authors: Elizabeth White, Pat Romanski, Jason Bloomberg, Kong Yang, Mark Leake

Related Topics: Microservices Expo

Microservices Expo: Article

ESB Myth Busters: 10 Enterprise Service Bus Myths Debunked

Clarity of Definition for a Growing Phenomenon

Myth #6: Portals can be connected to back-end systems by simply using a Web service call.
While Web service calls can theoretically be used to connect portals with back-end target systems, this approach won't scale past a few back-end systems. An ESB allows the portal server to have a single interface to the bus, with the bus providing the mediation between the diverse connectivity options, protocols, security, and data formats across all of the possible back-end systems that a portal server may call upon to fulfill its requests.

Using an ESB as the layer between the portal server and the various back-end applications that the portal server needs to interact with, ESB adopters are building for the future by providing a more scalable and flexible SOA that is capable of handling the ever-expanding scope of integration as the portal project becomes more successful and business requirements change.

Myth #7: ESBs will be obsolete once BPEL is widely available.
An ESB may support multiple ways of coordinating the interaction between event-driven service invocations using formal business process definitions. BPEL (Business Process Execution Language) is one way of doing it, and there are others as well. An ESB also has itinerary-based routing, which provides a message with a list of routing instructions. These routing instructions, which represent a business process definition, are carried with the message as it travels through the Bus across service invocations. The remote ESB service containers determine where to send the message next.

Itinerary-based routing significantly contributes to the highly distributed nature of the ESB, as there is no centralized rules engine to refer back to for each step in the process. A centralized rules engine for the routing of messages, such as those offered by the typical hub-and-spoke EAI broker approach, can be a bottleneck, and also a single point of failure. The use of message itineraries, messages and process definitions is self sufficient and can therefore allow different parts of the ESB to operate independently of one another.

Message itineraries are most useful for discreet process definitions that are stateless and usually contain a finite set of steps that don't take extended periods of time to complete. Gartner refers to this type of process definition as "microflows." Simple branching within itineraries may occur based on the use of content-based routing services.

When more sophisticated process definitions are required, a process orchestration engine may be layered onto the ESB as an additional service. The process orchestration may support stateful processes, which can span long durations of time. It may also support parallel execution paths, with branching, and merging of message flow execution paths based on join conditions or transition conditions being met. Such a process engine may support BPEL, or some other process definition grammar such as ebXML BPSS. Sophisticated process orchestration can be combined with stateless itinerary-based routing to create an SOA that solves complex integration problems.

Myth #8: The ESB technology category, like so many others, seems to have come out of nowhere and is now barreling its way up the hype curve and rapidly approaching the "trough of disillusionment."
The ESB concepts were created as a result of working with IT thought leaders across a variety of industries, including manufacturing, e-marketplaces, telco, financial services, and retail. The ESB as a concept was born out of a necessity, based on their desire for a new approach over distributed computing models and EAI technologies they had already been moderately successful with. These IT thought leaders all came with a common request: "We really like your distributed messaging infrastructure, and we would like to build upon it a standards-based event-driven SOA for integration. We would like it to include things like Web services, XML data transformation, content-based routing, and a service invocation model based on distributed process coordination." Because of this, the concepts presented in the ESB architecture are sound; they are grounded in reality. Also because of this, ESB technology has been adopted as it has been built. There are 100s of ESB deployments already in use today in areas such as supply chain and logistics automation, straight through processing in financial services, real-time provisioning in Telco, and remote storefront integration in retail.

Myth #9: ESBs are simply plumbing and do not provide sophisticated tooling, such as a graphical editor for designing business process flows.
There is a new breed of IDE, which Gartner Group refers to as an ISE (integrated services environment), that allows you to design, configure, test, and debug the integration services that you develop when building an SOA with an ESB. Using a graphical interface, an integration architect draws diagrams using UML notation to describe process definitions. You may also use the ISE to graphically create data transformations between different data formats, and create and debug XSLT style- sheets.

Myth #10: An ESB container can be implemented using an EJB container.
One of the key components of an ESB architecture is a highly distributed, lightweight service container. The service container allows the hosting of integration components as event-driven services, such as a content-based routing service that applies an XPath expression to an XML message to determine where to route it next. The service container can also host custom services or specialized adapters for hooking into packaged applications.

Unlike its distant cousins, the app server container and the integration broker, the ESB service container allows the selective deployment of integration services exactly when and where you need them, and nothing more. In contrast, you need to install an entire appserver stack everywhere that an individual piece of integration functionality is needed. This results in what is referred to as the "appservers everywhere" problem. There is an unnecessarily high cost in licensing, installation, and cost of ownership over time associated with this practice.

The mantra of the ESB is "configuration rather than coding." In an application server-centric approach to integration, you typically write code to describe the dependencies between services. The EJB model follows the client/server model of interaction, which usually results in tightly coupled interfaces between services in an SOA, which is built into code, and compiled into class files that need to be modified and redeployed every time a change needs to be made.

In an ESB, a service is configured with information regarding its input and output channels for sending and receiving message-based request/response patterns and one-way event notifications that are then coordinated by the surrounding invocation framework - not by the service itself.

An ESB service can be configured and deployed, by merely supplying it with the XSL stylesheets, XPath expression, scripts, and parameters which are read in from a configuration repository. Once deployed, the implementation is remarkably resilient to change.

Summary
To sum this up, make sure that your understanding of ESB contains these things:

  • An ESB provides the backbone for building an enterprise SOA-based integration environment.
  • The evolving WS-* specifications will help make ESBs even more interoperable than they are today. Adopting an ESB today will allow you to build for the future and be capable of adapting to the WS-* specifications as they become commercially viable.
  • ESB is not just an abstract pattern. It is a product category with a distinct definition and many vendor offerings.
  • ESBs and application servers are highly complementary.
  • ESBs can help portal server integration to back-end systems by providing the necessary diversity in connectivity and scalable infrastructure.
  • ESBs provide many choices for coordinating service interactions.
  • ESB technology is grounded in reality and is already being adopted by many industries.
  • ESBs can provide the higher-level visual tools for integrating services in an ISE environment.
  • ESBs provide a service container environment that is lightweight, configurable, and highly distributable.

More Stories By Dave Chappell

David Chappell is vice president and chief technologist for SOA at Oracle Corporation, and is driving the vision for Oracle’s SOA on App Grid initiative.

Comments (8) View Comments

Share your thoughts on this story.

Add your comment
You must be signed in to add a comment. Sign-in | Register

In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.


Most Recent Comments
Charlesy 07/03/13 07:28:00 AM EDT

An old post, but worth a small correction. Comparison with competitor products is always dangerous. You need to be very sure of your territory, and unfortunately, although David's description of BizTalk Server has some validity, it isn't 100% accurate. For example, the transformation services absolutely can be invoked separately to the rest of BTS via lightweight services, and load balanced across different boxes. More recent versions of BTS provide pre-built generic WCF and ASMX transformation web services as a courtesy to developers (reduces the need to build custom transformation services). You can use Windows Server AppFabric tools to create BizTalk maps in non-BizTalk projects - e.g., projects that define lightweight transformation services.

David doesn't clearly spell out what he means by 'cost...of the entire BizTalk Server'. If he means licencing cost, then he hits the mark, somewhat. BTS is certainly licensed in a fashion that encourages distribution and load balancing over a small farm of centralized servers, in contrast to the highly distributed approach advocated by Sonic. You can only invoke BTS transformation services on licensed boxes, so from that perspective, he is correct. However, invoking the transformation services does not require loading additional irrelevant BTS plumbing into memory. There is no heavy-weight performance cost imposed by code bloat, or anything like that!

BizTalk maps are emitted as code components containing an executable XSLT resource. You can distribute maps as freely as you wish and invoke the tranforms via code. Obviously, direct invocation in this case assumes the use of either .NET or Mono, although Java/.NET bridges could be used. If you use BizTalk Server's mapping tools to create maps, you may end up with a dependency on BizTalk-specific scripted components invoked in the XSLT, which ties you to licensed BizTalk Boxes. However, it is pretty easy to avoid this if you wish.

One thing Sonic has which BizTalk really does not is dynamic management of code deployment into the run-time environment. BizTalk Server has an built-in code repository, but this is simply a mechanism for managing and storing compiled artifacts and resources for the purpose of exporting installation packages. You still have to manually install those packages or deploy them via additional script. Frankly, though, this is rarely a significant drawback. The types of solution built using BizTalk Server tend to warrant close attention to managing dynamic deployment across a distributed environment using other frameworks and tools, and BizTalk Server plays well with the relevant frameworks. There is even a community-built deployment framework specifically designed for BizTalk Server.

yt67 03/03/05 07:09:34 AM EST

Myth-busting: always entertaining.

Jason 03/02/05 09:16:29 AM EST

A good read!

Javier Camara 02/10/05 04:19:02 AM EST

(This same feedback also posted to another WSJ article about ESBs)

I agree in that the ESB concept is over-hyped. For me, a SOA makes sense if it is viewed as a constellation of web services interacting among them. For this, something like a UDDI server is required for each service locating each other.

For me, all this (i.e. services + directory) is just enough if only synchronous communications are used. If asynchronous communications are needed, then you need also publish/subscribe and store-and-forward, i.e. roughly what a MOM does. You can call it an ESB if you want, although I think this concept in the market encompasses several roles:
1. Publish/subscribe to messages
2. Store-and-forward messages
3. Route messages
4. Transform messages

An interesting thing to note is to implement points 1. and 2. you do *not* need business logic, while to implement 3. and 4. you do.

As I said, I see roles 1 and 2 required in SOAs with asynchronous interactions.

Roles 3 and 4 are also needed in many cases, mainly for integrating disparate systems. However, my main point against an ESB is that, in order to perform these roles, you do *NOT* need of a new, special concept like the ESB. *Any* service in the constellation of services can perform both routing and transformation. It can range from being a single component like an ESB (which I think is a bad idea), or it can just be a set of services (e.g. a different service performing specific adaptation for a system being integrated).

For me, using a single ESB for 3. and 4. breaks the beauty of the SOA idea. You are supposed to made all your data and business logic of your organization available as services in order to be reused, and suddenly you put on top an ESB in which you put *more* business logic (routing and transformation). So my point is that this should be implemented just by means of regular services, and not by specific, central-piece new components called ESBs.

Now, if for implementing routing and transformation you want to use Tibco, WebSphere or whatever, fine - however, the logic created by these products should be at the same level as the other services in the SOA, and not above.

So I am not saying that orchestrating tools are not useful. They are. Only, they are not *imprescindible*; and at any rate they should be viewed just as more services in the SOA. However, this does not fit the marketing strategy of ESB vendors which show its ESB as an *enabler* of a SOA, instead of just one more *component* of it.

Dave Chappell 02/03/05 09:54:43 PM EST

We (Sonic Software) didn't re-lable our product to support the ESB wave, we actually invented the concept. We then worked with the analyst and journalist community to help create industry awareness of the new concepts that are introduced by ESB, which has resulted in a whole new product category.

I would agree with you that there is a great deal of hype right now due to lack of understanding of what ESB is, which is compounded by the number of traditional middleware and EAI vendors who have clamoured to get ESB in their marketing literature without having a full understanding of what it means to have an ESB. Your comment about middleware with new clothes is well taken. You might get that impression depending on where you learned about what an ESB is. That is exactly what I am trying to point out with myth #1 in this article.

A certain amount of hype is expected when a technology category begins to take hold and gain traction within serious IT projects. This can be disruptive to the industry as a whole. This is also the primary reason why I wrote the OReilly book on the subject of ESB--to act as the definitive guide to help educate and provide clarity on what makes up an ESB. Please don't shoot the concept of ESB down until you have had a chance to understand it.
Dave

Larry 02/03/05 04:15:19 AM EST

Not surprising that the representative of a company who over-hyped ESB in the first place, and relabeled their own product ESB to catch the service wave, should now try to claim that anyone who saw through the hype is guilty of spreading myths.
ESB is just the middleware emporor's new clothes.

@MicroservicesExpo Stories
Managing mission-critical SAP systems and landscapes has never been easy. Add public cloud with its myriad of powerful cloud native services and this may not change any time soon. Public cloud offers exciting new possibilities for enterprise workloads. But to make use of these possibilities and capabilities, IT teams need to re-think everything they have done before. Otherwise, they will just end up using public cloud as a hosting platform for their workloads, aka known as “lift and shift.”
You know you need the cloud, but you’re hesitant to simply dump everything at Amazon since you know that not all workloads are suitable for cloud. You know that you want the kind of ease of use and scalability that you get with public cloud, but your applications are architected in a way that makes the public cloud a non-starter. You’re looking at private cloud solutions based on hyperconverged infrastructure, but you’re concerned with the limits inherent in those technologies.
"Tintri focuses on the Ops side of the DevOps, which basically is pushing more and more of the accessibility of the infrastructure to the developers and trying to get behind the scenes," explained Dhiraj Sehgal of Tintri in this SYS-CON.tv interview at 20th Cloud Expo, held June 6-8, 2017, at the Javits Center in New York City, NY.
Both SaaS vendors and SaaS buyers are going “all-in” to hyperscale IaaS platforms such as AWS, which is disrupting the SaaS value proposition. Why should the enterprise SaaS consumer pay for the SaaS service if their data is resident in adjacent AWS S3 buckets? If both SaaS sellers and buyers are using the same cloud tools, automation and pay-per-transaction model offered by IaaS platforms, then why not host the “shrink-wrapped” software in the customers’ cloud? Further, serverless computing, cl...
In the decade following his article, cloud computing further cemented Carr’s perspective. Compute, storage, and network resources have become simple utilities, available at the proverbial turn of the faucet. The value they provide is immense, but the cloud playing field is amazingly level. Carr’s quote above presaged the cloud to a T. Today, however, we’re in the digital era. Mark Andreesen’s ‘software is eating the world’ prognostication is coming to pass, as enterprises realize they must be...
Hybrid IT is today’s reality, and while its implementation may seem daunting at times, more and more organizations are migrating to the cloud. In fact, according to SolarWinds 2017 IT Trends Index: Portrait of a Hybrid IT Organization 95 percent of organizations have migrated crucial applications to the cloud in the past year. As such, it’s in every IT professional’s best interest to know what to expect.
A common misconception about the cloud is that one size fits all. Companies expecting to run all of their operations using one cloud solution or service must realize that doing so is akin to forcing the totality of their business functionality into a straightjacket. Unlocking the full potential of the cloud means embracing the multi-cloud future where businesses use their own cloud, and/or clouds from different vendors, to support separate functions or product groups. There is no single cloud so...
In 2014, Amazon announced a new form of compute called Lambda. We didn't know it at the time, but this represented a fundamental shift in what we expect from cloud computing. Now, all of the major cloud computing vendors want to take part in this disruptive technology. In his session at 20th Cloud Expo, Doug Vanderweide, an instructor at Linux Academy, discussed why major players like AWS, Microsoft Azure, IBM Bluemix, and Google Cloud Platform are all trying to sidestep VMs and containers wit...
Companies have always been concerned that traditional enterprise software is slow and complex to install, often disrupting critical and time-sensitive operations during roll-out. With the growing need to integrate new digital technologies into the enterprise to transform business processes, this concern has become even more pressing. A 2016 Panorama Consulting Solutions study revealed that enterprise resource planning (ERP) projects took an average of 21 months to install, with 57 percent of th...
The taxi industry never saw Uber coming. Startups are a threat to incumbents like never before, and a major enabler for startups is that they are instantly “cloud ready.” If innovation moves at the pace of IT, then your company is in trouble. Why? Because your data center will not keep up with frenetic pace AWS, Microsoft and Google are rolling out new capabilities. In his session at 20th Cloud Expo, Don Browning, VP of Cloud Architecture at Turner, posited that disruption is inevitable for comp...
New competitors, disruptive technologies, and growing expectations are pushing every business to both adopt and deliver new digital services. This ‘Digital Transformation’ demands rapid delivery and continuous iteration of new competitive services via multiple channels, which in turn demands new service delivery techniques – including DevOps. In this power panel at @DevOpsSummit 20th Cloud Expo, moderated by DevOps Conference Co-Chair Andi Mann, panelists examined how DevOps helps to meet the de...
"When we talk about cloud without compromise what we're talking about is that when people think about 'I need the flexibility of the cloud' - it's the ability to create applications and run them in a cloud environment that's far more flexible,” explained Matthew Finnie, CTO of Interoute, in this SYS-CON.tv interview at 20th Cloud Expo, held June 6-8, 2017, at the Javits Center in New York City, NY.
"We are a monitoring company. We work with Salesforce, BBC, and quite a few other big logos. We basically provide monitoring for them, structure for their cloud services and we fit into the DevOps world" explained David Gildeh, Co-founder and CEO of Outlyer, in this SYS-CON.tv interview at DevOps Summit at 20th Cloud Expo, held June 6-8, 2017, at the Javits Center in New York City, NY.
Colocation is a central pillar of modern enterprise infrastructure planning because it provides greater control, insight, and performance than managed platforms. In spite of the inexorable rise of the cloud, most businesses with extensive IT hardware requirements choose to host their infrastructure in colocation data centers. According to a recent IDC survey, more than half of the businesses questioned use colocation services, and the number is even higher among established businesses and busine...
For most organizations, the move to hybrid cloud is now a question of when, not if. Fully 82% of enterprises plan to have a hybrid cloud strategy this year, according to Infoholic Research. The worldwide hybrid cloud computing market is expected to grow about 34% annually over the next five years, reaching $241.13 billion by 2022. Companies are embracing hybrid cloud because of the many advantages it offers compared to relying on a single provider for all of their cloud needs. Hybrid offers bala...
@DevOpsSummit at Cloud Expo taking place Oct 31 - Nov 2, 2017, at the Santa Clara Convention Center, Santa Clara, CA, is co-located with the 21st International Cloud Expo and will feature technical sessions from a rock star conference faculty and the leading industry players in the world. The widespread success of cloud computing is driving the DevOps revolution in enterprise IT. Now as never before, development teams must communicate and collaborate in a dynamic, 24/7/365 environment. There is ...
The reality of data ubiquity is here—data is buried in operational statistics, machine logs, stacks of overflowing tickets and customer details, among other things. How can any user get valuable information amid this rapid influx of data? Imagine a situation where your firm’s revenue takes a hit owing to an unexpected failure in some business process. It would be a nightmare for IT admins to sift through the interminable piles of data to deduce exactly why and where the problem occurred. To sav...
For organizations that have amassed large sums of software complexity, taking a microservices approach is the first step toward DevOps and continuous improvement / development. Integrating system-level analysis with microservices makes it easier to change and add functionality to applications at any time without the increase of risk. Before you start big transformation projects or a cloud migration, make sure these changes won’t take down your entire organization.
What's the role of an IT self-service portal when you get to continuous delivery and Infrastructure as Code? This general session showed how to create the continuous delivery culture and eight accelerators for leading the change. Don Demcsak is a DevOps and Cloud Native Modernization Principal for Dell EMC based out of New Jersey. He is a former, long time, Microsoft Most Valuable Professional, specializing in building and architecting Application Delivery Pipelines for hybrid legacy, and cloud ...
21st International Cloud Expo, taking place October 31 - November 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA, will feature technical sessions from a rock star conference faculty and the leading industry players in the world. Cloud computing is now being embraced by a majority of enterprises of all sizes. Yesterday's debate about public vs. private has transformed into the reality of hybrid cloud: a recent survey shows that 74% of enterprises have a hybrid cloud strategy. Me...