Welcome!

Microservices Expo Authors: Elizabeth White, Liz McMillan, PagerDuty Blog, Yeshim Deniz, JP Morgenthal

Related Topics: Microservices Expo

Microservices Expo: Article

ESB Myth Busters: 10 Enterprise Service Bus Myths Debunked

Clarity of Definition for a Growing Phenomenon

Myth #6: Portals can be connected to back-end systems by simply using a Web service call.
While Web service calls can theoretically be used to connect portals with back-end target systems, this approach won't scale past a few back-end systems. An ESB allows the portal server to have a single interface to the bus, with the bus providing the mediation between the diverse connectivity options, protocols, security, and data formats across all of the possible back-end systems that a portal server may call upon to fulfill its requests.

Using an ESB as the layer between the portal server and the various back-end applications that the portal server needs to interact with, ESB adopters are building for the future by providing a more scalable and flexible SOA that is capable of handling the ever-expanding scope of integration as the portal project becomes more successful and business requirements change.

Myth #7: ESBs will be obsolete once BPEL is widely available.
An ESB may support multiple ways of coordinating the interaction between event-driven service invocations using formal business process definitions. BPEL (Business Process Execution Language) is one way of doing it, and there are others as well. An ESB also has itinerary-based routing, which provides a message with a list of routing instructions. These routing instructions, which represent a business process definition, are carried with the message as it travels through the Bus across service invocations. The remote ESB service containers determine where to send the message next.

Itinerary-based routing significantly contributes to the highly distributed nature of the ESB, as there is no centralized rules engine to refer back to for each step in the process. A centralized rules engine for the routing of messages, such as those offered by the typical hub-and-spoke EAI broker approach, can be a bottleneck, and also a single point of failure. The use of message itineraries, messages and process definitions is self sufficient and can therefore allow different parts of the ESB to operate independently of one another.

Message itineraries are most useful for discreet process definitions that are stateless and usually contain a finite set of steps that don't take extended periods of time to complete. Gartner refers to this type of process definition as "microflows." Simple branching within itineraries may occur based on the use of content-based routing services.

When more sophisticated process definitions are required, a process orchestration engine may be layered onto the ESB as an additional service. The process orchestration may support stateful processes, which can span long durations of time. It may also support parallel execution paths, with branching, and merging of message flow execution paths based on join conditions or transition conditions being met. Such a process engine may support BPEL, or some other process definition grammar such as ebXML BPSS. Sophisticated process orchestration can be combined with stateless itinerary-based routing to create an SOA that solves complex integration problems.

Myth #8: The ESB technology category, like so many others, seems to have come out of nowhere and is now barreling its way up the hype curve and rapidly approaching the "trough of disillusionment."
The ESB concepts were created as a result of working with IT thought leaders across a variety of industries, including manufacturing, e-marketplaces, telco, financial services, and retail. The ESB as a concept was born out of a necessity, based on their desire for a new approach over distributed computing models and EAI technologies they had already been moderately successful with. These IT thought leaders all came with a common request: "We really like your distributed messaging infrastructure, and we would like to build upon it a standards-based event-driven SOA for integration. We would like it to include things like Web services, XML data transformation, content-based routing, and a service invocation model based on distributed process coordination." Because of this, the concepts presented in the ESB architecture are sound; they are grounded in reality. Also because of this, ESB technology has been adopted as it has been built. There are 100s of ESB deployments already in use today in areas such as supply chain and logistics automation, straight through processing in financial services, real-time provisioning in Telco, and remote storefront integration in retail.

Myth #9: ESBs are simply plumbing and do not provide sophisticated tooling, such as a graphical editor for designing business process flows.
There is a new breed of IDE, which Gartner Group refers to as an ISE (integrated services environment), that allows you to design, configure, test, and debug the integration services that you develop when building an SOA with an ESB. Using a graphical interface, an integration architect draws diagrams using UML notation to describe process definitions. You may also use the ISE to graphically create data transformations between different data formats, and create and debug XSLT style- sheets.

Myth #10: An ESB container can be implemented using an EJB container.
One of the key components of an ESB architecture is a highly distributed, lightweight service container. The service container allows the hosting of integration components as event-driven services, such as a content-based routing service that applies an XPath expression to an XML message to determine where to route it next. The service container can also host custom services or specialized adapters for hooking into packaged applications.

Unlike its distant cousins, the app server container and the integration broker, the ESB service container allows the selective deployment of integration services exactly when and where you need them, and nothing more. In contrast, you need to install an entire appserver stack everywhere that an individual piece of integration functionality is needed. This results in what is referred to as the "appservers everywhere" problem. There is an unnecessarily high cost in licensing, installation, and cost of ownership over time associated with this practice.

The mantra of the ESB is "configuration rather than coding." In an application server-centric approach to integration, you typically write code to describe the dependencies between services. The EJB model follows the client/server model of interaction, which usually results in tightly coupled interfaces between services in an SOA, which is built into code, and compiled into class files that need to be modified and redeployed every time a change needs to be made.

In an ESB, a service is configured with information regarding its input and output channels for sending and receiving message-based request/response patterns and one-way event notifications that are then coordinated by the surrounding invocation framework - not by the service itself.

An ESB service can be configured and deployed, by merely supplying it with the XSL stylesheets, XPath expression, scripts, and parameters which are read in from a configuration repository. Once deployed, the implementation is remarkably resilient to change.

Summary
To sum this up, make sure that your understanding of ESB contains these things:

  • An ESB provides the backbone for building an enterprise SOA-based integration environment.
  • The evolving WS-* specifications will help make ESBs even more interoperable than they are today. Adopting an ESB today will allow you to build for the future and be capable of adapting to the WS-* specifications as they become commercially viable.
  • ESB is not just an abstract pattern. It is a product category with a distinct definition and many vendor offerings.
  • ESBs and application servers are highly complementary.
  • ESBs can help portal server integration to back-end systems by providing the necessary diversity in connectivity and scalable infrastructure.
  • ESBs provide many choices for coordinating service interactions.
  • ESB technology is grounded in reality and is already being adopted by many industries.
  • ESBs can provide the higher-level visual tools for integrating services in an ISE environment.
  • ESBs provide a service container environment that is lightweight, configurable, and highly distributable.

More Stories By Dave Chappell

David Chappell is vice president and chief technologist for SOA at Oracle Corporation, and is driving the vision for Oracle’s SOA on App Grid initiative.

Comments (8) View Comments

Share your thoughts on this story.

Add your comment
You must be signed in to add a comment. Sign-in | Register

In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.


Most Recent Comments
Charlesy 07/03/13 07:28:00 AM EDT

An old post, but worth a small correction. Comparison with competitor products is always dangerous. You need to be very sure of your territory, and unfortunately, although David's description of BizTalk Server has some validity, it isn't 100% accurate. For example, the transformation services absolutely can be invoked separately to the rest of BTS via lightweight services, and load balanced across different boxes. More recent versions of BTS provide pre-built generic WCF and ASMX transformation web services as a courtesy to developers (reduces the need to build custom transformation services). You can use Windows Server AppFabric tools to create BizTalk maps in non-BizTalk projects - e.g., projects that define lightweight transformation services.

David doesn't clearly spell out what he means by 'cost...of the entire BizTalk Server'. If he means licencing cost, then he hits the mark, somewhat. BTS is certainly licensed in a fashion that encourages distribution and load balancing over a small farm of centralized servers, in contrast to the highly distributed approach advocated by Sonic. You can only invoke BTS transformation services on licensed boxes, so from that perspective, he is correct. However, invoking the transformation services does not require loading additional irrelevant BTS plumbing into memory. There is no heavy-weight performance cost imposed by code bloat, or anything like that!

BizTalk maps are emitted as code components containing an executable XSLT resource. You can distribute maps as freely as you wish and invoke the tranforms via code. Obviously, direct invocation in this case assumes the use of either .NET or Mono, although Java/.NET bridges could be used. If you use BizTalk Server's mapping tools to create maps, you may end up with a dependency on BizTalk-specific scripted components invoked in the XSLT, which ties you to licensed BizTalk Boxes. However, it is pretty easy to avoid this if you wish.

One thing Sonic has which BizTalk really does not is dynamic management of code deployment into the run-time environment. BizTalk Server has an built-in code repository, but this is simply a mechanism for managing and storing compiled artifacts and resources for the purpose of exporting installation packages. You still have to manually install those packages or deploy them via additional script. Frankly, though, this is rarely a significant drawback. The types of solution built using BizTalk Server tend to warrant close attention to managing dynamic deployment across a distributed environment using other frameworks and tools, and BizTalk Server plays well with the relevant frameworks. There is even a community-built deployment framework specifically designed for BizTalk Server.

yt67 03/03/05 07:09:34 AM EST

Myth-busting: always entertaining.

Jason 03/02/05 09:16:29 AM EST

A good read!

Javier Camara 02/10/05 04:19:02 AM EST

(This same feedback also posted to another WSJ article about ESBs)

I agree in that the ESB concept is over-hyped. For me, a SOA makes sense if it is viewed as a constellation of web services interacting among them. For this, something like a UDDI server is required for each service locating each other.

For me, all this (i.e. services + directory) is just enough if only synchronous communications are used. If asynchronous communications are needed, then you need also publish/subscribe and store-and-forward, i.e. roughly what a MOM does. You can call it an ESB if you want, although I think this concept in the market encompasses several roles:
1. Publish/subscribe to messages
2. Store-and-forward messages
3. Route messages
4. Transform messages

An interesting thing to note is to implement points 1. and 2. you do *not* need business logic, while to implement 3. and 4. you do.

As I said, I see roles 1 and 2 required in SOAs with asynchronous interactions.

Roles 3 and 4 are also needed in many cases, mainly for integrating disparate systems. However, my main point against an ESB is that, in order to perform these roles, you do *NOT* need of a new, special concept like the ESB. *Any* service in the constellation of services can perform both routing and transformation. It can range from being a single component like an ESB (which I think is a bad idea), or it can just be a set of services (e.g. a different service performing specific adaptation for a system being integrated).

For me, using a single ESB for 3. and 4. breaks the beauty of the SOA idea. You are supposed to made all your data and business logic of your organization available as services in order to be reused, and suddenly you put on top an ESB in which you put *more* business logic (routing and transformation). So my point is that this should be implemented just by means of regular services, and not by specific, central-piece new components called ESBs.

Now, if for implementing routing and transformation you want to use Tibco, WebSphere or whatever, fine - however, the logic created by these products should be at the same level as the other services in the SOA, and not above.

So I am not saying that orchestrating tools are not useful. They are. Only, they are not *imprescindible*; and at any rate they should be viewed just as more services in the SOA. However, this does not fit the marketing strategy of ESB vendors which show its ESB as an *enabler* of a SOA, instead of just one more *component* of it.

Dave Chappell 02/03/05 09:54:43 PM EST

We (Sonic Software) didn't re-lable our product to support the ESB wave, we actually invented the concept. We then worked with the analyst and journalist community to help create industry awareness of the new concepts that are introduced by ESB, which has resulted in a whole new product category.

I would agree with you that there is a great deal of hype right now due to lack of understanding of what ESB is, which is compounded by the number of traditional middleware and EAI vendors who have clamoured to get ESB in their marketing literature without having a full understanding of what it means to have an ESB. Your comment about middleware with new clothes is well taken. You might get that impression depending on where you learned about what an ESB is. That is exactly what I am trying to point out with myth #1 in this article.

A certain amount of hype is expected when a technology category begins to take hold and gain traction within serious IT projects. This can be disruptive to the industry as a whole. This is also the primary reason why I wrote the OReilly book on the subject of ESB--to act as the definitive guide to help educate and provide clarity on what makes up an ESB. Please don't shoot the concept of ESB down until you have had a chance to understand it.
Dave

Larry 02/03/05 04:15:19 AM EST

Not surprising that the representative of a company who over-hyped ESB in the first place, and relabeled their own product ESB to catch the service wave, should now try to claim that anyone who saw through the hype is guilty of spreading myths.
ESB is just the middleware emporor's new clothes.

@MicroservicesExpo Stories
After more than five years of DevOps, definitions are evolving, boundaries are expanding, ‘unicorns’ are no longer rare, enterprises are on board, and pundits are moving on. Can we now look at an evolution of DevOps? Should we? Is the foundation of DevOps ‘done’, or is there still too much left to do? What is mature, and what is still missing? What does the next 5 years of DevOps look like? In this Power Panel at DevOps Summit, moderated by DevOps Summit Conference Chair Andi Mann, panelists l...
Culture is the most important ingredient of DevOps. The challenge for most organizations is defining and communicating a vision of beneficial DevOps culture for their organizations, and then facilitating the changes needed to achieve that. Often this comes down to an ability to provide true leadership. As a CIO, are your direct reports IT managers or are they IT leaders? The hard truth is that many IT managers have risen through the ranks based on their technical skills, not their leadership abi...
The essence of cloud computing is that all consumable IT resources are delivered as services. In his session at 15th Cloud Expo, Yung Chou, Technology Evangelist at Microsoft, demonstrated the concepts and implementations of two important cloud computing deliveries: Infrastructure as a Service (IaaS) and Platform as a Service (PaaS). He discussed from business and technical viewpoints what exactly they are, why we care, how they are different and in what ways, and the strategies for IT to transi...
Thanks to Docker and the DevOps revolution, microservices have emerged as the new way to build and deploy applications — and there are plenty of great reasons to embrace the microservices trend. If you are going to adopt microservices, you also have to understand that microservice architectures have many moving parts. When it comes to incident management, this presents an important difference between microservices and monolithic architectures. More moving parts mean more complexity to monitor an...
All organizations that did not originate this moment have a pre-existing culture as well as legacy technology and processes that can be more or less amenable to DevOps implementation. That organizational culture is influenced by the personalities and management styles of Executive Management, the wider culture in which the organization is situated, and the personalities of key team members at all levels of the organization. This culture and entrenched interests usually throw a wrench in the work...
Microservices (μServices) are a fascinating evolution of the Distributed Object Computing (DOC) paradigm. Initial design of DOC attempted to solve the problem of simplifying developing complex distributed applications by applying object-oriented design principles to disparate components operating across networked infrastructure. In this model, DOC “hid” the complexity of making this work from the developer regardless of the deployment architecture through the use of complex frameworks, such as C...
DevOps is often described as a combination of technology and culture. Without both, DevOps isn't complete. However, applying the culture to outdated technology is a recipe for disaster; as response times grow and connections between teams are delayed by technology, the culture will die. A Nutanix Enterprise Cloud has many benefits that provide the needed base for a true DevOps paradigm.
TechTarget storage websites are the best online information resource for news, tips and expert advice for the storage, backup and disaster recovery markets. By creating abundant, high-quality editorial content across more than 140 highly targeted technology-specific websites, TechTarget attracts and nurtures communities of technology buyers researching their companies' information technology needs. By understanding these buyers' content consumption behaviors, TechTarget creates the purchase inte...
The IT industry is undergoing a significant evolution to keep up with cloud application demand. We see this happening as a mindset shift, from traditional IT teams to more well-rounded, cloud-focused job roles. The IT industry has become so cloud-minded that Gartner predicts that by 2020, this cloud shift will impact more than $1 trillion of global IT spending. This shift, however, has left some IT professionals feeling a little anxious about what lies ahead. The good news is that cloud computin...
DevOps is often described as a combination of technology and culture. Without both, DevOps isn't complete. However, applying the culture to outdated technology is a recipe for disaster; as response times grow and connections between teams are delayed by technology, the culture will die. A Nutanix Enterprise Cloud has many benefits that provide the needed base for a true DevOps paradigm. In his Day 3 Keynote at 20th Cloud Expo, Chris Brown, a Solutions Marketing Manager at Nutanix, will explore t...
We've all had that feeling before: The feeling that you're missing something that everyone else is in on. For today's IT leaders, that feeling might come up when you hear talk about cloud brokers. Meanwhile, you head back into your office and deal with your ever-growing shadow IT problem. But the cloud-broker whispers and your shadow IT issues are linked. If you're wondering "what the heck is a cloud broker?" we've got you covered.
What if you could build a web application that could support true web-scale traffic without having to ever provision or manage a single server? Sounds magical, and it is! In his session at 20th Cloud Expo, Chris Munns, Senior Developer Advocate for Serverless Applications at Amazon Web Services, will show how to build a serverless website that scales automatically using services like AWS Lambda, Amazon API Gateway, and Amazon S3. We will review several frameworks that can help you build serverle...
Everyone wants to use containers, but monitoring containers is hard. New ephemeral architecture introduces new challenges in how monitoring tools need to monitor and visualize containers, so your team can make sense of everything. In his session at @DevOpsSummit, David Gildeh, co-founder and CEO of Outlyer, will go through the challenges and show there is light at the end of the tunnel if you use the right tools and understand what you need to be monitoring to successfully use containers in your...
In today's enterprise, digital transformation represents organizational change even more so than technology change, as customer preferences and behavior drive end-to-end transformation across lines of business as well as IT. To capitalize on the ubiquitous disruption driving this transformation, companies must be able to innovate at an increasingly rapid pace. Traditional approaches for driving innovation are now woefully inadequate for keeping up with the breadth of disruption and change facing...
In his General Session at 16th Cloud Expo, David Shacochis, host of The Hybrid IT Files podcast and Vice President at CenturyLink, investigated three key trends of the “gigabit economy" though the story of a Fortune 500 communications company in transformation. Narrating how multi-modal hybrid IT, service automation, and agile delivery all intersect, he will cover the role of storytelling and empathy in achieving strategic alignment between the enterprise and its information technology.
Microservices are a very exciting architectural approach that many organizations are looking to as a way to accelerate innovation. Microservices promise to allow teams to move away from monolithic "ball of mud" systems, but the reality is that, in the vast majority of organizations, different projects and technologies will continue to be developed at different speeds. How to handle the dependencies between these disparate systems with different iteration cycles? Consider the "canoncial problem" ...
SYS-CON Events announced today that HTBase will exhibit at SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. HTBase (Gartner 2016 Cool Vendor) delivers a Composable IT infrastructure solution architected for agility and increased efficiency. It turns compute, storage, and fabric into fluid pools of resources that are easily composed and re-composed to meet each application’s needs. With HTBase, companies can quickly prov...
The rise of containers and microservices has skyrocketed the rate at which new applications are moved into production environments today. While developers have been deploying containers to speed up the development processes for some time, there still remain challenges with running microservices efficiently. Most existing IT monitoring tools don’t actually maintain visibility into the containers that make up microservices. As those container applications move into production, some IT operations t...
For organizations that have amassed large sums of software complexity, taking a microservices approach is the first step toward DevOps and continuous improvement / development. Integrating system-level analysis with microservices makes it easier to change and add functionality to applications at any time without the increase of risk. Before you start big transformation projects or a cloud migration, make sure these changes won’t take down your entire organization.
In recent years, containers have taken the world by storm. Companies of all sizes and industries have realized the massive benefits of containers, such as unprecedented mobility, higher hardware utilization, and increased flexibility and agility; however, many containers today are non-persistent. Containers without persistence miss out on many benefits, and in many cases simply pass the responsibility of persistence onto other infrastructure, adding additional complexity.