Welcome!

Microservices Expo Authors: Elizabeth White, Pat Romanski, Jason Bloomberg, Mehdi Daoudi, Liz McMillan

Related Topics: Microservices Expo

Microservices Expo: Article

ESB Myth Busters: 10 Enterprise Service Bus Myths Debunked

Clarity of Definition for a Growing Phenomenon

Since releasing my latest book, Enterprise Service Bus (O'Reilly Media, 2004), I have been doing a fair amount of visiting corporations, conducting seminars, and generally discussing with enterprise architects the subject of enterprise service-oriented architecture (SOA) and how an enterprise service bus (ESB) backbone can be leveraged to provide a framework for an enterprise SOA. Along the way, I have been asked many questions about the nature of an ESB. I have also fended off some misconceptions that have been growing in the general IT population regarding what an ESB is and when, where, and how it can be used. I have gathered together the most popular questions and misconceptions, and offer some clarity in the form of a "top ten" list.

Myth #1. ESB is just a new name for EAI.
While many IT architecture groups are focusing on building SOAs, they still inevitably beg the question of "how is ESB different from EAI?" An ESB is an infrastructure for building an enterprise SOA, and is capable of being used in a more general way than a conventional EAI broker. According to Forrester Research, an ESB helps enterprises obtain the value of SOA by increasing connectivity, adding flexibility that speeds change, and providing greater control over use of the important resources that it binds.

An ESB can be used to handle integration projects that have traditionally been relegated to EAI tools. However, an ESB can also be used for establishing B2B relationships across companies.

An ESB provides EAI capabilities, but is based on a fundamentally different architecture that is providing the basis of an industry transition from traditional integration to coordinated service interaction. EAI brokers are historically implemented as a monolithic stack, using centralized hub-and-spoke architecture.

An ESB provides the same base functionality as an EAI broker - connectivity, application adapters, routing of messages based on rules, and data transformation engine - yet, in an ESB, these capabilities are themselves SOA based in that they are spread out across the bus in a highly distributed fashion and hosted in separately deployable service containers. This allows the selective deployment of integration broker functionality exactly where you need it, with no additional over-bloating where it's not needed. The distributed nature of the ESB container model allows the independent scalability of integration components, which are plugged into your SOA as event-driven services on an as needed basis.

In order for an integration broker to be truly capable of supporting an SOA, and to be considered a true ESB, it would need to have its base functions broken up into their constituent parts, which would then be capable of being separately deployed across the bus while working together in harmony as necessary.

Let's use an example of an XSLT-based transformation engine that accepts an incoming XML document and applies an XSLT style sheet to it in order to produce an outgoing document in another XML format. I can tell you that there is nothing that can chew up computing resources more than the parsing and manipulation of XML. If this particular XSLT transformation sits between two popular applications that communicate regularly with each other, then that individual transformation can become a performance and scalability bottleneck. If you are using a monolithic hub-and-spoke integration broker approach, in order to remove the bottleneck and scale up the deployment you would need to either install that integration broker on one big powerful machine, or install the integration broker across multiple machines - just to support that one transform scenario! All the while, the other integration broker capabilities, such as the execution of routing rules, are competing for the same computing resources as the transformation operation.

In contrast to the monolithic hub-and-spoke architecture of an integration broker, the foundational core of an ESB provides a distributed services architecture. This architecture is built for integration and has the ability for integration broker functionality, such as message routing, data transformation, and application adapters to be selectively deployed on an as-needed basis.These are separate integration services that are a natural part of an SOA processing pipeline across the bus.

An individual XSLT transformation can be deployed as a service in its own ESB service container, and multiple instances of that container can be load-balanced across many machines. If the ESB container implementation is cross-platform, then you can be flexible as to what kinds of machines you spread the transform service across - Linux boxes, Solaris boxes, Windows boxes, and so on. And for those of you who don't find solace in the architectural purity of this discussion, consider this: the ESB vendors who are leading the charge in defining and delivering ESB products are also putting forth a license model where there is no additional cost for deploying as many of these lightweight ESB service containers as necessary to get the job done.

The integration services provided by the ESB can be combined with other services into SOA-based processing pipelines that can span business boundaries. The distributed services in an ESB can be combined with itinerary-based routing (see Myth #7) to allow self-directed, message-oriented service interactions, which allow different parts of the ESB to operate independently of one another, without relying on a centralized routing engine.

More Stories By Dave Chappell

David Chappell is vice president and chief technologist for SOA at Oracle Corporation, and is driving the vision for Oracle’s SOA on App Grid initiative.

Comments (8) View Comments

Share your thoughts on this story.

Add your comment
You must be signed in to add a comment. Sign-in | Register

In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.


Most Recent Comments
Charlesy 07/03/13 07:28:00 AM EDT

An old post, but worth a small correction. Comparison with competitor products is always dangerous. You need to be very sure of your territory, and unfortunately, although David's description of BizTalk Server has some validity, it isn't 100% accurate. For example, the transformation services absolutely can be invoked separately to the rest of BTS via lightweight services, and load balanced across different boxes. More recent versions of BTS provide pre-built generic WCF and ASMX transformation web services as a courtesy to developers (reduces the need to build custom transformation services). You can use Windows Server AppFabric tools to create BizTalk maps in non-BizTalk projects - e.g., projects that define lightweight transformation services.

David doesn't clearly spell out what he means by 'cost...of the entire BizTalk Server'. If he means licencing cost, then he hits the mark, somewhat. BTS is certainly licensed in a fashion that encourages distribution and load balancing over a small farm of centralized servers, in contrast to the highly distributed approach advocated by Sonic. You can only invoke BTS transformation services on licensed boxes, so from that perspective, he is correct. However, invoking the transformation services does not require loading additional irrelevant BTS plumbing into memory. There is no heavy-weight performance cost imposed by code bloat, or anything like that!

BizTalk maps are emitted as code components containing an executable XSLT resource. You can distribute maps as freely as you wish and invoke the tranforms via code. Obviously, direct invocation in this case assumes the use of either .NET or Mono, although Java/.NET bridges could be used. If you use BizTalk Server's mapping tools to create maps, you may end up with a dependency on BizTalk-specific scripted components invoked in the XSLT, which ties you to licensed BizTalk Boxes. However, it is pretty easy to avoid this if you wish.

One thing Sonic has which BizTalk really does not is dynamic management of code deployment into the run-time environment. BizTalk Server has an built-in code repository, but this is simply a mechanism for managing and storing compiled artifacts and resources for the purpose of exporting installation packages. You still have to manually install those packages or deploy them via additional script. Frankly, though, this is rarely a significant drawback. The types of solution built using BizTalk Server tend to warrant close attention to managing dynamic deployment across a distributed environment using other frameworks and tools, and BizTalk Server plays well with the relevant frameworks. There is even a community-built deployment framework specifically designed for BizTalk Server.

yt67 03/03/05 07:09:34 AM EST

Myth-busting: always entertaining.

Jason 03/02/05 09:16:29 AM EST

A good read!

Javier Camara 02/10/05 04:19:02 AM EST

(This same feedback also posted to another WSJ article about ESBs)

I agree in that the ESB concept is over-hyped. For me, a SOA makes sense if it is viewed as a constellation of web services interacting among them. For this, something like a UDDI server is required for each service locating each other.

For me, all this (i.e. services + directory) is just enough if only synchronous communications are used. If asynchronous communications are needed, then you need also publish/subscribe and store-and-forward, i.e. roughly what a MOM does. You can call it an ESB if you want, although I think this concept in the market encompasses several roles:
1. Publish/subscribe to messages
2. Store-and-forward messages
3. Route messages
4. Transform messages

An interesting thing to note is to implement points 1. and 2. you do *not* need business logic, while to implement 3. and 4. you do.

As I said, I see roles 1 and 2 required in SOAs with asynchronous interactions.

Roles 3 and 4 are also needed in many cases, mainly for integrating disparate systems. However, my main point against an ESB is that, in order to perform these roles, you do *NOT* need of a new, special concept like the ESB. *Any* service in the constellation of services can perform both routing and transformation. It can range from being a single component like an ESB (which I think is a bad idea), or it can just be a set of services (e.g. a different service performing specific adaptation for a system being integrated).

For me, using a single ESB for 3. and 4. breaks the beauty of the SOA idea. You are supposed to made all your data and business logic of your organization available as services in order to be reused, and suddenly you put on top an ESB in which you put *more* business logic (routing and transformation). So my point is that this should be implemented just by means of regular services, and not by specific, central-piece new components called ESBs.

Now, if for implementing routing and transformation you want to use Tibco, WebSphere or whatever, fine - however, the logic created by these products should be at the same level as the other services in the SOA, and not above.

So I am not saying that orchestrating tools are not useful. They are. Only, they are not *imprescindible*; and at any rate they should be viewed just as more services in the SOA. However, this does not fit the marketing strategy of ESB vendors which show its ESB as an *enabler* of a SOA, instead of just one more *component* of it.

Dave Chappell 02/03/05 09:54:43 PM EST

We (Sonic Software) didn't re-lable our product to support the ESB wave, we actually invented the concept. We then worked with the analyst and journalist community to help create industry awareness of the new concepts that are introduced by ESB, which has resulted in a whole new product category.

I would agree with you that there is a great deal of hype right now due to lack of understanding of what ESB is, which is compounded by the number of traditional middleware and EAI vendors who have clamoured to get ESB in their marketing literature without having a full understanding of what it means to have an ESB. Your comment about middleware with new clothes is well taken. You might get that impression depending on where you learned about what an ESB is. That is exactly what I am trying to point out with myth #1 in this article.

A certain amount of hype is expected when a technology category begins to take hold and gain traction within serious IT projects. This can be disruptive to the industry as a whole. This is also the primary reason why I wrote the OReilly book on the subject of ESB--to act as the definitive guide to help educate and provide clarity on what makes up an ESB. Please don't shoot the concept of ESB down until you have had a chance to understand it.
Dave

Larry 02/03/05 04:15:19 AM EST

Not surprising that the representative of a company who over-hyped ESB in the first place, and relabeled their own product ESB to catch the service wave, should now try to claim that anyone who saw through the hype is guilty of spreading myths.
ESB is just the middleware emporor's new clothes.

@MicroservicesExpo Stories
Many organizations adopt DevOps to reduce cycle times and deliver software faster; some take on DevOps to drive higher quality and better end-user experience; others look to DevOps for a clearer line-of-sight to customers to drive better business impacts. In truth, these three foundations go together. In this power panel at @DevOpsSummit 21st Cloud Expo, moderated by DevOps Conference Co-Chair Andi Mann, industry experts will discuss how leading organizations build application success from all...
The last two years has seen discussions about cloud computing evolve from the public / private / hybrid split to the reality that most enterprises will be creating a complex, multi-cloud strategy. Companies are wary of committing all of their resources to a single cloud, and instead are choosing to spread the risk – and the benefits – of cloud computing across multiple providers and internal infrastructures, as they follow their business needs. Will this approach be successful? How large is the ...
Many organizations are now looking to DevOps maturity models to gauge their DevOps adoption and compare their maturity to their peers. However, as enterprise organizations rush to adopt DevOps, moving past experimentation to embrace it at scale, they are in danger of falling into the trap that they have fallen into time and time again. Unfortunately, we've seen this movie before, and we know how it ends: badly.
‘Trend’ is a pretty common business term, but its definition tends to vary by industry. In performance monitoring, trend, or trend shift, is a key metric that is used to indicate change. Change is inevitable. Today’s websites must frequently update and change to keep up with competition and attract new users, but such changes can have a negative impact on the user experience if not managed properly. The dynamic nature of the Internet makes it necessary to constantly monitor different metrics. O...
Enterprises are moving to the cloud faster than most of us in security expected. CIOs are going from 0 to 100 in cloud adoption and leaving security teams in the dust. Once cloud is part of an enterprise stack, it’s unclear who has responsibility for the protection of applications, services, and data. When cloud breaches occur, whether active compromise or a publicly accessible database, the blame must fall on both service providers and users. In his session at 21st Cloud Expo, Ben Johnson, C...
21st International Cloud Expo, taking place October 31 - November 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA, will feature technical sessions from a rock star conference faculty and the leading industry players in the world. Cloud computing is now being embraced by a majority of enterprises of all sizes. Yesterday's debate about public vs. private has transformed into the reality of hybrid cloud: a recent survey shows that 74% of enterprises have a hybrid cloud strategy. Me...
Today companies are looking to achieve cloud-first digital agility to reduce time-to-market, optimize utilization of resources, and rapidly deliver disruptive business solutions. However, leveraging the benefits of cloud deployments can be complicated for companies with extensive legacy computing environments. In his session at 21st Cloud Expo, Craig Sproule, founder and CEO of Metavine, will outline the challenges enterprises face in migrating legacy solutions to the cloud. He will also prese...
Most of the time there is a lot of work involved to move to the cloud, and most of that isn't really related to AWS or Azure or Google Cloud. Before we talk about public cloud vendors and DevOps tools, there are usually several technical and non-technical challenges that are connected to it and that every company needs to solve to move to the cloud. In his session at 21st Cloud Expo, Stefano Bellasio, CEO and founder of Cloud Academy Inc., will discuss what the tools, disciplines, and cultural...
The “Digital Era” is forcing us to engage with new methods to build, operate and maintain applications. This transformation also implies an evolution to more and more intelligent applications to better engage with the customers, while creating significant market differentiators. In both cases, the cloud has become a key enabler to embrace this digital revolution. So, moving to the cloud is no longer the question; the new questions are HOW and WHEN. To make this equation even more complex, most ...
With the rise of DevOps, containers are at the brink of becoming a pervasive technology in Enterprise IT to accelerate application delivery for the business. When it comes to adopting containers in the enterprise, security is the highest adoption barrier. Is your organization ready to address the security risks with containers for your DevOps environment? In his session at @DevOpsSummit at 21st Cloud Expo, Chris Van Tuin, Chief Technologist, NA West at Red Hat, will discuss: The top security r...
Agile has finally jumped the technology shark, expanding outside the software world. Enterprises are now increasingly adopting Agile practices across their organizations in order to successfully navigate the disruptive waters that threaten to drown them. In our quest for establishing change as a core competency in our organizations, this business-centric notion of Agile is an essential component of Agile Digital Transformation. In the years since the publication of the Agile Manifesto, the conn...
The nature of the technology business is forward-thinking. It focuses on the future and what’s coming next. Innovations and creativity in our world of software development strive to improve the status quo and increase customer satisfaction through speed and increased connectivity. Yet, while it's exciting to see enterprises embrace new ways of thinking and advance their processes with cutting edge technology, it rarely happens rapidly or even simultaneously across all industries.
These days, APIs have become an integral part of the digital transformation journey for all enterprises. Every digital innovation story is connected to APIs . But have you ever pondered over to know what are the source of these APIs? Let me explain - APIs sources can be varied, internal or external, solving different purposes, but mostly categorized into the following two categories. Data lakes is a term used to represent disconnected but relevant data that are used by various business units wit...
"NetApp's vision is how we help organizations manage data - delivering the right data in the right place, in the right time, to the people who need it, and doing it agnostic to what the platform is," explained Josh Atwell, Developer Advocate for NetApp, in this SYS-CON.tv interview at 20th Cloud Expo, held June 6-8, 2017, at the Javits Center in New York City, NY.
One of the biggest challenges with adopting a DevOps mentality is: new applications are easily adapted to cloud-native, microservice-based, or containerized architectures - they can be built for them - but old applications need complex refactoring. On the other hand, these new technologies can require relearning or adapting new, oftentimes more complex, methodologies and tools to be ready for production. In his general session at @DevOpsSummit at 20th Cloud Expo, Chris Brown, Solutions Marketi...
Leading companies, from the Global Fortune 500 to the smallest companies, are adopting hybrid cloud as the path to business advantage. Hybrid cloud depends on cloud services and on-premises infrastructure working in unison. Successful implementations require new levels of data mobility, enabled by an automated and seamless flow across on-premises and cloud resources. In his general session at 21st Cloud Expo, Greg Tevis, an IBM Storage Software Technical Strategist and Customer Solution Architec...
DevOps at Cloud Expo – being held October 31 - November 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA – announces that its Call for Papers is open. Born out of proven success in agile development, cloud computing, and process automation, DevOps is a macro trend you cannot afford to miss. From showcase success stories from early adopters and web-scale businesses, DevOps is expanding to organizations of all sizes, including the world's largest enterprises – and delivering real r...
You know you need the cloud, but you’re hesitant to simply dump everything at Amazon since you know that not all workloads are suitable for cloud. You know that you want the kind of ease of use and scalability that you get with public cloud, but your applications are architected in a way that makes the public cloud a non-starter. You’re looking at private cloud solutions based on hyperconverged infrastructure, but you’re concerned with the limits inherent in those technologies.
As DevOps methodologies expand their reach across the enterprise, organizations face the daunting challenge of adapting related cloud strategies to ensure optimal alignment, from managing complexity to ensuring proper governance. How can culture, automation, legacy apps and even budget be reexamined to enable this ongoing shift within the modern software factory?
"When we talk about cloud without compromise what we're talking about is that when people think about 'I need the flexibility of the cloud' - it's the ability to create applications and run them in a cloud environment that's far more flexible,” explained Matthew Finnie, CTO of Interoute, in this SYS-CON.tv interview at 20th Cloud Expo, held June 6-8, 2017, at the Javits Center in New York City, NY.