Welcome!

Microservices Expo Authors: AppDynamics Blog, Anders Wallgren, Liz McMillan, Elizabeth White, Martin Etmajer

Related Topics: Microservices Expo

Microservices Expo: Article

ESB Myth Busters: 10 Enterprise Service Bus Myths Debunked

Clarity of Definition for a Growing Phenomenon

Myth #6: Portals can be connected to back-end systems by simply using a Web service call.
While Web service calls can theoretically be used to connect portals with back-end target systems, this approach won't scale past a few back-end systems. An ESB allows the portal server to have a single interface to the bus, with the bus providing the mediation between the diverse connectivity options, protocols, security, and data formats across all of the possible back-end systems that a portal server may call upon to fulfill its requests.

Using an ESB as the layer between the portal server and the various back-end applications that the portal server needs to interact with, ESB adopters are building for the future by providing a more scalable and flexible SOA that is capable of handling the ever-expanding scope of integration as the portal project becomes more successful and business requirements change.

Myth #7: ESBs will be obsolete once BPEL is widely available.
An ESB may support multiple ways of coordinating the interaction between event-driven service invocations using formal business process definitions. BPEL (Business Process Execution Language) is one way of doing it, and there are others as well. An ESB also has itinerary-based routing, which provides a message with a list of routing instructions. These routing instructions, which represent a business process definition, are carried with the message as it travels through the Bus across service invocations. The remote ESB service containers determine where to send the message next.

Itinerary-based routing significantly contributes to the highly distributed nature of the ESB, as there is no centralized rules engine to refer back to for each step in the process. A centralized rules engine for the routing of messages, such as those offered by the typical hub-and-spoke EAI broker approach, can be a bottleneck, and also a single point of failure. The use of message itineraries, messages and process definitions is self sufficient and can therefore allow different parts of the ESB to operate independently of one another.

Message itineraries are most useful for discreet process definitions that are stateless and usually contain a finite set of steps that don't take extended periods of time to complete. Gartner refers to this type of process definition as "microflows." Simple branching within itineraries may occur based on the use of content-based routing services.

When more sophisticated process definitions are required, a process orchestration engine may be layered onto the ESB as an additional service. The process orchestration may support stateful processes, which can span long durations of time. It may also support parallel execution paths, with branching, and merging of message flow execution paths based on join conditions or transition conditions being met. Such a process engine may support BPEL, or some other process definition grammar such as ebXML BPSS. Sophisticated process orchestration can be combined with stateless itinerary-based routing to create an SOA that solves complex integration problems.

Myth #8: The ESB technology category, like so many others, seems to have come out of nowhere and is now barreling its way up the hype curve and rapidly approaching the "trough of disillusionment."
The ESB concepts were created as a result of working with IT thought leaders across a variety of industries, including manufacturing, e-marketplaces, telco, financial services, and retail. The ESB as a concept was born out of a necessity, based on their desire for a new approach over distributed computing models and EAI technologies they had already been moderately successful with. These IT thought leaders all came with a common request: "We really like your distributed messaging infrastructure, and we would like to build upon it a standards-based event-driven SOA for integration. We would like it to include things like Web services, XML data transformation, content-based routing, and a service invocation model based on distributed process coordination." Because of this, the concepts presented in the ESB architecture are sound; they are grounded in reality. Also because of this, ESB technology has been adopted as it has been built. There are 100s of ESB deployments already in use today in areas such as supply chain and logistics automation, straight through processing in financial services, real-time provisioning in Telco, and remote storefront integration in retail.

Myth #9: ESBs are simply plumbing and do not provide sophisticated tooling, such as a graphical editor for designing business process flows.
There is a new breed of IDE, which Gartner Group refers to as an ISE (integrated services environment), that allows you to design, configure, test, and debug the integration services that you develop when building an SOA with an ESB. Using a graphical interface, an integration architect draws diagrams using UML notation to describe process definitions. You may also use the ISE to graphically create data transformations between different data formats, and create and debug XSLT style- sheets.

Myth #10: An ESB container can be implemented using an EJB container.
One of the key components of an ESB architecture is a highly distributed, lightweight service container. The service container allows the hosting of integration components as event-driven services, such as a content-based routing service that applies an XPath expression to an XML message to determine where to route it next. The service container can also host custom services or specialized adapters for hooking into packaged applications.

Unlike its distant cousins, the app server container and the integration broker, the ESB service container allows the selective deployment of integration services exactly when and where you need them, and nothing more. In contrast, you need to install an entire appserver stack everywhere that an individual piece of integration functionality is needed. This results in what is referred to as the "appservers everywhere" problem. There is an unnecessarily high cost in licensing, installation, and cost of ownership over time associated with this practice.

The mantra of the ESB is "configuration rather than coding." In an application server-centric approach to integration, you typically write code to describe the dependencies between services. The EJB model follows the client/server model of interaction, which usually results in tightly coupled interfaces between services in an SOA, which is built into code, and compiled into class files that need to be modified and redeployed every time a change needs to be made.

In an ESB, a service is configured with information regarding its input and output channels for sending and receiving message-based request/response patterns and one-way event notifications that are then coordinated by the surrounding invocation framework - not by the service itself.

An ESB service can be configured and deployed, by merely supplying it with the XSL stylesheets, XPath expression, scripts, and parameters which are read in from a configuration repository. Once deployed, the implementation is remarkably resilient to change.

Summary
To sum this up, make sure that your understanding of ESB contains these things:

  • An ESB provides the backbone for building an enterprise SOA-based integration environment.
  • The evolving WS-* specifications will help make ESBs even more interoperable than they are today. Adopting an ESB today will allow you to build for the future and be capable of adapting to the WS-* specifications as they become commercially viable.
  • ESB is not just an abstract pattern. It is a product category with a distinct definition and many vendor offerings.
  • ESBs and application servers are highly complementary.
  • ESBs can help portal server integration to back-end systems by providing the necessary diversity in connectivity and scalable infrastructure.
  • ESBs provide many choices for coordinating service interactions.
  • ESB technology is grounded in reality and is already being adopted by many industries.
  • ESBs can provide the higher-level visual tools for integrating services in an ISE environment.
  • ESBs provide a service container environment that is lightweight, configurable, and highly distributable.

More Stories By Dave Chappell

David Chappell is vice president and chief technologist for SOA at Oracle Corporation, and is driving the vision for Oracle’s SOA on App Grid initiative.

Comments (8) View Comments

Share your thoughts on this story.

Add your comment
You must be signed in to add a comment. Sign-in | Register

In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.


Most Recent Comments
Charlesy 07/03/13 07:28:00 AM EDT

An old post, but worth a small correction. Comparison with competitor products is always dangerous. You need to be very sure of your territory, and unfortunately, although David's description of BizTalk Server has some validity, it isn't 100% accurate. For example, the transformation services absolutely can be invoked separately to the rest of BTS via lightweight services, and load balanced across different boxes. More recent versions of BTS provide pre-built generic WCF and ASMX transformation web services as a courtesy to developers (reduces the need to build custom transformation services). You can use Windows Server AppFabric tools to create BizTalk maps in non-BizTalk projects - e.g., projects that define lightweight transformation services.

David doesn't clearly spell out what he means by 'cost...of the entire BizTalk Server'. If he means licencing cost, then he hits the mark, somewhat. BTS is certainly licensed in a fashion that encourages distribution and load balancing over a small farm of centralized servers, in contrast to the highly distributed approach advocated by Sonic. You can only invoke BTS transformation services on licensed boxes, so from that perspective, he is correct. However, invoking the transformation services does not require loading additional irrelevant BTS plumbing into memory. There is no heavy-weight performance cost imposed by code bloat, or anything like that!

BizTalk maps are emitted as code components containing an executable XSLT resource. You can distribute maps as freely as you wish and invoke the tranforms via code. Obviously, direct invocation in this case assumes the use of either .NET or Mono, although Java/.NET bridges could be used. If you use BizTalk Server's mapping tools to create maps, you may end up with a dependency on BizTalk-specific scripted components invoked in the XSLT, which ties you to licensed BizTalk Boxes. However, it is pretty easy to avoid this if you wish.

One thing Sonic has which BizTalk really does not is dynamic management of code deployment into the run-time environment. BizTalk Server has an built-in code repository, but this is simply a mechanism for managing and storing compiled artifacts and resources for the purpose of exporting installation packages. You still have to manually install those packages or deploy them via additional script. Frankly, though, this is rarely a significant drawback. The types of solution built using BizTalk Server tend to warrant close attention to managing dynamic deployment across a distributed environment using other frameworks and tools, and BizTalk Server plays well with the relevant frameworks. There is even a community-built deployment framework specifically designed for BizTalk Server.

yt67 03/03/05 07:09:34 AM EST

Myth-busting: always entertaining.

Jason 03/02/05 09:16:29 AM EST

A good read!

Javier Camara 02/10/05 04:19:02 AM EST

(This same feedback also posted to another WSJ article about ESBs)

I agree in that the ESB concept is over-hyped. For me, a SOA makes sense if it is viewed as a constellation of web services interacting among them. For this, something like a UDDI server is required for each service locating each other.

For me, all this (i.e. services + directory) is just enough if only synchronous communications are used. If asynchronous communications are needed, then you need also publish/subscribe and store-and-forward, i.e. roughly what a MOM does. You can call it an ESB if you want, although I think this concept in the market encompasses several roles:
1. Publish/subscribe to messages
2. Store-and-forward messages
3. Route messages
4. Transform messages

An interesting thing to note is to implement points 1. and 2. you do *not* need business logic, while to implement 3. and 4. you do.

As I said, I see roles 1 and 2 required in SOAs with asynchronous interactions.

Roles 3 and 4 are also needed in many cases, mainly for integrating disparate systems. However, my main point against an ESB is that, in order to perform these roles, you do *NOT* need of a new, special concept like the ESB. *Any* service in the constellation of services can perform both routing and transformation. It can range from being a single component like an ESB (which I think is a bad idea), or it can just be a set of services (e.g. a different service performing specific adaptation for a system being integrated).

For me, using a single ESB for 3. and 4. breaks the beauty of the SOA idea. You are supposed to made all your data and business logic of your organization available as services in order to be reused, and suddenly you put on top an ESB in which you put *more* business logic (routing and transformation). So my point is that this should be implemented just by means of regular services, and not by specific, central-piece new components called ESBs.

Now, if for implementing routing and transformation you want to use Tibco, WebSphere or whatever, fine - however, the logic created by these products should be at the same level as the other services in the SOA, and not above.

So I am not saying that orchestrating tools are not useful. They are. Only, they are not *imprescindible*; and at any rate they should be viewed just as more services in the SOA. However, this does not fit the marketing strategy of ESB vendors which show its ESB as an *enabler* of a SOA, instead of just one more *component* of it.

Dave Chappell 02/03/05 09:54:43 PM EST

We (Sonic Software) didn't re-lable our product to support the ESB wave, we actually invented the concept. We then worked with the analyst and journalist community to help create industry awareness of the new concepts that are introduced by ESB, which has resulted in a whole new product category.

I would agree with you that there is a great deal of hype right now due to lack of understanding of what ESB is, which is compounded by the number of traditional middleware and EAI vendors who have clamoured to get ESB in their marketing literature without having a full understanding of what it means to have an ESB. Your comment about middleware with new clothes is well taken. You might get that impression depending on where you learned about what an ESB is. That is exactly what I am trying to point out with myth #1 in this article.

A certain amount of hype is expected when a technology category begins to take hold and gain traction within serious IT projects. This can be disruptive to the industry as a whole. This is also the primary reason why I wrote the OReilly book on the subject of ESB--to act as the definitive guide to help educate and provide clarity on what makes up an ESB. Please don't shoot the concept of ESB down until you have had a chance to understand it.
Dave

Larry 02/03/05 04:15:19 AM EST

Not surprising that the representative of a company who over-hyped ESB in the first place, and relabeled their own product ESB to catch the service wave, should now try to claim that anyone who saw through the hype is guilty of spreading myths.
ESB is just the middleware emporor's new clothes.

@MicroservicesExpo Stories
Microservices are a type of software architecture where large applications are made up of small, self-contained units working together through APIs that are not dependent on a specific language. Each service has a limited scope, concentrates on a specific task and is highly independent. This setup allows IT managers and developers to build systems in a modular way. In his book, “Building Microservices,” Sam Newman said microservices are small, focused components built to do a single thing very w...
How is your DevOps transformation coming along? How do you measure Agility? Reliability? Efficiency? Quality? Success?! How do you optimize your processes? This morning on #c9d9 we talked about some of the metrics that matter for the different stakeholders throughout the software delivery pipeline. Our panelists shared their best practices.
The (re?)emergence of Microservices was especially prominent in this week’s news. What are they good for? do they make sense for your application? should you take the plunge? and what do Microservices mean for your DevOps and Continuous Delivery efforts? Continue reading for more on Microservices, containers, DevOps culture, and more top news from the past week. As always, stay tuned to all the news coming from@ElectricCloud on DevOps and Continuous Delivery throughout the week and retweet/favo...
The cloud promises new levels of agility and cost-savings for Big Data, data warehousing and analytics. But it’s challenging to understand all the options – from IaaS and PaaS to newer services like HaaS (Hadoop as a Service) and BDaaS (Big Data as a Service). In her session at @BigDataExpo at @ThingsExpo, Hannah Smalltree, a director at Cazena, will provide an educational overview of emerging “as-a-service” options for Big Data in the cloud. This is critical background for IT and data profes...
Father business cycles and digital consumers are forcing enterprises to respond faster to customer needs and competitive demands. Successful integration of DevOps and Agile development will be key for business success in today’s digital economy. In his session at DevOps Summit, Pradeep Prabhu, Co-Founder & CEO of Cloudmunch, covered the critical practices that enterprises should consider to seamlessly integrate Agile and DevOps processes, barriers to implementing this in the enterprise, and pr...
In a previous article, I demonstrated how to effectively and efficiently install the Dynatrace Application Monitoring solution using Ansible. In this post, I am going to explain how to achieve the same results using Chef with our official dynatrace cookbook available on GitHub and on the Chef Supermarket. In the following hands-on tutorial, we’ll also apply what we see as good practice on working with and extending our deployment automation blueprints to suit your needs.
If we look at slow, traditional IT and jump to the conclusion that just because we found its issues intractable before, that necessarily means we will again, then it’s time for a rethink. As a matter of fact, the world of IT has changed over the last ten years or so. We’ve been experiencing unprecedented innovation across the board – innovation in technology as well as in how people organize and accomplish tasks. Let’s take a look at three differences between today’s modern, digital context...
The principles behind DevOps are not new - for decades people have been automating system administration and decreasing the time to deploy apps and perform other management tasks. However, only recently did we see the tools and the will necessary to share the benefits and power of automation with a wider circle of people. In his session at DevOps Summit, Bernard Sanders, Chief Technology Officer at CloudBolt Software, explored the latest tools including Puppet, Chef, Docker, and CMPs needed to...
Sensors and effectors of IoT are solving problems in new ways, but small businesses have been slow to join the quantified world. They’ll need information from IoT using applications as varied as the businesses themselves. In his session at @ThingsExpo, Roger Meike, Distinguished Engineer, Director of Technology Innovation at Intuit, showed how IoT manufacturers can use open standards, public APIs and custom apps to enable the Quantified Small Business. He used a Raspberry Pi to connect sensors...
Let’s face it, embracing new storage technologies, capabilities and upgrading to new hardware often adds complexity and increases costs. In his session at 18th Cloud Expo, Seth Oxenhorn, Vice President of Business Development & Alliances at FalconStor, will discuss how a truly heterogeneous software-defined storage approach can add value to legacy platforms and heterogeneous environments. The result reduces complexity, significantly lowers cost, and provides IT organizations with improved effi...
Data-as-a-Service is the complete package for the transformation of raw data into meaningful data assets and the delivery of those data assets. In her session at 18th Cloud Expo, Lakshmi Randall, an industry expert, analyst and strategist, will address: What is DaaS (Data-as-a-Service)? Challenges addressed by DaaS Vendors that are enabling DaaS Architecture options for DaaS
One of the bewildering things about DevOps is integrating the massive toolchain including the dozens of new tools that seem to crop up every year. Part of DevOps is Continuous Delivery and having a complex toolchain can add additional integration and setup to your developer environment. In his session at @DevOpsSummit at 18th Cloud Expo, Miko Matsumura, Chief Marketing Officer of Gradle Inc., will discuss which tools to use in a developer stack, how to provision the toolchain to minimize onboa...
SYS-CON Events announced today that Catchpoint Systems, Inc., a provider of innovative web and infrastructure monitoring solutions, has been named “Silver Sponsor” of SYS-CON's DevOps Summit at 18th Cloud Expo New York, which will take place June 7-9, 2016, at the Javits Center in New York City, NY. Catchpoint is a leading Digital Performance Analytics company that provides unparalleled insight into customer-critical services to help consistently deliver an amazing customer experience. Designed...
With the proliferation of both SQL and NoSQL databases, organizations can now target specific fit-for-purpose database tools for their different application needs regarding scalability, ease of use, ACID support, etc. Platform as a Service offerings make this even easier now, enabling developers to roll out their own database infrastructure in minutes with minimal management overhead. However, this same amount of flexibility also comes with the challenges of picking the right tool, on the right ...
SYS-CON Events announced today that Interoute, owner-operator of one of Europe's largest networks and a global cloud services platform, has been named “Bronze Sponsor” of SYS-CON's 18th Cloud Expo, which will take place on June 7-9, 2015 at the Javits Center in New York, New York. Interoute is the owner-operator of one of Europe's largest networks and a global cloud services platform which encompasses 12 data centers, 14 virtual data centers and 31 colocation centers, with connections to 195 ad...
Microservices are all the rage right now — and the industry is still learning, experimenting, and developing patterns, for successfully designing, deploying and managing Microservices in the real world. Are you considering jumping on the Microservices-wagon? Do Microservices make sense for your particular use case? What are some of the “gotchas” you should be aware of? This morning on #c9d9 we had experts from popular chat app Kik, SMB SaaS platform Yodle and hosted CI solution Semaphore sha...
SYS-CON Events announced today that Commvault, a global leader in enterprise data protection and information management, has been named “Bronze Sponsor” of SYS-CON's 18th International Cloud Expo, which will take place on June 7–9, 2016, at the Javits Center in New York City, NY, and the 19th International Cloud Expo, which will take place on November 1–3, 2016, at the Santa Clara Convention Center in Santa Clara, CA. Commvault is a leading provider of data protection and information management...
With an estimated 50 billion devices connected to the Internet by 2020, several industries will begin to expand their capabilities for retaining end point data at the edge to better utilize the range of data types and sheer volume of M2M data generated by the Internet of Things. In his session at @ThingsExpo, Don DeLoach, CEO and President of Infobright, will discuss the infrastructures businesses will need to implement to handle this explosion of data by providing specific use cases for filte...
SYS-CON Events announced today that Avere Systems, a leading provider of enterprise storage for the hybrid cloud, will exhibit at SYS-CON's 18th International Cloud Expo®, which will take place on June 7-9, 2016, at the Javits Center in New York City, NY. Avere delivers a more modern architectural approach to storage that doesn’t require the overprovisioning of storage capacity to achieve performance, overspending on expensive storage media for inactive data or the overbuilding of data centers ...
CIOs and those charged with running IT Operations are challenged to deliver secure, audited, and reliable compute environments for the applications and data for the business. Behind the scenes these tasks are often accomplished by following onerous time-consuming processes and often the management of these environments and processes will be outsourced to multiple IT service providers. In addition, the division of work is often siloed into traditional "towers" that are not well integrated for cro...