|By Dave Chappell||
|May 25, 2005 07:30 PM EDT||
Since releasing my latest book, Enterprise Service Bus (O'Reilly Media, 2004), I have been doing a fair amount of visiting corporations, conducting seminars, and generally discussing with enterprise architects the subject of enterprise service-oriented architecture (SOA) and how an enterprise service bus (ESB) backbone can be leveraged to provide a framework for an enterprise SOA. Along the way, I have been asked many questions about the nature of an ESB. I have also fended off some misconceptions that have been growing in the general IT population regarding what an ESB is and when, where, and how it can be used. I have gathered together the most popular questions and misconceptions, and offer some clarity in the form of a "top ten" list.
Myth #1. ESB is just a new name for EAI.
While many IT architecture groups are focusing on building SOAs, they still inevitably beg the question of "how is ESB different from EAI?" An ESB is an infrastructure for building an enterprise SOA, and is capable of being used in a more general way than a conventional EAI broker. According to Forrester Research, an ESB helps enterprises obtain the value of SOA by increasing connectivity, adding flexibility that speeds change, and providing greater control over use of the important resources that it binds.
An ESB can be used to handle integration projects that have traditionally been relegated to EAI tools. However, an ESB can also be used for establishing B2B relationships across companies.
An ESB provides EAI capabilities, but is based on a fundamentally different architecture that is providing the basis of an industry transition from traditional integration to coordinated service interaction. EAI brokers are historically implemented as a monolithic stack, using centralized hub-and-spoke architecture.
An ESB provides the same base functionality as an EAI broker - connectivity, application adapters, routing of messages based on rules, and data transformation engine - yet, in an ESB, these capabilities are themselves SOA based in that they are spread out across the bus in a highly distributed fashion and hosted in separately deployable service containers. This allows the selective deployment of integration broker functionality exactly where you need it, with no additional over-bloating where it's not needed. The distributed nature of the ESB container model allows the independent scalability of integration components, which are plugged into your SOA as event-driven services on an as needed basis.
In order for an integration broker to be truly capable of supporting an SOA, and to be considered a true ESB, it would need to have its base functions broken up into their constituent parts, which would then be capable of being separately deployed across the bus while working together in harmony as necessary.
Let's use an example of an XSLT-based transformation engine that accepts an incoming XML document and applies an XSLT style sheet to it in order to produce an outgoing document in another XML format. I can tell you that there is nothing that can chew up computing resources more than the parsing and manipulation of XML. If this particular XSLT transformation sits between two popular applications that communicate regularly with each other, then that individual transformation can become a performance and scalability bottleneck. If you are using a monolithic hub-and-spoke integration broker approach, in order to remove the bottleneck and scale up the deployment you would need to either install that integration broker on one big powerful machine, or install the integration broker across multiple machines - just to support that one transform scenario! All the while, the other integration broker capabilities, such as the execution of routing rules, are competing for the same computing resources as the transformation operation.
In contrast to the monolithic hub-and-spoke architecture of an integration broker, the foundational core of an ESB provides a distributed services architecture. This architecture is built for integration and has the ability for integration broker functionality, such as message routing, data transformation, and application adapters to be selectively deployed on an as-needed basis.These are separate integration services that are a natural part of an SOA processing pipeline across the bus.
An individual XSLT transformation can be deployed as a service in its own ESB service container, and multiple instances of that container can be load-balanced across many machines. If the ESB container implementation is cross-platform, then you can be flexible as to what kinds of machines you spread the transform service across - Linux boxes, Solaris boxes, Windows boxes, and so on. And for those of you who don't find solace in the architectural purity of this discussion, consider this: the ESB vendors who are leading the charge in defining and delivering ESB products are also putting forth a license model where there is no additional cost for deploying as many of these lightweight ESB service containers as necessary to get the job done.
The integration services provided by the ESB can be combined with other services into SOA-based processing pipelines that can span business boundaries. The distributed services in an ESB can be combined with itinerary-based routing (see Myth #7) to allow self-directed, message-oriented service interactions, which allow different parts of the ESB to operate independently of one another, without relying on a centralized routing engine.
|Charlesy 07/03/13 07:28:00 AM EDT|
An old post, but worth a small correction. Comparison with competitor products is always dangerous. You need to be very sure of your territory, and unfortunately, although David's description of BizTalk Server has some validity, it isn't 100% accurate. For example, the transformation services absolutely can be invoked separately to the rest of BTS via lightweight services, and load balanced across different boxes. More recent versions of BTS provide pre-built generic WCF and ASMX transformation web services as a courtesy to developers (reduces the need to build custom transformation services). You can use Windows Server AppFabric tools to create BizTalk maps in non-BizTalk projects - e.g., projects that define lightweight transformation services.
David doesn't clearly spell out what he means by 'cost...of the entire BizTalk Server'. If he means licencing cost, then he hits the mark, somewhat. BTS is certainly licensed in a fashion that encourages distribution and load balancing over a small farm of centralized servers, in contrast to the highly distributed approach advocated by Sonic. You can only invoke BTS transformation services on licensed boxes, so from that perspective, he is correct. However, invoking the transformation services does not require loading additional irrelevant BTS plumbing into memory. There is no heavy-weight performance cost imposed by code bloat, or anything like that!
BizTalk maps are emitted as code components containing an executable XSLT resource. You can distribute maps as freely as you wish and invoke the tranforms via code. Obviously, direct invocation in this case assumes the use of either .NET or Mono, although Java/.NET bridges could be used. If you use BizTalk Server's mapping tools to create maps, you may end up with a dependency on BizTalk-specific scripted components invoked in the XSLT, which ties you to licensed BizTalk Boxes. However, it is pretty easy to avoid this if you wish.
One thing Sonic has which BizTalk really does not is dynamic management of code deployment into the run-time environment. BizTalk Server has an built-in code repository, but this is simply a mechanism for managing and storing compiled artifacts and resources for the purpose of exporting installation packages. You still have to manually install those packages or deploy them via additional script. Frankly, though, this is rarely a significant drawback. The types of solution built using BizTalk Server tend to warrant close attention to managing dynamic deployment across a distributed environment using other frameworks and tools, and BizTalk Server plays well with the relevant frameworks. There is even a community-built deployment framework specifically designed for BizTalk Server.
|yt67 03/03/05 07:09:34 AM EST|
Myth-busting: always entertaining.
|Jason 03/02/05 09:16:29 AM EST|
A good read!
|Javier Camara 02/10/05 04:19:02 AM EST|
(This same feedback also posted to another WSJ article about ESBs)
I agree in that the ESB concept is over-hyped. For me, a SOA makes sense if it is viewed as a constellation of web services interacting among them. For this, something like a UDDI server is required for each service locating each other.
For me, all this (i.e. services + directory) is just enough if only synchronous communications are used. If asynchronous communications are needed, then you need also publish/subscribe and store-and-forward, i.e. roughly what a MOM does. You can call it an ESB if you want, although I think this concept in the market encompasses several roles:
An interesting thing to note is to implement points 1. and 2. you do *not* need business logic, while to implement 3. and 4. you do.
As I said, I see roles 1 and 2 required in SOAs with asynchronous interactions.
Roles 3 and 4 are also needed in many cases, mainly for integrating disparate systems. However, my main point against an ESB is that, in order to perform these roles, you do *NOT* need of a new, special concept like the ESB. *Any* service in the constellation of services can perform both routing and transformation. It can range from being a single component like an ESB (which I think is a bad idea), or it can just be a set of services (e.g. a different service performing specific adaptation for a system being integrated).
For me, using a single ESB for 3. and 4. breaks the beauty of the SOA idea. You are supposed to made all your data and business logic of your organization available as services in order to be reused, and suddenly you put on top an ESB in which you put *more* business logic (routing and transformation). So my point is that this should be implemented just by means of regular services, and not by specific, central-piece new components called ESBs.
Now, if for implementing routing and transformation you want to use Tibco, WebSphere or whatever, fine - however, the logic created by these products should be at the same level as the other services in the SOA, and not above.
So I am not saying that orchestrating tools are not useful. They are. Only, they are not *imprescindible*; and at any rate they should be viewed just as more services in the SOA. However, this does not fit the marketing strategy of ESB vendors which show its ESB as an *enabler* of a SOA, instead of just one more *component* of it.
|Dave Chappell 02/03/05 09:54:43 PM EST|
We (Sonic Software) didn't re-lable our product to support the ESB wave, we actually invented the concept. We then worked with the analyst and journalist community to help create industry awareness of the new concepts that are introduced by ESB, which has resulted in a whole new product category.
I would agree with you that there is a great deal of hype right now due to lack of understanding of what ESB is, which is compounded by the number of traditional middleware and EAI vendors who have clamoured to get ESB in their marketing literature without having a full understanding of what it means to have an ESB. Your comment about middleware with new clothes is well taken. You might get that impression depending on where you learned about what an ESB is. That is exactly what I am trying to point out with myth #1 in this article.
A certain amount of hype is expected when a technology category begins to take hold and gain traction within serious IT projects. This can be disruptive to the industry as a whole. This is also the primary reason why I wrote the OReilly book on the subject of ESB--to act as the definitive guide to help educate and provide clarity on what makes up an ESB. Please don't shoot the concept of ESB down until you have had a chance to understand it.
|Larry 02/03/05 04:15:19 AM EST|
Not surprising that the representative of a company who over-hyped ESB in the first place, and relabeled their own product ESB to catch the service wave, should now try to claim that anyone who saw through the hype is guilty of spreading myths.
I read an insightful article this morning from Bernard Golden on DZone discussing the DevOps conundrum facing many enterprises today – is it better to build your own DevOps tools or go commercial? For Golden, the question arose from his observations at a number of DevOps Days events he has attended, where typically the audience is composed of startup professionals: “I have to say, though, that a typical feature of most presentations is a recitation of the various open source products and compo...
May. 27, 2015 11:45 PM EDT Reads: 1,096
In the first four parts of this series I presented an introduction to microservices along with a handful of emerging microservices patterns, and a discussion of some of the downsides and challenges to using microservices. The most recent installment of this series looked at ten ways that PaaS facilitates microservices development and adoption. In this post I’ll cover some words of wisdom, advice intended for individuals, teams, and organizations considering a move to microservices. I've gleaned...
May. 27, 2015 10:30 PM EDT Reads: 3,061
Many people recognize DevOps as an enormous benefit – faster application deployment, automated toolchains, support of more granular updates, better cooperation across groups. However, less appreciated is the journey enterprise IT groups need to make to achieve this outcome. The plain fact is that established IT processes reflect a very different set of goals: stability, infrequent change, hands-on administration, and alignment with ITIL. So how does an enterprise IT organization implement change...
May. 27, 2015 08:00 PM EDT Reads: 1,236
Containers Expo Blog covers the world of containers, as this lightweight alternative to virtual machines enables developers to work with identical dev environments and stacks. Containers Expo Blog offers top articles, news stories, and blog posts from the world's well-known experts and guarantees better exposure for its authors than any other publication. Bookmark Containers Expo Blog ▸ Here Follow new article posts on Twitter at @ContainersExpo
May. 27, 2015 08:00 PM EDT Reads: 1,791
Right off the bat, Newman advises that we should "think of microservices as a specific approach for SOA in the same way that XP or Scrum are specific approaches for Agile Software development". These analogies are very interesting because my expectation was that microservices is a pattern. So I might infer that microservices is a set of process techniques as opposed to an architectural approach. Yet in the book, Newman clearly includes some elements of concept model and architecture as well as p...
May. 27, 2015 07:00 PM EDT Reads: 3,842
There’s a lot of discussion around managing outages in production via the likes of DevOps principles and the corresponding software development lifecycles that does enable higher quality output from development, however, one cannot lay all blame for “bugs” and failures at the feet of those responsible for coding and development. As developers incorporate features and benefits of these paradigm shift, there is a learning curve and a point of not-knowing-what-is-not-known. Sometimes, the only way ...
May. 27, 2015 07:00 PM EDT Reads: 1,873
SYS-CON Events announced today that MetraTech, now part of Ericsson, has been named “Silver Sponsor” of SYS-CON's 16th International Cloud Expo®, which will take place on June 9–11, 2015, at the Javits Center in New York, NY. Ericsson is the driving force behind the Networked Society- a world leader in communications infrastructure, software and services. Some 40% of the world’s mobile traffic runs through networks Ericsson has supplied, serving more than 2.5 billion subscribers.
May. 27, 2015 06:45 PM EDT Reads: 2,222
This is the final installment of the six-part series Microservices and PaaS. It seems like forever since I attended Adrian Cockroft's meetup focusing on microservices. It's actually only been a couple of months, but much has happened since then: countless articles, meetups, and conference sessions focusing on microservices have been delivered, many meetings and design efforts at companies moving towards a microservices-based approach have been endured, and five installments of this blog series ...
May. 27, 2015 05:45 PM EDT Reads: 2,622
Over the years, a variety of methodologies have emerged in order to overcome the challenges related to project constraints. The successful use of each methodology seems highly context-dependent. However, communication seems to be the common denominator of the many challenges that project management methodologies intend to resolve. In this respect, Information and Communication Technologies (ICTs) can be viewed as powerful tools for managing projects. Few research papers have focused on the way...
May. 27, 2015 05:00 PM EDT Reads: 2,167
As the world moves from DevOps to NoOps, application deployment to the cloud ought to become a lot simpler. However, applications have been architected with a much tighter coupling than it needs to be which makes deployment in different environments and migration between them harder. The microservices architecture, which is the basis of many new age distributed systems such as OpenStack, Netflix and so on is at the heart of CloudFoundry – a complete developer-oriented Platform as a Service (PaaS...
May. 27, 2015 05:00 PM EDT Reads: 2,089
How can you compare one technology or tool to its competitors? Usually, there is no objective comparison available. So how do you know which is better? Eclipse or IntelliJ IDEA? Java EE or Spring? C# or Java? All you can usually find is a holy war and biased comparisons on vendor sites. But luckily, sometimes, you can find a fair comparison. How does this come to be? By having it co-authored by the stakeholders. The binary repository comparison matrix is one of those rare resources. It is edite...
May. 27, 2015 04:00 PM EDT Reads: 2,070
SYS-CON Events announced today that O'Reilly Media has been named “Media Sponsor” of SYS-CON's 16th International Cloud Expo®, which will take place on June 9–11, 2015, at the Javits Center in New York City, NY. O'Reilly Media spreads the knowledge of innovators through its books, online services, magazines, and conferences. Since 1978, O'Reilly Media has been a chronicler and catalyst of cutting-edge development, homing in on the technology trends that really matter and spurring their adoption...
May. 27, 2015 03:37 PM EDT Reads: 763
The widespread success of cloud computing is driving the DevOps revolution in enterprise IT. Now as never before, development teams must communicate and collaborate in a dynamic, 24/7/365 environment. There is no time to wait for long development cycles that produce software that is obsolete at launch. DevOps may be disruptive, but it is essential. The DevOps Summit at Cloud Expo – to be held June 3-5, 2015, at the Javits Center in New York City – will expand the DevOps community, enable a wide...
May. 27, 2015 03:00 PM EDT Reads: 2,787
In her General Session at 15th Cloud Expo, Anne Plese, Senior Consultant, Cloud Product Marketing, at Verizon Enterprise, focused on finding the right mix of renting vs. buying Oracle capacity to scale to meet business demands, and offer validated Oracle database TCO models for Oracle development and testing environments. Anne Plese is a marketing and technology enthusiast/realist with over 19+ years in high tech. At Verizon Enterprise, she focuses on driving growth for the Verizon Cloud platfo...
May. 27, 2015 03:00 PM EDT Reads: 5,697
I’ve been thinking a bit about microservices (μServices) recently. My immediate reaction is to think: “Isn’t this just yet another new term for the same stuff, Web Services->SOA->APIs->Microservices?” Followed shortly by the thought, “well yes it is, but there are some important differences/distinguishing factors.” Microservices is an evolutionary paradigm born out of the need for simplicity (i.e., get away from the ESB) and alignment with agile (think DevOps) and scalable (think Containerizati...
May. 27, 2015 03:00 PM EDT Reads: 1,722
Cloud Expo, Inc. has announced today that Andi Mann returns to DevOps Summit 2015 as Conference Chair. The 4th International DevOps Summit will take place on June 9-11, 2015, at the Javits Center in New York City. "DevOps is set to be one of the most profound disruptions to hit IT in decades," said Andi Mann. "It is a natural extension of cloud computing, and I have seen both firsthand and in independent research the fantastic results DevOps delivers. So I am excited to help the great team at ...
May. 27, 2015 02:00 PM EDT Reads: 2,311
Software is eating the world. Companies that were not previously in the technology space now find themselves competing with Google and Amazon on speed of innovation. As the innovation cycle accelerates, companies must embrace rapid and constant change to both applications and their infrastructure, and find a way to deliver speed and agility of development without sacrificing reliability or efficiency of operations. In her Day 2 Keynote DevOps Summit, Victoria Livschitz, CEO of Qubell, discussed...
May. 27, 2015 02:00 PM EDT Reads: 5,856
The 5th International DevOps Summit, co-located with 17th International Cloud Expo – being held November 3-5, 2015, at the Santa Clara Convention Center in Santa Clara, CA – announces that its Call for Papers is open. Born out of proven success in agile development, cloud computing, and process automation, DevOps is a macro trend you cannot afford to miss. From showcase success stories from early adopters and web-scale businesses, DevOps is expanding to organizations of all sizes, including the...
May. 27, 2015 02:00 PM EDT Reads: 4,712
How does one bridge the gap between traditional enterprise storage infrastructures and the private, hybrid, and public cloud? In his session at 15th Cloud Expo, Dan Pollack, Chief Architect of Storage Operations at AOL Inc., examed the workload differences and required changes to reuse existing knowledge and components when building and using a cloud infrastructure. He also looked into the operational considerations, tool requirements, and behavioral changes required for private cloud storage s...
May. 27, 2015 02:00 PM EDT Reads: 2,974
The speed of product development has increased massively in the past 10 years. At the same time our formal secure development and SDL methodologies have fallen behind. This forces product developers to choose between rapid release times and security. In his session at DevOps Summit, Michael Murray, Director of Cyber Security Consulting and Assessment at GE Healthcare, examined the problems and presented some solutions for moving security into the DevOps lifecycle to ensure that we get fast AND ...
May. 27, 2015 02:00 PM EDT Reads: 5,141