|By Dave Chappell||
|May 25, 2005 07:30 PM EDT||
Since releasing my latest book, Enterprise Service Bus (O'Reilly Media, 2004), I have been doing a fair amount of visiting corporations, conducting seminars, and generally discussing with enterprise architects the subject of enterprise service-oriented architecture (SOA) and how an enterprise service bus (ESB) backbone can be leveraged to provide a framework for an enterprise SOA. Along the way, I have been asked many questions about the nature of an ESB. I have also fended off some misconceptions that have been growing in the general IT population regarding what an ESB is and when, where, and how it can be used. I have gathered together the most popular questions and misconceptions, and offer some clarity in the form of a "top ten" list.
Myth #1. ESB is just a new name for EAI.
While many IT architecture groups are focusing on building SOAs, they still inevitably beg the question of "how is ESB different from EAI?" An ESB is an infrastructure for building an enterprise SOA, and is capable of being used in a more general way than a conventional EAI broker. According to Forrester Research, an ESB helps enterprises obtain the value of SOA by increasing connectivity, adding flexibility that speeds change, and providing greater control over use of the important resources that it binds.
An ESB can be used to handle integration projects that have traditionally been relegated to EAI tools. However, an ESB can also be used for establishing B2B relationships across companies.
An ESB provides EAI capabilities, but is based on a fundamentally different architecture that is providing the basis of an industry transition from traditional integration to coordinated service interaction. EAI brokers are historically implemented as a monolithic stack, using centralized hub-and-spoke architecture.
An ESB provides the same base functionality as an EAI broker - connectivity, application adapters, routing of messages based on rules, and data transformation engine - yet, in an ESB, these capabilities are themselves SOA based in that they are spread out across the bus in a highly distributed fashion and hosted in separately deployable service containers. This allows the selective deployment of integration broker functionality exactly where you need it, with no additional over-bloating where it's not needed. The distributed nature of the ESB container model allows the independent scalability of integration components, which are plugged into your SOA as event-driven services on an as needed basis.
In order for an integration broker to be truly capable of supporting an SOA, and to be considered a true ESB, it would need to have its base functions broken up into their constituent parts, which would then be capable of being separately deployed across the bus while working together in harmony as necessary.
Let's use an example of an XSLT-based transformation engine that accepts an incoming XML document and applies an XSLT style sheet to it in order to produce an outgoing document in another XML format. I can tell you that there is nothing that can chew up computing resources more than the parsing and manipulation of XML. If this particular XSLT transformation sits between two popular applications that communicate regularly with each other, then that individual transformation can become a performance and scalability bottleneck. If you are using a monolithic hub-and-spoke integration broker approach, in order to remove the bottleneck and scale up the deployment you would need to either install that integration broker on one big powerful machine, or install the integration broker across multiple machines - just to support that one transform scenario! All the while, the other integration broker capabilities, such as the execution of routing rules, are competing for the same computing resources as the transformation operation.
In contrast to the monolithic hub-and-spoke architecture of an integration broker, the foundational core of an ESB provides a distributed services architecture. This architecture is built for integration and has the ability for integration broker functionality, such as message routing, data transformation, and application adapters to be selectively deployed on an as-needed basis.These are separate integration services that are a natural part of an SOA processing pipeline across the bus.
An individual XSLT transformation can be deployed as a service in its own ESB service container, and multiple instances of that container can be load-balanced across many machines. If the ESB container implementation is cross-platform, then you can be flexible as to what kinds of machines you spread the transform service across - Linux boxes, Solaris boxes, Windows boxes, and so on. And for those of you who don't find solace in the architectural purity of this discussion, consider this: the ESB vendors who are leading the charge in defining and delivering ESB products are also putting forth a license model where there is no additional cost for deploying as many of these lightweight ESB service containers as necessary to get the job done.
The integration services provided by the ESB can be combined with other services into SOA-based processing pipelines that can span business boundaries. The distributed services in an ESB can be combined with itinerary-based routing (see Myth #7) to allow self-directed, message-oriented service interactions, which allow different parts of the ESB to operate independently of one another, without relying on a centralized routing engine.
|Charlesy 07/03/13 07:28:00 AM EDT|
An old post, but worth a small correction. Comparison with competitor products is always dangerous. You need to be very sure of your territory, and unfortunately, although David's description of BizTalk Server has some validity, it isn't 100% accurate. For example, the transformation services absolutely can be invoked separately to the rest of BTS via lightweight services, and load balanced across different boxes. More recent versions of BTS provide pre-built generic WCF and ASMX transformation web services as a courtesy to developers (reduces the need to build custom transformation services). You can use Windows Server AppFabric tools to create BizTalk maps in non-BizTalk projects - e.g., projects that define lightweight transformation services.
David doesn't clearly spell out what he means by 'cost...of the entire BizTalk Server'. If he means licencing cost, then he hits the mark, somewhat. BTS is certainly licensed in a fashion that encourages distribution and load balancing over a small farm of centralized servers, in contrast to the highly distributed approach advocated by Sonic. You can only invoke BTS transformation services on licensed boxes, so from that perspective, he is correct. However, invoking the transformation services does not require loading additional irrelevant BTS plumbing into memory. There is no heavy-weight performance cost imposed by code bloat, or anything like that!
BizTalk maps are emitted as code components containing an executable XSLT resource. You can distribute maps as freely as you wish and invoke the tranforms via code. Obviously, direct invocation in this case assumes the use of either .NET or Mono, although Java/.NET bridges could be used. If you use BizTalk Server's mapping tools to create maps, you may end up with a dependency on BizTalk-specific scripted components invoked in the XSLT, which ties you to licensed BizTalk Boxes. However, it is pretty easy to avoid this if you wish.
One thing Sonic has which BizTalk really does not is dynamic management of code deployment into the run-time environment. BizTalk Server has an built-in code repository, but this is simply a mechanism for managing and storing compiled artifacts and resources for the purpose of exporting installation packages. You still have to manually install those packages or deploy them via additional script. Frankly, though, this is rarely a significant drawback. The types of solution built using BizTalk Server tend to warrant close attention to managing dynamic deployment across a distributed environment using other frameworks and tools, and BizTalk Server plays well with the relevant frameworks. There is even a community-built deployment framework specifically designed for BizTalk Server.
|yt67 03/03/05 07:09:34 AM EST|
Myth-busting: always entertaining.
|Jason 03/02/05 09:16:29 AM EST|
A good read!
|Javier Camara 02/10/05 04:19:02 AM EST|
(This same feedback also posted to another WSJ article about ESBs)
I agree in that the ESB concept is over-hyped. For me, a SOA makes sense if it is viewed as a constellation of web services interacting among them. For this, something like a UDDI server is required for each service locating each other.
For me, all this (i.e. services + directory) is just enough if only synchronous communications are used. If asynchronous communications are needed, then you need also publish/subscribe and store-and-forward, i.e. roughly what a MOM does. You can call it an ESB if you want, although I think this concept in the market encompasses several roles:
An interesting thing to note is to implement points 1. and 2. you do *not* need business logic, while to implement 3. and 4. you do.
As I said, I see roles 1 and 2 required in SOAs with asynchronous interactions.
Roles 3 and 4 are also needed in many cases, mainly for integrating disparate systems. However, my main point against an ESB is that, in order to perform these roles, you do *NOT* need of a new, special concept like the ESB. *Any* service in the constellation of services can perform both routing and transformation. It can range from being a single component like an ESB (which I think is a bad idea), or it can just be a set of services (e.g. a different service performing specific adaptation for a system being integrated).
For me, using a single ESB for 3. and 4. breaks the beauty of the SOA idea. You are supposed to made all your data and business logic of your organization available as services in order to be reused, and suddenly you put on top an ESB in which you put *more* business logic (routing and transformation). So my point is that this should be implemented just by means of regular services, and not by specific, central-piece new components called ESBs.
Now, if for implementing routing and transformation you want to use Tibco, WebSphere or whatever, fine - however, the logic created by these products should be at the same level as the other services in the SOA, and not above.
So I am not saying that orchestrating tools are not useful. They are. Only, they are not *imprescindible*; and at any rate they should be viewed just as more services in the SOA. However, this does not fit the marketing strategy of ESB vendors which show its ESB as an *enabler* of a SOA, instead of just one more *component* of it.
|Dave Chappell 02/03/05 09:54:43 PM EST|
We (Sonic Software) didn't re-lable our product to support the ESB wave, we actually invented the concept. We then worked with the analyst and journalist community to help create industry awareness of the new concepts that are introduced by ESB, which has resulted in a whole new product category.
I would agree with you that there is a great deal of hype right now due to lack of understanding of what ESB is, which is compounded by the number of traditional middleware and EAI vendors who have clamoured to get ESB in their marketing literature without having a full understanding of what it means to have an ESB. Your comment about middleware with new clothes is well taken. You might get that impression depending on where you learned about what an ESB is. That is exactly what I am trying to point out with myth #1 in this article.
A certain amount of hype is expected when a technology category begins to take hold and gain traction within serious IT projects. This can be disruptive to the industry as a whole. This is also the primary reason why I wrote the OReilly book on the subject of ESB--to act as the definitive guide to help educate and provide clarity on what makes up an ESB. Please don't shoot the concept of ESB down until you have had a chance to understand it.
|Larry 02/03/05 04:15:19 AM EST|
Not surprising that the representative of a company who over-hyped ESB in the first place, and relabeled their own product ESB to catch the service wave, should now try to claim that anyone who saw through the hype is guilty of spreading myths.
A company’s collection of online systems is like a delicate ecosystem – all components must integrate with and complement each other, and one single malfunction in any of them can bring the entire system to a screeching halt. That’s why, when monitoring and analyzing the health of your online systems, you need a broad arsenal of different tools for your different needs. In addition to a wide-angle lens that provides a snapshot of the overall health of your system, you must also have precise, ...
Aug. 25, 2016 06:45 PM EDT Reads: 1,529
Using new techniques of information modeling, indexing, and processing, new cloud-based systems can support cloud-based workloads previously not possible for high-throughput insurance, banking, and case-based applications. In his session at 18th Cloud Expo, John Newton, CTO, Founder and Chairman of Alfresco, described how to scale cloud-based content management repositories to store, manage, and retrieve billions of documents and related information with fast and linear scalability. He addres...
Aug. 25, 2016 06:30 PM EDT Reads: 1,914
The following fictional case study is a composite of actual horror stories I’ve heard over the years. Unfortunately, this scenario often occurs when in-house integration teams take on the complexities of DevOps and ALM integration with an enterprise service bus (ESB) or custom integration. It is written from the perspective of an enterprise architect tasked with leading an organization’s effort to adopt Agile to become more competitive. The company has turned to Scaled Agile Framework (SAFe) as ...
Aug. 25, 2016 05:00 PM EDT Reads: 590
Monitoring of Docker environments is challenging. Why? Because each container typically runs a single process, has its own environment, utilizes virtual networks, or has various methods of managing storage. Traditional monitoring solutions take metrics from each server and applications they run. These servers and applications running on them are typically very static, with very long uptimes. Docker deployments are different: a set of containers may run many applications, all sharing the resource...
Aug. 25, 2016 04:00 PM EDT Reads: 1,921
DevOps at Cloud Expo, taking place Nov 1-3, 2016, at the Santa Clara Convention Center in Santa Clara, CA, is co-located with 19th Cloud Expo and will feature technical sessions from a rock star conference faculty and the leading industry players in the world. The widespread success of cloud computing is driving the DevOps revolution in enterprise IT. Now as never before, development teams must communicate and collaborate in a dynamic, 24/7/365 environment. There is no time to wait for long dev...
Aug. 25, 2016 03:15 PM EDT Reads: 2,212
Internet of @ThingsExpo, taking place November 1-3, 2016, at the Santa Clara Convention Center in Santa Clara, CA, is co-located with 19th Cloud Expo and will feature technical sessions from a rock star conference faculty and the leading industry players in the world. The Internet of Things (IoT) is the most profound change in personal and enterprise IT since the creation of the Worldwide Web more than 20 years ago. All major researchers estimate there will be tens of billions devices - comp...
Aug. 25, 2016 02:00 PM EDT Reads: 3,546
It's been a busy time for tech's ongoing infatuation with containers. Amazon just announced EC2 Container Registry to simply container management. The new Azure container service taps into Microsoft's partnership with Docker and Mesosphere. You know when there's a standard for containers on the table there's money on the table, too. Everyone is talking containers because they reduce a ton of development-related challenges and make it much easier to move across production and testing environm...
Aug. 25, 2016 01:30 PM EDT Reads: 4,951
SYS-CON Events announced today that Venafi, the Immune System for the Internet™ and the leading provider of Next Generation Trust Protection, will exhibit at @DevOpsSummit at 19th International Cloud Expo, which will take place on November 1–3, 2016, at the Santa Clara Convention Center in Santa Clara, CA. Venafi is the Immune System for the Internet™ that protects the foundation of all cybersecurity – cryptographic keys and digital certificates – so they can’t be misused by bad guys in attacks...
Aug. 25, 2016 01:00 PM EDT Reads: 2,596
Cloud Expo 2016 New York at the Javits Center New York was characterized by increased attendance and a new focus on operations. These were both encouraging signs for all involved in Cloud Computing and all that it touches. As Conference Chair, I work with the Cloud Expo team to structure three keynotes, numerous general sessions, and more than 150 breakout sessions along 10 tracks. Our job is to balance the state of enterprise IT today with the trends that will be commonplace tomorrow. Mobile...
Aug. 25, 2016 01:00 PM EDT Reads: 3,264
The 19th International Cloud Expo has announced that its Call for Papers is open. Cloud Expo, to be held November 1-3, 2016, at the Santa Clara Convention Center in Santa Clara, CA, brings together Cloud Computing, Big Data, Internet of Things, DevOps, Digital Transformation, Microservices and WebRTC to one location. With cloud computing driving a higher percentage of enterprise IT budgets every year, it becomes increasingly important to plant your flag in this fast-expanding business opportuni...
Aug. 25, 2016 01:00 PM EDT Reads: 3,891
[session] Architecting for the Cloud By @RagsS | @CloudExpo @IBMBluemix #Cloud #Docker #Microservices
As the world moves toward more DevOps and Microservices, application deployment to the cloud ought to become a lot simpler. The Microservices architecture, which is the basis of many new age distributed systems such as OpenStack, NetFlix and so on, is at the heart of Cloud Foundry - a complete developer-oriented Platform as a Service (PaaS) that is IaaS agnostic and supports vCloud, OpenStack and AWS. Serverless computing is revolutionizing computing. In his session at 19th Cloud Expo, Raghav...
Aug. 25, 2016 12:45 PM EDT Reads: 587
DevOps at Cloud Expo – being held November 1-3, 2016, at the Santa Clara Convention Center in Santa Clara, CA – announces that its Call for Papers is open. Born out of proven success in agile development, cloud computing, and process automation, DevOps is a macro trend you cannot afford to miss. From showcase success stories from early adopters and web-scale businesses, DevOps is expanding to organizations of all sizes, including the world's largest enterprises – and delivering real results. Am...
Aug. 25, 2016 12:15 PM EDT Reads: 3,387
Modern organizations face great challenges as they embrace innovation and integrate new tools and services. They begin to mature and move away from the complacency of maintaining traditional technologies and systems that only solve individual, siloed problems and work “well enough.” In order to build...
SYS-CON Events announced today that eCube Systems, a leading provider of middleware modernization, integration, and management solutions, will exhibit at @DevOpsSummit at 19th International Cloud Expo, which will take place on November 1–3, 2016, at the Santa Clara Convention Center in Santa Clara, CA. eCube Systems offers a family of middleware evolution products and services that maximize return on technology investment by leveraging existing technical equity to meet evolving business needs. ...
Aug. 25, 2016 09:00 AM EDT Reads: 510
To leverage Continuous Delivery, enterprises must consider impacts that span functional silos, as well as applications that touch older, slower moving components. Managing the many dependencies can cause slowdowns. See how to achieve continuous delivery in the enterprise.
Aug. 25, 2016 08:15 AM EDT Reads: 1,540
Akana has announced the availability of version 8 of its API Management solution. The Akana Platform provides an end-to-end API Management solution for designing, implementing, securing, managing, monitoring, and publishing APIs. It is available as a SaaS platform, on-premises, and as a hybrid deployment. Version 8 introduces a lot of new functionality, all aimed at offering customers the richest API Management capabilities in a way that is easier than ever for API and app developers to use.
Aug. 25, 2016 06:00 AM EDT Reads: 1,440
SYS-CON Events announced today that Isomorphic Software will exhibit at DevOps Summit at 19th International Cloud Expo, which will take place on November 1–3, 2016, at the Santa Clara Convention Center in Santa Clara, CA. Isomorphic Software provides the SmartClient HTML5/AJAX platform, the most advanced technology for building rich, cutting-edge enterprise web applications for desktop and mobile. SmartClient combines the productivity and performance of traditional desktop software with the simp...
Aug. 25, 2016 03:30 AM EDT Reads: 2,160
SYS-CON Events announced today that 910Telecom will exhibit at the 19th International Cloud Expo, which will take place on November 1–3, 2016, at the Santa Clara Convention Center in Santa Clara, CA. Housed in the classic Denver Gas & Electric Building, 910 15th St., 910Telecom is a carrier-neutral telecom hotel located in the heart of Denver. Adjacent to CenturyLink, AT&T, and Denver Main, 910Telecom offers connectivity to all major carriers, Internet service providers, Internet backbones and ...
Aug. 25, 2016 02:00 AM EDT Reads: 1,812
The burgeoning trends around DevOps are translating into new types of IT infrastructure that both developers and operators can take advantage of. The next BriefingsDirect Voice of the Customer thought leadership discussion focuses on the burgeoning trends around DevOps and how that’s translating into new types of IT infrastructure that both developers and operators can take advantage of.
Aug. 25, 2016 02:00 AM EDT Reads: 2,434
With so much going on in this space you could be forgiven for thinking you were always working with yesterday’s technologies. So much change, so quickly. What do you do if you have to build a solution from the ground up that is expected to live in the field for at least 5-10 years? This is the challenge we faced when we looked to refresh our existing 10-year-old custom hardware stack to measure the fullness of trash cans and compactors.
Aug. 25, 2016 01:15 AM EDT Reads: 1,644