|By Dave Chappell||
|May 25, 2005 07:30 PM EDT||
Since releasing my latest book, Enterprise Service Bus (O'Reilly Media, 2004), I have been doing a fair amount of visiting corporations, conducting seminars, and generally discussing with enterprise architects the subject of enterprise service-oriented architecture (SOA) and how an enterprise service bus (ESB) backbone can be leveraged to provide a framework for an enterprise SOA. Along the way, I have been asked many questions about the nature of an ESB. I have also fended off some misconceptions that have been growing in the general IT population regarding what an ESB is and when, where, and how it can be used. I have gathered together the most popular questions and misconceptions, and offer some clarity in the form of a "top ten" list.
Myth #1. ESB is just a new name for EAI.
While many IT architecture groups are focusing on building SOAs, they still inevitably beg the question of "how is ESB different from EAI?" An ESB is an infrastructure for building an enterprise SOA, and is capable of being used in a more general way than a conventional EAI broker. According to Forrester Research, an ESB helps enterprises obtain the value of SOA by increasing connectivity, adding flexibility that speeds change, and providing greater control over use of the important resources that it binds.
An ESB can be used to handle integration projects that have traditionally been relegated to EAI tools. However, an ESB can also be used for establishing B2B relationships across companies.
An ESB provides EAI capabilities, but is based on a fundamentally different architecture that is providing the basis of an industry transition from traditional integration to coordinated service interaction. EAI brokers are historically implemented as a monolithic stack, using centralized hub-and-spoke architecture.
An ESB provides the same base functionality as an EAI broker - connectivity, application adapters, routing of messages based on rules, and data transformation engine - yet, in an ESB, these capabilities are themselves SOA based in that they are spread out across the bus in a highly distributed fashion and hosted in separately deployable service containers. This allows the selective deployment of integration broker functionality exactly where you need it, with no additional over-bloating where it's not needed. The distributed nature of the ESB container model allows the independent scalability of integration components, which are plugged into your SOA as event-driven services on an as needed basis.
In order for an integration broker to be truly capable of supporting an SOA, and to be considered a true ESB, it would need to have its base functions broken up into their constituent parts, which would then be capable of being separately deployed across the bus while working together in harmony as necessary.
Let's use an example of an XSLT-based transformation engine that accepts an incoming XML document and applies an XSLT style sheet to it in order to produce an outgoing document in another XML format. I can tell you that there is nothing that can chew up computing resources more than the parsing and manipulation of XML. If this particular XSLT transformation sits between two popular applications that communicate regularly with each other, then that individual transformation can become a performance and scalability bottleneck. If you are using a monolithic hub-and-spoke integration broker approach, in order to remove the bottleneck and scale up the deployment you would need to either install that integration broker on one big powerful machine, or install the integration broker across multiple machines - just to support that one transform scenario! All the while, the other integration broker capabilities, such as the execution of routing rules, are competing for the same computing resources as the transformation operation.
In contrast to the monolithic hub-and-spoke architecture of an integration broker, the foundational core of an ESB provides a distributed services architecture. This architecture is built for integration and has the ability for integration broker functionality, such as message routing, data transformation, and application adapters to be selectively deployed on an as-needed basis.These are separate integration services that are a natural part of an SOA processing pipeline across the bus.
An individual XSLT transformation can be deployed as a service in its own ESB service container, and multiple instances of that container can be load-balanced across many machines. If the ESB container implementation is cross-platform, then you can be flexible as to what kinds of machines you spread the transform service across - Linux boxes, Solaris boxes, Windows boxes, and so on. And for those of you who don't find solace in the architectural purity of this discussion, consider this: the ESB vendors who are leading the charge in defining and delivering ESB products are also putting forth a license model where there is no additional cost for deploying as many of these lightweight ESB service containers as necessary to get the job done.
The integration services provided by the ESB can be combined with other services into SOA-based processing pipelines that can span business boundaries. The distributed services in an ESB can be combined with itinerary-based routing (see Myth #7) to allow self-directed, message-oriented service interactions, which allow different parts of the ESB to operate independently of one another, without relying on a centralized routing engine.
|Charlesy 07/03/13 07:28:00 AM EDT|
An old post, but worth a small correction. Comparison with competitor products is always dangerous. You need to be very sure of your territory, and unfortunately, although David's description of BizTalk Server has some validity, it isn't 100% accurate. For example, the transformation services absolutely can be invoked separately to the rest of BTS via lightweight services, and load balanced across different boxes. More recent versions of BTS provide pre-built generic WCF and ASMX transformation web services as a courtesy to developers (reduces the need to build custom transformation services). You can use Windows Server AppFabric tools to create BizTalk maps in non-BizTalk projects - e.g., projects that define lightweight transformation services.
David doesn't clearly spell out what he means by 'cost...of the entire BizTalk Server'. If he means licencing cost, then he hits the mark, somewhat. BTS is certainly licensed in a fashion that encourages distribution and load balancing over a small farm of centralized servers, in contrast to the highly distributed approach advocated by Sonic. You can only invoke BTS transformation services on licensed boxes, so from that perspective, he is correct. However, invoking the transformation services does not require loading additional irrelevant BTS plumbing into memory. There is no heavy-weight performance cost imposed by code bloat, or anything like that!
BizTalk maps are emitted as code components containing an executable XSLT resource. You can distribute maps as freely as you wish and invoke the tranforms via code. Obviously, direct invocation in this case assumes the use of either .NET or Mono, although Java/.NET bridges could be used. If you use BizTalk Server's mapping tools to create maps, you may end up with a dependency on BizTalk-specific scripted components invoked in the XSLT, which ties you to licensed BizTalk Boxes. However, it is pretty easy to avoid this if you wish.
One thing Sonic has which BizTalk really does not is dynamic management of code deployment into the run-time environment. BizTalk Server has an built-in code repository, but this is simply a mechanism for managing and storing compiled artifacts and resources for the purpose of exporting installation packages. You still have to manually install those packages or deploy them via additional script. Frankly, though, this is rarely a significant drawback. The types of solution built using BizTalk Server tend to warrant close attention to managing dynamic deployment across a distributed environment using other frameworks and tools, and BizTalk Server plays well with the relevant frameworks. There is even a community-built deployment framework specifically designed for BizTalk Server.
|yt67 03/03/05 07:09:34 AM EST|
Myth-busting: always entertaining.
|Jason 03/02/05 09:16:29 AM EST|
A good read!
|Javier Camara 02/10/05 04:19:02 AM EST|
(This same feedback also posted to another WSJ article about ESBs)
I agree in that the ESB concept is over-hyped. For me, a SOA makes sense if it is viewed as a constellation of web services interacting among them. For this, something like a UDDI server is required for each service locating each other.
For me, all this (i.e. services + directory) is just enough if only synchronous communications are used. If asynchronous communications are needed, then you need also publish/subscribe and store-and-forward, i.e. roughly what a MOM does. You can call it an ESB if you want, although I think this concept in the market encompasses several roles:
An interesting thing to note is to implement points 1. and 2. you do *not* need business logic, while to implement 3. and 4. you do.
As I said, I see roles 1 and 2 required in SOAs with asynchronous interactions.
Roles 3 and 4 are also needed in many cases, mainly for integrating disparate systems. However, my main point against an ESB is that, in order to perform these roles, you do *NOT* need of a new, special concept like the ESB. *Any* service in the constellation of services can perform both routing and transformation. It can range from being a single component like an ESB (which I think is a bad idea), or it can just be a set of services (e.g. a different service performing specific adaptation for a system being integrated).
For me, using a single ESB for 3. and 4. breaks the beauty of the SOA idea. You are supposed to made all your data and business logic of your organization available as services in order to be reused, and suddenly you put on top an ESB in which you put *more* business logic (routing and transformation). So my point is that this should be implemented just by means of regular services, and not by specific, central-piece new components called ESBs.
Now, if for implementing routing and transformation you want to use Tibco, WebSphere or whatever, fine - however, the logic created by these products should be at the same level as the other services in the SOA, and not above.
So I am not saying that orchestrating tools are not useful. They are. Only, they are not *imprescindible*; and at any rate they should be viewed just as more services in the SOA. However, this does not fit the marketing strategy of ESB vendors which show its ESB as an *enabler* of a SOA, instead of just one more *component* of it.
|Dave Chappell 02/03/05 09:54:43 PM EST|
We (Sonic Software) didn't re-lable our product to support the ESB wave, we actually invented the concept. We then worked with the analyst and journalist community to help create industry awareness of the new concepts that are introduced by ESB, which has resulted in a whole new product category.
I would agree with you that there is a great deal of hype right now due to lack of understanding of what ESB is, which is compounded by the number of traditional middleware and EAI vendors who have clamoured to get ESB in their marketing literature without having a full understanding of what it means to have an ESB. Your comment about middleware with new clothes is well taken. You might get that impression depending on where you learned about what an ESB is. That is exactly what I am trying to point out with myth #1 in this article.
A certain amount of hype is expected when a technology category begins to take hold and gain traction within serious IT projects. This can be disruptive to the industry as a whole. This is also the primary reason why I wrote the OReilly book on the subject of ESB--to act as the definitive guide to help educate and provide clarity on what makes up an ESB. Please don't shoot the concept of ESB down until you have had a chance to understand it.
|Larry 02/03/05 04:15:19 AM EST|
Not surprising that the representative of a company who over-hyped ESB in the first place, and relabeled their own product ESB to catch the service wave, should now try to claim that anyone who saw through the hype is guilty of spreading myths.
With major technology companies and startups seriously embracing IoT strategies, now is the perfect time to attend @ThingsExpo 2016 in New York and Silicon Valley. Learn what is going on, contribute to the discussions, and ensure that your enterprise is as "IoT-Ready" as it can be! Internet of @ThingsExpo, taking place Nov 3-5, 2015, at the Santa Clara Convention Center in Santa Clara, CA, is co-located with 17th Cloud Expo and will feature technical sessions from a rock star conference faculty ...
Nov. 25, 2015 02:45 PM EST Reads: 487
Internet of @ThingsExpo, taking place June 7-9, 2016 at Javits Center, New York City and Nov 1-3, 2016, at the Santa Clara Convention Center in Santa Clara, CA, is co-located with the 18th International @CloudExpo and will feature technical sessions from a rock star conference faculty and the leading industry players in the world and ThingsExpo New York Call for Papers is now open.
Nov. 25, 2015 02:30 PM EST Reads: 495
Growth hacking is common for startups to make unheard-of progress in building their business. Career Hacks can help Geek Girls and those who support them (yes, that's you too, Dad!) to excel in this typically male-dominated world. Get ready to learn the facts: Is there a bias against women in the tech / developer communities? Why are women 50% of the workforce, but hold only 24% of the STEM or IT positions? Some beginnings of what to do about it! In her Day 2 Keynote at 17th Cloud Expo, San...
Nov. 25, 2015 02:00 PM EST Reads: 501
In his General Session at DevOps Summit, Asaf Yigal, Co-Founder & VP of Product at Logz.io, explored the value of Kibana 4 for log analysis and provided a hands-on tutorial on how to set up Kibana 4 and get the most out of Apache log files. He examined three use cases: IT operations, business intelligence, and security and compliance. Asaf Yigal is co-founder and VP of Product at log analytics software company Logz.io. In the past, he was co-founder of social-trading platform Currensee, which...
Nov. 25, 2015 12:45 PM EST Reads: 107
There are over 120 breakout sessions in all, with Keynotes, General Sessions, and Power Panels adding to three days of incredibly rich presentations and content. Join @ThingsExpo conference chair Roger Strukhoff (@IoT2040), June 7-9, 2016 in New York City, for three days of intense 'Internet of Things' discussion and focus, including Big Data's indespensable role in IoT, Smart Grids and Industrial Internet of Things, Wearables and Consumer IoT, as well as (new) IoT's use in Vertical Markets.
Nov. 25, 2015 12:00 PM EST Reads: 515
The Internet of Things (IoT) is growing rapidly by extending current technologies, products and networks. By 2020, Cisco estimates there will be 50 billion connected devices. Gartner has forecast revenues of over $300 billion, just to IoT suppliers. Now is the time to figure out how you’ll make money – not just create innovative products. With hundreds of new products and companies jumping into the IoT fray every month, there’s no shortage of innovation. Despite this, McKinsey/VisionMobile data...
Nov. 25, 2015 10:00 AM EST Reads: 425
One of the most important tenets of digital transformation is that it’s customer-driven. In fact, the only reason technology is involved at all is because today’s customers demand technology-based interactions with the companies they do business with. It’s no surprise, therefore, that we at Intellyx agree with Patrick Maes, CTO, ANZ Bank, when he said, “the fundamental element in digital transformation is extreme customer centricity.” So true – but note the insightful twist that Maes adde...
Nov. 25, 2015 09:00 AM EST Reads: 394
Just over a week ago I received a long and loud sustained applause for a presentation I delivered at this year’s Cloud Expo in Santa Clara. I was extremely pleased with the turnout and had some very good conversations with many of the attendees. Over the next few days I had many more meaningful conversations and was not only happy with the results but also learned a few new things. Here is everything I learned in those three days distilled into three short points.
Nov. 25, 2015 09:00 AM EST Reads: 264
Nov. 25, 2015 08:30 AM EST Reads: 142
As organizations realize the scope of the Internet of Things, gaining key insights from Big Data, through the use of advanced analytics, becomes crucial. However, IoT also creates the need for petabyte scale storage of data from millions of devices. A new type of Storage is required which seamlessly integrates robust data analytics with massive scale. These storage systems will act as “smart systems” provide in-place analytics that speed discovery and enable businesses to quickly derive meaningf...
Nov. 25, 2015 08:15 AM EST Reads: 346
DevOps is about increasing efficiency, but nothing is more inefficient than building the same application twice. However, this is a routine occurrence with enterprise applications that need both a rich desktop web interface and strong mobile support. With recent technological advances from Isomorphic Software and others, rich desktop and tuned mobile experiences can now be created with a single codebase – without compromising functionality, performance or usability. In his session at DevOps Su...
Nov. 25, 2015 07:45 AM EST Reads: 341
You may have heard about the pets vs. cattle discussion – a reference to the way application servers are deployed in the cloud native world. If an application server goes down it can simply be dropped from the mix and a new server added in its place. The practice so far has mostly been applied to application deployments. Management software on the other hand is treated in a very special manner. Dedicated resources are set aside to run the management software components and several alerting syst...
Nov. 25, 2015 07:30 AM EST Reads: 128
It's been a busy time for tech's ongoing infatuation with containers. Amazon just announced EC2 Container Registry to simply container management. The new Azure container service taps into Microsoft's partnership with Docker and Mesosphere. You know when there's a standard for containers on the table there's money on the table, too. Everyone is talking containers because they reduce a ton of development-related challenges and make it much easier to move across production and testing environm...
Nov. 25, 2015 04:00 AM EST Reads: 550
People want to get going with DevOps or Continuous Delivery, but need a place to start. Others are already on their way, but need some validation of their choices. A few months ago, I published the first volume of DevOps and Continuous Delivery reference architectures which has now been viewed over 50,000 times on SlideShare (it's free to download...no registration required). Three things helped people in the deck: (1) the reference architectures, (2) links to the sources for each architectur...
Nov. 25, 2015 02:30 AM EST Reads: 197
Continuous processes around the development and deployment of applications are both impacted by -- and a benefit to -- the Internet of Things trend. To help better understand the relationship between DevOps and a plethora of new end-devices and data please welcome Gary Gruver, consultant, author and a former IT executive who has led many large-scale IT transformation projects, and John Jeremiah, Technology Evangelist at Hewlett Packard Enterprise (HPE), on Twitter at @j_jeremiah. The discussion...
Nov. 25, 2015 02:30 AM EST Reads: 681
Hiring the wrong candidate can cost a company hundreds of thousands of dollars, and result in lost profit and productivity during the search for a replacement. In fact, the Harvard Business Review has found that as much as 80 percent of turnover is caused by bad hiring decisions. But when your organization has implemented DevOps, the job is about more than just technical chops. It’s also about core behaviors: how they work with others, how they make decisions, and how those decisions translate t...
Nov. 25, 2015 12:45 AM EST Reads: 138
In today's enterprise, digital transformation represents organizational change even more so than technology change, as customer preferences and behavior drive end-to-end transformation across lines of business as well as IT. To capitalize on the ubiquitous disruption driving this transformation, companies must be able to innovate at an increasingly rapid pace. Traditional approaches for driving innovation are now woefully inadequate for keeping up with the breadth of disruption and change facin...
Nov. 25, 2015 12:30 AM EST Reads: 417
The Internet of Things is clearly many things: data collection and analytics, wearables, Smart Grids and Smart Cities, the Industrial Internet, and more. Cool platforms like Arduino, Raspberry Pi, Intel's Galileo and Edison, and a diverse world of sensors are making the IoT a great toy box for developers in all these areas. In this Power Panel at @ThingsExpo, moderated by Conference Chair Roger Strukhoff, panelists discussed what things are the most important, which will have the most profound...
Nov. 25, 2015 12:30 AM EST Reads: 411
PubNub has announced the release of BLOCKS, a set of customizable microservices that give developers a simple way to add code and deploy features for realtime apps.PubNub BLOCKS executes business logic directly on the data streaming through PubNub’s network without splitting it off to an intermediary server controlled by the customer. This revolutionary approach streamlines app development, reduces endpoint-to-endpoint latency, and allows apps to better leverage the enormous scalability of PubNu...
Nov. 24, 2015 10:00 PM EST Reads: 261
I recently attended and was a speaker at the 4th International Internet of @ThingsExpo at the Santa Clara Convention Center. I also had the opportunity to attend this event last year and I wrote a blog from that show talking about how the “Enterprise Impact of IoT” was a key theme of last year’s show. I was curious to see if the same theme would still resonate 365 days later and what, if any, changes I would see in the content presented.
Nov. 24, 2015 08:00 PM EST Reads: 343