Microservices Expo Authors: Liz McMillan, Pat Romanski, Carmen Gonzalez, Elizabeth White, Jason Bloomberg

Related Topics: Microservices Expo

Microservices Expo: Article

Understanding the Real Costs of Integration

What are the true costs of integration, and how does SOA seek to reduce those costs?

These days, it seems that everyone is pitching the merits of using SOA to better integrate systems and businesses. "Drastically reduce integration costs, increase time-to-market, and improve business flexibility using SOA!" claim many Web Services (cum-SOA) integration vendors. How much of these promises are hype, and how much is reality? In order to understand what impact Web Services will have on integration, it is important to understand its true costs, which means comparing different integration approaches and finding out whether SOA, rather than simply Web Services, can offer a positive return on investment. With this knowledge businesses can decide whether their SOA implementations will truly save them money, or conversely, their Web Services integrations add to the chaotic proliferation of spaghetti code in the enterprise.

The Four Phases of Integration

The costs of a typical integration project go through four distinct phases: the initial setup costs, the cost of configuring and customizing the integration project, ongoing maintenance costs, and costs involved when any of the elements of the integration project change. ZapThink compares four approaches to integration: custom integration, traditional EAI and B2Bi approaches, Web Services "adapters" for integration, and loosely coupled Service-Oriented Integration, as shown in the chart below:


First, let's take a look at the solid black curve, which represents the cost of custom integration. Custom integration involves dedicating current IT resources to the task of achieving some application goal that requires integrated results from multiple systems. Such custom integration involves the smallest up-front cost of the four integration approaches because the skills and tools necessary to complete the integration task are typically already in house. However, as the project progresses, it consumes an increasing amount of developer time, in proportion to the complexity of the integration task at hand. During the maintenance phase, then, things start to get costly. Traditionally, companies spend most of their time and money maintaining existing integration projects and then dealing with the changes that occur in those systems as business requirements and technology change. With custom integration, the costs for both maintaining and changing systems can become exorbitant since developers must recode all applications that are impacted by changes.

The second approach to integration lumps together traditional EAI (Enterprise Application Integration) and B2Bi (business-to-business integration) approaches, as shown by the dashed green line in the graph above. Traditional EAI and B2Bi solutions have sought to solve many integration headaches by presenting an architecture that efficiently manages and maintains connections among systems and between enterprises. The primary downside to EAI is that up-front costs are much higher than custom integration, as shown in the first column of the graph. In a typical EAI solution, end-users must spend from tens of thousands to millions of dollars on software licenses and server systems prior to completing any integration. The actual integration project itself costs many times more than the initial costs and can easily dwarf the costs of custom integration projects.

However, EAI and B2Bi achieve their the real win during the maintenance phase of an integration project. As long as the endpoints, business processes, or for that matter any major business assumptions, don't change, then the costs of running an EAI solution over the long haul can be substantially lower than that of custom integration projects. However, the hidden cost of EAI is when things do change. When systems, business processes, or major assumptions change, EAI system costs can spike, as shown in the fourth column of the graph. In fact, we say that EAI systems "pour concrete on business processes," since they tend to solidify existing processes rather than enable an IT environment that allows companies to deal easily with change.

Enter Web Services. By enabling standards-based approaches to integration, many people believe that Web Services will turn the EAI/B2Bi cost curve on its head. However, ZapThink believes that blindly applying Web Services to integration is a mistake. Just because Web Services promise dramatically reduced integration costs, that doesn't mean that any arbitrary use of Web Services will achieve this effect. After all, just because a new technology has promise doesn't guarantee that it will be applied correctly.

In particular, we are talking about vendors that provide SOAP interfaces to various software components with Web Services "wrappers." The critical issue is the difference between using Web Services in a loosely-coupled, Service-oriented manner, as opposed to misusing Web Services in a tightly-coupled, point-to-point manner. Vendors that pitch their wares as "SOAP wrappers" or "Web Services adapters" offer a dramatically short-sighted view of integration that seeks to lower only the initial costs without solving any of the fundamental problems of the traditional EAI/B2Bi approaches. In fact, the blue dashed line above shows that only initial and configuration costs are saved while long term costs of change mirror those of traditional EAI/B2Bi costs.

While it is true that the use of standards-based technology lowers initial and perhaps even configuration costs, these tightly-coupled Web Services adapters are just as brittle as the EAI adapters of old. Instead of having to reconfigure 300 DCOM or CORBA calls when a system changes, integrators will have to reprogram 300 SOAP calls. All they have done is play a shell game, moving the pea from proprietary to standards-based adapters. This approach doesn't do squat to loosely couple the systems or change their level of granularity. In fact, all that is accomplished is that developers now pour standards-based concrete over existing business processes -- wrapping EJBs, COM, or CORBA in SOAP is simply not going to get us anywhere. In fact, the Web Services wrapper approach may actually increase the long-term cost of integration, because of the inefficiencies inherent in XML when compared to binary message formats, as the blue dashed line on the graph shows.

In stark contrast to the "Web Services wrapper" straw man is the loosely coupled Service-Oriented Architecture (SOA) approach to integration, which we call Service-Oriented Integration (SOI). While tightly coupled Services have been around for a while -- both CORBA and DCOM have elements of service-oriented architecture -- SOA approaches allow architects to build loosely coupled Services that expose business functionality at varying levels of granularity. The real costs in building and integrating such Services is in the system design. In an SOI approach, it is not sufficient to simply wrap an existing API with a Web Services interface. It is too easy (and almost lazy) to create a one-to-one mapping between system APIs and Web Services interfaces. Rather, businesses must spend time analyzing their business processes and creating business Services at varying levels of granularity, perhaps even requiring the orchestration and choreography of multiple layers of Web Services to accomplish a single task. This support for multiple levels of granularity enables the system to support frequent changes in the underlying systems, as well as changes to business processes and underlying business assumptions, without the need to make interface changes that break the loose coupling of the Services. The real win with SOI, therefore, is in the dramatic reduction of cost at the maintenance and change phases of integration, as shown by the dotted red line in the graph.

The ZapThink Take

ZapThink encourages both vendors and adopters of SOA to take a close look at the mistakes of the past. What are the true costs of integration, and how does SOA, rather than simply Web Services, seek to reduce those costs? Are implementations of Web Services simply "old wine in new bottles," with interfaces every bit as brittle and tightly-coupled as in the past, or are they really implementing Service-Oriented Integration among Services at many levels of granularity? Clearly, "Web Services Integration" does not equal "Service-Oriented Integration." We are not doomed to repeat the mistakes of the past, as long as we understand the appropriate use of the Web Services technologies behind Service-Oriented Integration.

You can learn more about Service-Oriented Integration and accelerate your SOA skills at one of ZapThink's LZA SOA training & certification boot camps: http://www.zapthink.com/eventreg.html.

More Stories By Ron Schmelzer

Ron Schmelzer is founder and senior analyst of ZapThink. A well-known expert in the field of XML and XML-based standards and initiatives, Ron has been featured in and written for periodicals and has spoken on the subject of XML at numerous industry conferences.

Microservices Articles
Modern software design has fundamentally changed how we manage applications, causing many to turn to containers as the new virtual machine for resource management. As container adoption grows beyond stateless applications to stateful workloads, the need for persistent storage is foundational - something customers routinely cite as a top pain point. In his session at @DevOpsSummit at 21st Cloud Expo, Bill Borsari, Head of Systems Engineering at Datera, explored how organizations can reap the bene...
"NetApp's vision is how we help organizations manage data - delivering the right data in the right place, in the right time, to the people who need it, and doing it agnostic to what the platform is," explained Josh Atwell, Developer Advocate for NetApp, in this SYS-CON.tv interview at 20th Cloud Expo, held June 6-8, 2017, at the Javits Center in New York City, NY.
The Jevons Paradox suggests that when technological advances increase efficiency of a resource, it results in an overall increase in consumption. Writing on the increased use of coal as a result of technological improvements, 19th-century economist William Stanley Jevons found that these improvements led to the development of new ways to utilize coal. In his session at 19th Cloud Expo, Mark Thiele, Chief Strategy Officer for Apcera, compared the Jevons Paradox to modern-day enterprise IT, examin...
In his session at 20th Cloud Expo, Mike Johnston, an infrastructure engineer at Supergiant.io, discussed how to use Kubernetes to set up a SaaS infrastructure for your business. Mike Johnston is an infrastructure engineer at Supergiant.io with over 12 years of experience designing, deploying, and maintaining server and workstation infrastructure at all scales. He has experience with brick and mortar data centers as well as cloud providers like Digital Ocean, Amazon Web Services, and Rackspace. H...
Skeuomorphism usually means retaining existing design cues in something new that doesn’t actually need them. However, the concept of skeuomorphism can be thought of as relating more broadly to applying existing patterns to new technologies that, in fact, cry out for new approaches. In his session at DevOps Summit, Gordon Haff, Senior Cloud Strategy Marketing and Evangelism Manager at Red Hat, will discuss why containers should be paired with new architectural practices such as microservices ra...
In his session at 20th Cloud Expo, Scott Davis, CTO of Embotics, discussed how automation can provide the dynamic management required to cost-effectively deliver microservices and container solutions at scale. He also discussed how flexible automation is the key to effectively bridging and seamlessly coordinating both IT and developer needs for component orchestration across disparate clouds – an increasingly important requirement at today’s multi-cloud enterprise.
The Software Defined Data Center (SDDC), which enables organizations to seamlessly run in a hybrid cloud model (public + private cloud), is here to stay. IDC estimates that the software-defined networking market will be valued at $3.7 billion by 2016. Security is a key component and benefit of the SDDC, and offers an opportunity to build security 'from the ground up' and weave it into the environment from day one. In his session at 16th Cloud Expo, Reuven Harrison, CTO and Co-Founder of Tufin, ...
DevOps is often described as a combination of technology and culture. Without both, DevOps isn't complete. However, applying the culture to outdated technology is a recipe for disaster; as response times grow and connections between teams are delayed by technology, the culture will die. A Nutanix Enterprise Cloud has many benefits that provide the needed base for a true DevOps paradigm. In their Day 3 Keynote at 20th Cloud Expo, Chris Brown, a Solutions Marketing Manager at Nutanix, and Mark Lav...
Many organizations are now looking to DevOps maturity models to gauge their DevOps adoption and compare their maturity to their peers. However, as enterprise organizations rush to adopt DevOps, moving past experimentation to embrace it at scale, they are in danger of falling into the trap that they have fallen into time and time again. Unfortunately, we've seen this movie before, and we know how it ends: badly.
TCP (Transmission Control Protocol) is a common and reliable transmission protocol on the Internet. TCP was introduced in the 70s by Stanford University for US Defense to establish connectivity between distributed systems to maintain a backup of defense information. At the time, TCP was introduced to communicate amongst a selected set of devices for a smaller dataset over shorter distances. As the Internet evolved, however, the number of applications and users, and the types of data accessed and...