It is also important during this phase to address areas not fully covered by EAI vendor products or methodologies such as configuration management, change control, organizational impact, security, system integration testing, and production system deployment. Addressing such requirements early in the architecture phase helps identify integration patterns and uncover opportunities to create reusable integration artifacts (business process models, integration entities (data), adapters, transformations, etc.). This is crucial for delivering on a fundamental value proposition of EAI - reuse - that is often not realized in practice.

EAI projects that minimize or ignore the architectural approach do so at their peril. One client launched multiple independent integration projects using a shared EAI infrastructure, with little coordination between the projects. In the short term, this strategy worked. However, around the time that the fifth or sixth project was deployed, the client's EAI IT infrastructure fell over! Upon investigation, they discovered that system resources were depleted, due in large part to each project having created redundant schema that consumed a large amount of memory (e.g., four almost-identical definitions for a purchase order). With proper architecture and project coordination, these could have been designed for shared use, dramatically reducing both the system load and the effort expended by each project.

Another EAI project left configuration management until very late in the project. Much to their chagrin, the development team discovered, just prior to system integration test, that it was very difficult to migrate code between development, test, and production environments in an automated manner using their EAI tool. In an 11th hour scramble, they were forced to hand-migrate all the code in order to meet the project deadlines. Taking a holistic approach that includes configuration management as part of architecture could have avoided this embarrassing situation.

Increasing Competition
EAI vendors are continually challenged to recast their value propositions. More and more integration services are being standardized by the J2EE and .NET platforms, and industry consortia such as W3C. Emerging integration standards include JMS for messaging, JCA for application connectors, XSLT for data transformation, and multiple Web services standards for application service definition, invocation, and coordination. While these developing standards still lack the functionality currently available from leading pure-play EAI vendors, it's clear that many of the services offered by the pure-play integration vendors are rapidly becoming commodity items. Consequently, EAI vendors are being forced to move up the value chain. They now emphasize such capabilities as business process management (BPM), business activity monitoring (BAM), Web services integration, and increased support for vertical industry processes or protocols.

Competitors are coming out of the woodwork. In terms of functionality, business process management vendors - including Fuegotech, Savvion, and Intalio - continue to push the envelope of process-centric business integration. BPM provides visibility and control over the work of people, systems, and partners. This goes beyond simply integrating applications and automating business processes; sophisticated BPM systems also support business process design, process simulation, process execution, process monitoring, and process analysis. In terms of price, a host of newer vendors such as Fiorano and Sonic Software now offer JMS-based integration suites at a fraction of the cost of established players. Additional competition comes from vendors in related disciplines such as extract-transform-load (ETL) and electronic data interchange (EDI), who are redefining their offerings as integration solutions.

Most threatening are the industry "800 lb. gorillas," who are clearly taking aim at the business integration market. These offerings will appeal to those customers who prefer to stick with an industry leader even if a smaller, less proven company may have superior technology. IBM announced significant advancements to its WebSphere Business Integration portfolio that provides businesses with a broad set of functions to integrate, manage, and monitor business processes across an enterprise and with trading partners. BEA announced WebLogic Integration 8.1, which provides customers with a unified framework for business integration, simplified production and management, and a new extensible architecture for the rapid assembly and integration of applications, business processes, and trading partner communities. Microsoft's upcoming "Jupiter" technologies will include revolutionary business process management and monitoring capabilities; additional support for XML Web services standards, including Business Process Execution Language for Web Services (BPEL 4WS); and richer developer and information worker support through enhanced integration with Visual Studio .NET and Office.

The leading application vendors are also joining the party. Having finally recognized that they are not the center of the universe, most have responded by adding integration brokers and/or pre-built business processes to their application suites. Some have partnered with EAI vendors to provide integration technology; others opted to build it themselves. SAP recently announced NetWeaver, the foundation of the SAP xApps and mySAP Business Suite solution, which offers a comprehensive integration and applications platform. PeopleSoft's Integration Broker offers integration using Web services, flat files, custom code, or a JMS-based connector to IBM MQSeries. Yantra's enterprise software for real-time coordination of the extended supply chain includes a business process-oriented integration platform and prebuilt processes for coordinating and controlling distributed orders and inventory across multiple business units, customers, suppliers and trading partners. Siebel has taken a slightly different tack by creating the Universal Application Network (UAN), which claims to transform application integration "from a complex and expensive technical challenge into the strategic ability to implement customer-facing business processes across and beyond the enterprise." (All the leading pure-play EAI vendors, as well as IBM and Microsoft, have announced support for UAN.)

Economically, it was a tough year for the pure-play EAI software vendors. All have seen dramatic revenue reductions as EAI projects are delayed, scaled back, or cancelled outright. Several vendors are responding to the perception that EAI projects are too expensive by reducing license costs. At least one vendor now reports more revenue from software services than software licenses (an untenable situation for a software product vendor). Most pure-play EAI vendors have reduced their workforces and all are rapidly burning through their cash reserves; some have seen their stock market valuations essentially evaporate.

Despite competing claims, the integration marketplace is still very much a horse race. Although there are half a dozen "usual suspects," none can yet really be considered the dominant player. Stiff competition means everyone must continue to spend significantly on marketing, sales, and engineering in order to win market share, while continuing to reduce costs. Now that the established industry players are joining the fray, the competition will only intensify.

Going Forward
Though EAI technology has been available for several years now, significant challenges still exist to successfully deploy it. To be fair, not all the problems are directly related to EAI technology itself; they have as much to do with effectively managing complex programs, managing change in large organizations, and nuts-and-bolts system integration expertise. As with any new technology, there have been some teething problems but the integration approach is essentially sound. While the road to business integration has been rocky, it would be a mistake to conclude that the future for EAI is bleak.

Far from being a flash in the pan, it seems clear that EAI technology is following the historical path described by the Gartner Group's Technology Hype Curve (see Figure 1).

 

Initially, EAI technology climbed the Peak of Inflated Expectations, overhyped by analysts and vendors alike as the next new thing. As we've discussed, challenges in early business integration deployments resulted in disappointing ROI for some projects. Others predicted that new technologies like Web Services would make EAI obsolete. (Personally, I believe Web services and EAI are complementary rather than competing technologies, as I discussed here last May [WSJ, Vol. 2, issue 5).) This encouraged doomsayers to push EAI into the Trough of Disillusionment. Yet increasing demand for intra- and interenterprise integration validates the merits of the EAI approach, as evidenced by efforts to standardize integration-related services and the race to embrace business integration by leading application and platform vendors. Today, many customers across a broad range of industries are steadily climbing the Slope of Enlightenment using EAI. They have more realistic expectations and the benefits of lessons learned on how (and how not) to implement this powerful technology. Thus, the future for EAI technology (if not for certain vendors) is still pretty bright. It should only be a matter of time until we reach the Plateau of Productivity, in which real-world benefits of business integration are broadly demonstrated and accepted.