On the one hand, there's extreme pressure on businesses to deliver new customer offerings and innovative business capabilities, match increased competition, and deal with new partners and providers of niche services to offer cheaper service.

On the other hand, most of us have legacy systems that prevent us from delivering to the business at the speed of business opportunities. Legacy applications house only parts of the business entity and the data is stored in proprietary structures. The data is also owned by proprietary application logic (packaged or custom-built) that represents very narrowly defined business functions where the business rules might be applicable for a single business area or line of business.

The industry is looking to SOA as a mechanism to help businesses become more agile. However, the question is how this would be achieved in an enterprise that has numerous critical path legacy systems that have the limitations mentioned above.

This article provides one possible solution to this conundrum. The architecture requires the transport of legacy data on-demand or in a scheduled manner to a common staging repository that lets the service layer apply non-legacy, non-line-of-business-specific business logic for enterprise-wide information sharing. Further, the service layer has logic that works on the base business data captured in disparate sources across the enterprise and transforms this into information.

This architecture pattern, displayed in Figure 1, offers a possible alternative to insulate the legacy systems from changes while promoting the reuse of the data collected by the legacy system. The solution also enables IT to react to the new business rules that need to be applied to one or more aspects of a business entity.

The fundamental theme encapsulated in the architecture pattern is the fact that an enterprise can leverage powerful ETL tools and processes to gather, reconcile, and finally populate enterprise repositories in an attempt to reuse information stored in multiple legacy data structures/sources. Also shown here is the fact that the consumer is unaware of how the service provider gathers the relevant data and what the sources of the data might be. The consumer's only view to the business entity and the business rules governing the business entity is via the use of an enterprise-worthy service provider interface.

Standardized ETL processes are leveraged to transport, reconcile, and transform the granular data that represents various aspects of the business entity into enterprise business information, while services are leveraged to apply enterprise-wide business rules to make business information and business functions available to the business in a consistent manner.

The pattern also demonstrates how an enterprise can extend the use of specialized or silo legacy data to service-enable all of the various legacy applications. The cleansed and reconciled data is populated into an enterprise repository that is then accessed using enterprise rules. This insulates the consumer from having to know or care about the details of invoking these legacy batch processes. The service provider offers a clean layer of indirection between the consumers and the sources of enterprise-worthy information that is locked up in legacy repositories.

Here is a step-by-step definition of the sequencing of calls as shown in Figure 1 that represents how SOA and ETL can work together in the real world. Figure 1 also shows how the architectural components are wired together to achieve the goals of the pattern:

1.  The Service Consumer calls the Enterprise Service to submit a request (scheduled) or invoke a service in real-time
2.  The Enterprise Service checks to see if this information exists in its Enterprise Staging Repository
3.  If the information isn't available, standardized ETL processes are executed (on-demand/scheduled) that gather the relevant legacy data, transform, and load the complete enterprise-worthy business entity information into the Enterprise Staging Repository. The ETL process also applies enterprise-worthy transformation and data reconciliation rules to the legacy data prior to uploading the Enterprise Staging Repository.
4.  The Enterprise Service executes enterprise-worthy business rules and processing logic to perform a commonly used business function on behalf of the consumer
5.  The Service Consumer is notified of the result of the execution of the business function

This pattern was applied by me at a large retailer to deal with "collecting" and "populating" merchandise assortment information from various vendor merchandise assortment repositories and private brand merchandise assortment repositories. The ETL process transformed the relevant information regarding various types of merchandise assortments prior to uploading the enterprise repository. This enabled the service provider layer to apply enterprise-level "merchandise assortment rules" to satisfy the "optimize merchandise assortment by region" requests.

In conclusion, the attempt made in the architectural pattern is to show how an enterprise can extend its legacy information assets while insulating the consumer layer from the details of the process. The legacy systems continue applying localized rules to capturing the data while the enterprise service only adds on the enterprise rules layer thus avoiding redundant application of business rules. In addition, front-ending the legacy data sources with a SOA-style service lets the SOA service management infrastructure be leveraged to manage and monitor consumer SLAs without affecting the fragile and customized legacy application code base. Thus, an enterprise that has legacy assets can still move ahead and apply the core principles of SOA such as loose coupling and modular design without having to sacrifice the stability of the legacy systems or decommissioning them.

The result is SOA-style services that can now expose enterprise information to satisfy any business process whether it's from within an enterprise context or from an extended enterprise business context.

Finally, the key value provided by this pattern is to demonstrate how enterprises can embark on the SOA path feeling empowered by the portfolio of legacy assets they have at their disposal instead of looking at them as a liability.