Mainframes were the first computing platform of corporate information technology. As the industry has grown, mainframes have continued to evolve and integrate into the various incarnations of enterprise architecture. In fact, as the first computing system, mainframes enjoy a special role when looking at enterprise architecture, as mainframes have participated in virtually every flavor of architecture, starting with the incarnation of IT, when a mainframe was the architecture.

Mainframes were initially deployed as monolithic, standalone systems. The prevailing attitude of the day was that all applications could and would reside on a single large computing platform, the mainframe. Application integration was all nicely handled through mainframe resident data stores, and callable transactions. However, as IT evolved into client/server and other non-mainframe architectures, integration shifted as well.

Enterprise application integration (EAI) was born out of the need to integrate disparate systems, without using ad hoc point-to-point custom-developed interfaces for every project. EAI provided the promise of unwinding the tangled mess of interfaces that had grown to ensnare every corporate data center. Mainframes were still the bedrock of corporate IT, however, now there were thousands of interfaces feeding other systems in myriad point-to-point application-specific standards and semantics. For the mainframe, MQ was the answer to standard EAI integration. However, as we quickly learned, EAI didn't solve the tangled mess of integration points. EAI made integration easier; however, a lack of standards didn't let different vendors' EAI tooling work well with each other, and the integration semantics were still developed specifically to meet the needs of the specific point-to-point application integration driving the project.

Service Oriented Architecture (SOA) is being heralded as the savior of enterprise integration, finally creating re-usable interfaces to unwind the complex point-to-point integration. SOA does in fact nicely solve the standards process, and while SOA doesn't require or mandate any technology, most SOA initiatives are creating easily consumable services as Web Services. However, as for the second issue of creating reusable services it's useful to look back at history at the lessons learned about creating a reusable integration point.

Mainframe applications represent large chunks of business process logic. Consider a mainframe application that's used solely for order processing. The monolithic mainframe application would hand a number of sub-processes involved with order processing, for example, order entry, accounts receivable, credit, and customer maintenance. The mainframe system and the associated application flow represented and supported the business processes in the organization. As the business processes of the organization evolved, the mainframe application logic evolves as well, or eventually is replaced.

Once the environment evolves into a heterogeneous environment of mainframe, client/server, and other technology, the integration will mirror the macro business processes associated with the capabilities of the application packages on each platform. Let's consider what happens to our mainframe shop when the business decides to replace the customer maintenance and credit management functions with a package to allow consolidation of the customer and credit maintenance business processes across various divisions.

Consider that the macro business process was that the consolidated customer operations group would be responsible for initially setting up all customers, and that no orders could be placed by any of the divisional entities until that customer was established in the system and the credit department could determine an appropriate credit line based on the terms and credit worthiness of the customer. As such, the IT department determined that the new client/server CRM suite would be the master of the customer data, and a batch interface to the mainframe order entry system was constructed to populate/update the system with the latest information. The IT interface matched the macro business process, with a formal document, the interface between systems of the customer master, being passed between the credit department and the order processing department nightly.

Of course, then another interface is written to a reporting system to provide management reports, and another to reconcile the accounts receivable, and another to update a sales force automation product, and another to update the Web-based order status, and so on.

But, besides the issue of interface proliferation, there's another issue associated with the granularity of the business process the interface is supporting. The interface mirrors the business process; however, it mirrors a very large course business process differentiation (order processing department versus the customer management department). Let's consider the impact of our division's plans to offer special promotional pricing based on total cross-divisional sales.

Our CRM system keeps track of total divisional sales; however, that information wasn't passed back to the mainframe in the interface so rather than waiting for IT to update the interface, our sales team uses the standard desktop integration tool of choice since PCs were introduced into the IT stack, a spreadsheet to pull the information from the CRM, manipulate the order from the order management system, and decide the correct discount.

Like it or not, the lack of agility and flexibility in IT systems and the associated integration has created a whole secondary market of PC-based integration needs and techniques. Mainframe terminal emulators have scripting languages to pull information off the mainframe to the client, client/server systems have the ability to export reports locally, and desktop tools to manage, manipulate, and dissect these datasets have grown very powerful. The user community in your organization has learned how to integrate their business sub-processes without using IT to achieve the flexibility and agility that the business requires. Not an ideal solution, but workable, until the Internet came along, and all of a sudden, customers didn't expect to have to deal with a person. The systems needed to be as smart, flexible, and agile as your business users (Figure 1).

The hope for SOA, therefore, isn't just that it's standards-based, and therefore reusable. It's also that the integration points into the systems have to mirror the details of the business process. Too coarse of a service definition and you have the same problems with lack of flexibility and agility. Of course, if you define a too finely grained service, most of the business process logic wrapped up in your legacy assets will have to be recoded into the workflow orchestrating these services. Therefore, the real hope for SOA to be able to provide the agility and flexibility to the enterprise is defining services that mirror the business process at the correct level of granularity, allowing for easy reusability to quickly compose supporting new business processes.

The standard process of defining and documenting business processes is a top-down analysis starting with the large macro-business process definition and iteratively breaking down the processes step-by-step. For example, order management consists of entry, change, inquiry, etc. Business process analysis isn't a new discipline; however, it's a long and expensive process. Often, by the time the business processes are fully identified and documented, the business itself has changed. And all too frequently, as the process becomes more detailed, the analysis stops - and in the world of SOA, where getting the micro-process details right is essential, taking business process analysis down to the details is an absolute necessity.

Business process discovery (BPD) is an emerging field of tools and methodologies that allow for a bottoms-up approach to business process analysis. Rather than starting with a top-down course-grained to finer-grained approach, BPD examines the current work processes starting with the details, and builds up a picture of the business process based on the actual evidence of the work being done.

The allure of BPD is to very quickly discover, detail, and document localized business processes to enable the definition and construction of SOA services. Rather than waiting for a long, expensive top-down analysis, BPD can very quickly provide the evidence required to make intelligent decisions on the granularity of a service in the context of the supporting business process.

BPD is not only quicker; it also produces a higher-quality business process description. The traditional top-down approach makes process assumptions based on interviews with business process experts. However, no matter how skilled the expert, and how good the BPA methodology, there is often a great deal of error associated with the process, both in terms of omission of process details and exceptions, as well as misunderstanding the process nuances executed by different business groups.

Flaws in business process details become critical in the context of creating a service that mirrors the detailed business process. Interestingly, the standard method to resolve these unforeseen situations is to do a variant of BPD, which in standard software development would be called testing. Long arduous testing cycles are the standard pre-requisite to any major systems project, and perform the role of BPD, after the system has been designed and built, to find all of the details forgotten in the design process. While effective, this slow after-the-fact process seems to limit the promise of finally providing the agility and flexibility in IT the business requires.

SOA brings the promise of IT agility through existing system and interface reuse if the services are defined appropriately. Defining what an appropriate business process is to encapsulate as a service is the key to SOA success. Either too fine a grain, or too course, will result in continued investment in the service, at best re-versioning the service, at worst, non-reusable service proliferation.

Business process analysis tools and methodologies provide an excellent top-down analysis to put business processes into context. However, they are poor at deriving the details of a business process. Business process discovery tools excel at discovering the details of a business process from real business use. Applying both of these tools together can quickly provide a business process map, complete with details, allowing your services to finally meet the needs of the business, providing a quick, agile IT architecture to meet the ever-evolving needs of every business (Figure 2).