SOA Deployment Example
With the growing complexity of Web applications, having a process for continued performance improvement and problem avoidance is critical. Yes, performance problems will occur and are sometimes unavoidable due to situations beyond the control of developers or operations personnel. The key is to "bake in" performance through a culture of cooperation where developers and operations work together so that performance problems can either be resolved proactively or detected and resolved quickly before they impact user satisfaction.

The ITIL Application Management Volume sets forth the lifecycle of software development projects. The ITIL framework extends the traditional four-step software development lifecycle of requirements-design-build-deploy to include two additional steps: operate and optimize (Table 1). These last two phases of the lifecycle are particularly important when dealing with complex Web applications. On the one hand, SOA applications are never fully tested because of (i) complexity, (ii) unknown performance issues with constituent services, (iii) lack of control over third-party infrastructure service providers, and (iv) the ever increasing time-to-market pressure that prevents exhaustive testing of all possible usage scenarios. Hence, the significance of the additional two steps: operate and optimize.

The addition of the two extra phases to the ITIL application lifecycle means there has to be a link between traditionally discrete development and operations functions. In other words, development and operations must have a common view of the business impact, end-user performance, application infrastructure performance, incidents and problems. Ideally there's a single tool and set of metadata that can bridge these two functional groups and offer (i) developers opportunities for optimization, and (ii) operations personnel the ability to identify and remedy problems efficiently.

Besides the unknown performance issues inherent in the constituent services, SOA application response time can also be impacted by:
• Adding or changing application functionalities;
• Degrading or enhancing infrastructure capacity; and
• Changing or modifying infrastructure components.

For complex SOA applications, developers are constantly called on to deal with production problems, whether to patch a code-level problem that impacts performance, or an infrastructure problem that requires a workaround. Because of their knowledge of the application, they're also called on to serve on triage teams attempting to recreate or diagnose potential or real performance problems. This activity has a tremendous impact on the development schedule and the developer's productivity.

The automated and continuous monitoring and diagnosis of transactional problems from a real user perspective performs three important functions in a production setting:
• Provide a common case management tool and relevant set of data so that developers and operations people can collaborate to quickly pinpoint the cause (problems in ITIL terminology) of performance problems (incidents in ITIL terminology), whether inside or outside the firewall. If the problem is located inside the firewall, the team can quickly narrow down which tier of the application infrastructure, which server, which method call, which SQL query, or combination thereof caused the performance problem. The benefit of this approach is to eliminate, in most cases, the need to organize cross-discipline triage teams to debate, reproduce, and diagnose problems. The bottom-line effect is improved IT efficiency and quicker problem resolution.
• Afford a continuous feedback loop whereby developers can gain insight into how the application's performance is being impacted by actual usage, features, and infrastructure issues. For example, if a feature turns out to be a major consumer of computational resources such that it impacts the service level of more critical features then development might consider allocating resources to re-engineer this problematic feature. Having this information will give developers the foresight to proactively make modifications to the application and/or code-level implementation trade-offs so that performance issues aren't just considered in hindsight.
• Assist in regression testing based on real user traffic after code-level and/or infrastructure changes.

To facilitate the cooperation between development and operations in matters of SOA application performance, there has to be a common platform for sharing performance information (metadata) that is relevant to both teams and a defined process for acting on the information. In a way, this is akin to replacing the traditional Chinese wall that separates development from operations with a "glass wall." A metaphorical wall, or predefined and enforceable set of business policies, is important so that developers can't arbitrarily modify released code or the underlying database or infrastructure running the code without following proper release, change, and configuration management protocols. Instead of being opaque, this "wall" should be a "glass wall" so there is informational transparency between the two functional groups.

Development and operations use different tools. Data collected or generated by development tools has no meaning to the system management or DBA tools used by operations personnel and vice versa. This tower of Babel situation, if not remedied through the use of a common tool and metadata, makes the implementation of a ITIL process impractical.

ROI Considerations
Performance issues can easily eliminate the expected ROI from SOA initiatives. Slow performance could lead to excess customer or user complaints and impact the productivity of IT staff. Two statistics clearly highlight these hidden costs:
• Forrester Research reported that 72% of user complaints are not detected by existing IT tools or process.
• And Gartner reported that up to 30% of developers' time is spent troubleshooting applications in production.

Both statistics would be even higher for Web-based SOA applications because of increased complexity and lack of control and visibility into the constituent service "black boxes."

For most organizations, the business impact of customers receiving sub-par application performance is immense and not limited to e-commerce merchants. For example, while frustrated e-banking customers can't click to a competitive bank's Web site to pay bills, performance issues will drive up help desk or customer support costs, and eventually drive customers to move their business to a competing bank. Even for internal or non-customer-facing applications, performance issue will lead to a productivity loss among employees or partners.

Beyond the cost to the business, performance challenges also impact the development cost of SOA applications. The cost to an organization could be in the millions of dollars if a good part of developers' time is spent fixing production problems.

All of these potential "downside" costs of SOA should be considered and minimized prior to embarking on a SOA initiative. Figure 1 is a screen capture of an interactive tool that can help IT management or architects quantitatively determine the hidden cost if browser-to-database monitoring and diagnosis tools aren't used to deploy and manage SOA application in production. This free interactive tool is available at www.symphoniq.com/products/roi.asp.

Conclusion
To maximize the benefits of SOA initiatives, it's imperative that the process-oriented guidelines advocated by ITIL be followed. Only with the insight and knowledge offered by understanding the performance profile of each real transaction from the real user's perspective can IT successfully deploy and manage the complexity of SOA applications.