One of the ongoing challenges for business today is finding ways to do more with less. Companies are under relentless pressure to deliver products and services to market faster, better and cheaper than ever before. Investments in information technology are expected to drive the business forward, not only in terms of gaining efficiencies and increasing responsiveness, but in creating new top-line opportunities.

Ironically, most corporate IT organizations allocate 75%-85% of their annual budget just to "keeping the lights on." At the same time, research shows that a typical server's average utilization is 15%-25%. While these figures may not reflect your particular organization, they highlight the fact that there's a great deal of inefficiency in IT today, consuming large portions of budgets that could be put to better use for more strategic initiatives.

What's driving this inefficiency?
While there is no single cause, much of it results from mergers, acquisitions or functional silos in the organization - all of which contribute to a bloated IT infrastructure, rife with overlapping and duplicate applications and systems. In the face of tight deadlines and the ongoing demands of the business, IT often opts to live with these redundancies - integrating systems where necessary to keep them consistent. While this approach may result in quick time-to-delivery, it severely inflates recurring costs.

While some IT organizations may be content with this status quo, executive management sees "aligning IT with business" as its number one IT-related concern. However, with as little as 15% of the budget available to service the changing needs of the business, IT just doesn't have room to maneuver. It's this realization that's leading many organizations toward a service-oriented architecture (SOA), an approach to architecting the IT infrastructure that eliminates redundancy and accelerates project delivery via consolidation and reuse of services, often referred to as Web Services.

In the past, SOA adoption was hampered by a lack of agreement on a "common language" for applications to communicate with each other. Happily, Web Services in the form of XML, SOAP, WSDL and other related standards has begun breaking the logjam that made SOA impractical and so has become the foundation technology for building an SOA.

Of course, using Web Services is not a panacea. Though many companies are launching tactical Web Services projects, few are reaping their full value. The value of Web Services escalates when organizations harness the economies of scale of consolidation and reuse. In other words, when customers move from an ad hoc use of miscellaneous collections of Web Services to a more formalized SOA, the value of those services rises dramatically.

Case in point: If you have just one Web Service, but it's used by five different applications due to consolidation or reuse then you have significantly reduced total cost of ownership (TCO) - the essence of SOA. How is that TCO benefit achieved? Every time you can reuse an existing service, whether proactively by planned reuse or reactively by decommissioning redundant services, you buy and operate fewer machines, license and operate less software and manage fewer individual service lifecycles - all significant quantifiable benefits. So the number of times you reuse services in different projects - not the number of services, but, the number of different uses of services - is a quick measure of success in SOA adoption.

SOA Challenges
The fundamental goal of Web Services and SOA is to improve ROI and TCO. However, the use of Web Services doesn't guarantee your results. There are a number of challenges, many of which are organizational and not strictly technical, that your company needs to address proactively to achieve measurable benefit.

One problem is that standards don't guarantee interoperability. By their very nature, standards are designed to support many different uses across many different types of organizations.

For example, security standards allow many different types of credentials including how a user is identified. If the sender and receiver don't both understand the same types of credentials, they can't talk together. The common language breaks down.

Narrowing down the set of acceptable ways to use standards from all the available options is something your organization needs to address with policies and procedures. Of course, many of these policies and procedures will naturally need to change over time as regulatory or other compliance requirements change, so when thinking about TCO you need to factor in the inevitable cost of change.

A second challenge is determining who pays for a service shared by many applications. With traditional line of business applications, figuring out who pays is easy - since one team owns everything. For a shared service, ideally each line of business should pay proportional to their use. Those who use it most should pay the most. Essentially this is a transfer-pricing model. Beyond the accounting considerations of how to implement transfer pricing policies and procedures effectively across lines of business, you must also consider how to track usage by each line of business accurately - if you can't measure usage, you can't charge for it!

A third challenge is ensuring that service levels are met. To end users, the customers of the IT infrastructure, the order management application is still the order management application whether it's built as a monolith or it leverages shared services. In either case it must meet the same expectations of performance, availability and functionality. Conversely, if the same user uses two different applications, he may have different expectations of each - even if both applications leverage the same set of shared services.

While shared services serve many applications, they need to be approached in the context of the applications they serve to ensure the "customer experience." For example, changes may lead to unintended consequences. For instance if an application using a service has an unexpected load increase it consumes the service's capacity most likely reducing the service level for all the other applications. The use of shared services means that understanding the nature of the interdependencies plays a key role in understanding the effects of any changes.

A final challenge is that, as one team begins leveraging services built by other teams, they no longer have visibility into all the moving parts that make up their overall application. This factor has a number of implications. For example, how does an application team ensure that its overall application is secure, when it's built using services from other teams? If its application is exposed to partners and customers via the Internet, how does the team ensure that the services it's using aren't vulnerable to malicious attacks aimed at stealing or corrupting information? Maybe the services the team are leveraging haven't been built with the same requirements in mind. Similarly, when something goes wrong, how can you figure out if the problem is due to the application itself, or to services leveraged by the application? And who dictates security and business policy as it relates to shared services?

SOA Command and Control
While the core Web Services standards successfully address the mechanics of letting applications to talk to one another, successful SOA implementations mean addressing the challenges that lie beyond the pure mechanics of communication. The complexities are numerous: stakeholders with different agendas, policies with cross-functional implications, service levels that must be maintained at all costs, complex interrelationships between services and no lines painted on the data center floor to connect any of the dots.

Left to grow on its own, a network of Web Services will quickly degenerate into a tangled spaghetti of brittle, single-use integrations and fail to achieve the economies of scale or the cost and flexibility benefits of SOA.

These challenges call for a new breed of solution - SOA Command and Control - that addresses the various technical, business, and organizational requirements and coalesces all pertinent knowledge in the SOA into a form that's understandable and actionable.

There are five key imperatives of SOA Command and Control: Align, comply, observe, respond, optimize. These five imperatives encapsulate the required , and the associated value, of SOA Command and Control. Let's take a look at each of them.

Align
IT is successful only if the business is successful, so IT must always be aligned with the business. SOA Command and Control must allow you to measure SOA activity against your business objectives to understand its current impact on the business, to determine how it's trending, and to predict where it will go in future.

In addition, SOA Command and Control should let you customize or tailor the services offered by your SOA quickly and easily to address tactical business opportunities. With SOA Command and Control you can:

• Understand which of your most important consumers, regions or divisions are getting the best service.
• Provision services to meet the unique requirements of a new high-value consumer.
• Plan and execute a migration to a new version of a service so that your standard customers roll onto the new version before your most important customers.

Comply
True SOA Command and Control empowers stakeholders to move from a passive role to an active role of driving policy changes immediately and automatically across the organization. In addition, stakeholders gain visibility as to where and when the policies and procedures are being applied and/or violated. With SOA Command and Control you can:

• Uniformly apply and enforce a security policy across many services without a service-by- service development effort.
• Become compliant with changed privacy regulations.
• Define what service levels can be promised to different consumers.

Observe
By definition, SOA Command and Control provides both detailed and at-a-glance visibility into the inner workings of your SOA at any point in time - automatically, without expensive, time-consuming manual configuration. This facility lets you understand SOA-wide patterns and trends that would never be uncovered with solutions that provide simple statistics and only threshold, rule-based or predictive alerting. With SOA Command and Control you can:

• Understand what your service level is right now, broken down by geographical region, customer segment and service.
• Discover who depends on which service, and what their different usage patterns are.
• Determine what "hotspots" are most vulnerable to attack or which will be the first to fail as loads increase.

Respond
When a symptom of a problem is detected - whether it's a functional issue with a newly rolled-out version of a service, a malicious attack or a performance issue - you need to be able to determine the root cause of the problem and how to respond to it effectively. Root cause analysis is important in an SOA because symptoms rarely appear at the location of the root cause - and the root cause may be a service owned by a different group. With SOA Command and Control, you can accurately and automatically determine the root cause of problems, without expensive, time-consuming manual configuration of rules or relationships. And, once you determine the root cause, you can respond in one of many different ways such as notifying administrators, black-listing users, rolling back service changes, rationing capacity, or modifying documents in transit. These responses can be triggered manually, fully automated or even manually overridden when automated responses don't produce the desired result. With SOA Command and Control you can:

• Roll back consumers to use a previous version of a service if a problem is encountered during the migration.
• Enrich or cleanse incomplete or ill-formed documents rather than reject them.
• Continue to meet the service-level expectations of your most important customers while under unexpectedly heavy loads.

Optimize
As with any IT infrastructure, services have a finite capacity to process consumer requests. Determining the capacity requirements of services is especially complicated because each consumer has a different pattern to use with different kinds of requests, and different peak usage periods. And, as new consumers come online, they consume capacity from the service and potentially affect the service level of everybody else. SOA Command and Control lets you both proactively and reactively optimize the allocation of scarce service resources. With SOA Command and Control you can:

• Offload resource intensive processing for frequently changing policies.
• Predict what changes will cause your service to become a bottleneck, breaching the service-level expectations of consumers.
• Ration capacity, giving more capacity to more important consumers or reserving specific servers for high-value customers.

Achieving SOA Command and Control
Traditionally the five imperatives of SOA Command and Control have been addressed directly by development or line-of-business application teams as part of the applications they build. Because application teams are intimately familiar with their own applications, the strategy of building security or operational policy into the applications was often effective.

In this environment, the role of cross-functional IT stakeholders such as security officers or deployment architects is to write documents that outline the policies that application teams must follow. Unfortunately, by delegating the implementation of policy to each application team, you lose economies of scale and control over whether the policy has been correctly understood and implemented. Worse still, to change policy requires a complex one-off project for each and every application or service.

Because of the limitations of this model, products have emerged that attempt to empower cross-functional teams to gain complete control over policies that span applications and services. Unfortunately, while this model appears effective on the surface, it breaks down because these cross-functional teams don't understand the business context of the different applications and services. For example, if a change to privacy policy requires that all personal identities be signed and encrypted, the cross-functional teams will have no idea what data in any of the tens to thousands of documents are classified as personal identities.

What is required instead is a fundamentally new approach to addressing this challenge. Instead of providing tools that let one team or another be responsible for every aspect of the problem, you need a solution that enables the different stakeholders to collaborate effectively - each maintaining control over its own area of responsibility, while seamlessly sharing the knowledge necessary to let the others do their job once.

With an effective SOA Command and Control infrastructure, policies can not only be defined once, centrally, but also automatically enforced in the fabric of the network itself. The capabilities of an effective SOA Command and Control platform lets organizations bypass the knowledge gap and successfully achieve the economies of scale as well as the critical cost, time and flexibility benefits of SOA.