The saying "you can't manage what you don't measure" has never been so applicable to software engineering as it currently is to SOA development. As organizations are struggling with the day-to-day development implications of SOA, business leadership is beginning to realize that success does not come without effective organizational measures. Cross-organizational measures are needed to bring transparency to those operational benefactors impacted by SOA's promise of agility and cost reduction. But what should you measure and are all measures equally important?

The answer to these questions is rooted in understanding the nature of the first part of the opening quote - you can't manage. We manage because we want to control and we need to control because we want to reduce risk to have outcome certainty. As such, every organization that implements SOA and every SOA implementation will need to identify and address the specific risks associated with the project. For example, if agility is an important driver for your SOA implementation, what does that mean to your organization and what specific metrics can be used to measure it?

There are several characteristics of measures and metrics, in general, that should be considered when identifying those specific to a SOA implementation. Are the measures predictive or reflective - that is, do they tell us about the future or the past? Are they theoretical or practical - considering theory tends to shed light on emerging areas, while practical tends to be more useful in narrowly defined situations. Are the measures internal or external - do they look internally into the services or externally towards their application? Are they objective or subjective - bearing in mind that objective measures tend to be less dependent on personal perception. Are they quantitative or qualitative? Quantitative measures lend themselves to classification, where qualification tends to be more descriptive. Who is the audience for the measure - are they meant for corporate, management, project, or developmental use? As you can see, there are lots of options and it is unlikely that any two organizations should have the same exact measurement system.

That said, there are a few measurement areas that should be looked into by any group and could be used as a starting point. I've broken those areas down into four core categories: Corporate Metrics, Management Metrics, Project Metrics, and Service Development Metrics. Across those categories are 10 measures that seem to get the most attention and relate directly to successful SOA implementations:

Corporate Metrics
1. Revenue Per Service
One of the most important measures for any company is revenue per employee. This number captures both the value an organization delivers, and the productivity achieved based on the value created by the employee base. Unfortunately, today there is no such universally accepted measure for services. Why? Mostly because SOA, up until now, has been dealt with as an engineering activity by technical people, which is normally tracked as cost in R&D. But this needs to change if SOA is going to be able to cross the business chasm.

For the same reasons that measuring revenue per employee is important, having a historical perspective on revenue per service will enable organizations to quantitatively assess both the value of the service and the productivity of the service oriented architecture through which it is delivered.