Lately we've done a lot of research and publishing on how Dev & QA teams can ensure better quality throughout the software development and release lifecycle. But what about the IT Operations guys?

It seems we are encountering a disconnect between the Enterprise Architecture and Integration disciplines that are moving to SOA, and the people who need to monitor and maintain these systems in deployment. There are a lot of companies starting to focus on better SOA Governance, which includes the management of business processes and workflows, and the services and applications behind that. Great stuff happening on that side of the IT shop.

However, often the actual running applications are still in the purview of IT Operations teams. Don't get me wrong, it is a great thing that we have a team that is manning Mission Control -- hopefully providing an impartial report and monitoring function that is crucial, when you have live customers depending upon these apps in deployment. And there are mature tools out there that provide this kind of live "dashboard" of how the implemented apps are performing, like TIBCO Hawk, HP OpenView, IBM Tivoli, CA/Wily, etc.

So why would these IT Operations teams be interested in SOA testing and validation? Well, it's really about "causing the cause" of performance and latency issues in these systems. If you are trying to prove that live SOA infrastructures are ready to roll, it's not enough to sit and watch the dashboard for that "Check Engine" warning light when a condition is caused by too many customers using those apps.

The point I'm making is to use automated Test Execution as a way to drive expected and unexpected behaviors into your SOA Monitoring dashboards. This can be done both at integration time in staging, and against live applications, if you can provide a test window of time to make this process happen.

For instance, we recently worked with a large energy industry customer who wanted to prove that their live implementation was able to handle a realistic profile of varied transactions in their system. So instead of waiting for a rush, they manufactured one -- by executing automated test transactions into the system, and monitoring the performance results within their Operations dashboards. Then they could feed the same data out to their SOA Testing platforms to tell the tests if they performed according to service levels.

We recently put out a little news on this kind of integration - it's a pretty exciting way to apply Test & Validation to the Monitoring applications that have been around and evolving for some time: http://www.itko.com/site/company/news_65.jsp.