For services that don't have staging instances available what is the external effect of development and test messages? How should load testing the consumer application or service? Are there chargebacks based on service use that apply? There are several techniques to manage the development and test interactions of consumers on dependent services.

1.  Staging Instances: A service provider can make a separate service instance available with an identical service interface that's not connected to the live back-end. Well-mannered interaction with this staging system isn't important. Consumer development can have its way with the staging system that may even automate refreshing its back-end data periodically. It's expensive to build and maintain but there's not substitute for running the same code in a safe environment.
2.  Simulations: Instead of building a full-fledged duplicate instance of a service a mock service-style simulation provides many of the same benefits as a service stand-in. The fidelity of the simulation is less likely than a full staging instance but it can be created and maintained by anyone if the service provider doesn't offer an alternative.
3.  Special Accounts: Using a live service endpoint it's possible to identify your intention by using an explicit test account during authentication. A test account issued by the service provider lets the service provider handle test messages differently or at least correlate transactions with a test account.
4.  Message Header: A custom SOAP header can assert that a message is intended as a test message. The message can trip the service implementation to operate in an idempotent mode or to correlate transactions for post processing and possible compensation.
5.  Data Partitioning: A degenerate case of two-four is the use of specific application data to partition development and test requests from "real" requests. Domain-specific options include using a distinguished warehouse name in an inventory service or a particular book title/ISBN in an online store to indicate a test request.
6.  Compensating Transactions: If none of the above techniques are suitable live transactions may need to be performed and followed up with compensating transactions to return the system to a base-level state. In HR systems hiring a long running test might involve "test employee," giving them a raise then firing them. Keeping the transaction state manually during development and test is probably more trouble than it's worth. It seems plain that sooner rather than later something will fail to get cleaned up properly.

Service Design Time & Consumer Runtime
This is the service provider's chance to break all its consumers and make them wary of ever using that service, or any service, ever again.

Clearly service providers need to keep their consumers in mind when making changes to the service's interface or even its implementation particularly if the new service version drops in at the existing endpoint. Of course this is easier said than done.

Understanding consumer expectations of service behavior is a complex analysis that begins with analyzing changes to the service interface description but must also include runtime semantics like data coherency and consistency. It may also include understanding how your service is used with other services you've never heard of.

Knowing when you're about to break a consumer is very difficult in practice. Analyzing the service interface changes is only the beginning. Service providers likely don't know how their consumers are using the service in conjunction with other services. The best way for service providers to know is for consumers to tell them very specifically by sharing their tests. When service consumers assert the precise nature of service usage, service providers can include their consumer's perspectives in their regression suites. Consumer-provided shared tests form a folksonomy of semantic expectations from the consumer community.

Once a mechanism for sharing tests exists it's a natural extension to begin testing for behavior that doesn't exist yet as a way to drive enhancements in service behavior. Voila, test driven development for SOA!

Consumer applications make build-versus-reuse decisions of services based on what's available in the service inventory and how well it matches requirements in much the same way we buy software from vendors.

Some reuse decisions will be simple. However; we're all human and have imperfect knowledge of future requirements so consumers will eventually need changes to a service interface, its implementation, runtime characteristics like availability or SLA, or even back-end data access and consistency.

This feedback flows as new requirements to the service provider team and can influence future development the same way new features appear in commercial software applications: when the next version ships. This can tie the delivery schedule of a consumer application to the delivery schedule of its dependent services. Schedule coupling rewards service providers who anticipate the future as much as possible and productize service delivery by following the Software as a Service (SaaS) model where the service is productized.

By treating your services as products delivered as services you can take a more holistic approach including:

1. A mechanism for gathering consumer feedback
2. Notification of upcoming versions
3. Forward and backward compatibility expectations
4. Old version support expectations
5. Impact analysis of change

Metadata Design Time?
Once services and consumers have cycled through their respective lifecycle phases and are both at runtime there's still room for change. New policies and policy changes that manifest in changes to runtime metadata can drastically affect deployed services and applications. Security, transformation location, and routing changes can change the way service availability, semantics, or performance in catastrophic or insidiously subtle ways react.

Overall SOA Quality
When a system exists in a single lifecycle its overall quality is under the direct influence of the development team. Time and resources can be added or changes made to improve system quality. For service consumers application boundaries are vague and no longer in their direct control so assessing and assuring quality becomes a distributed problem. Depending on the actions of each dependent service and changes to intermediary metadata the consumer application's quality is now a function of time and no longer isolated to any identifiable test phase.

This raises the question of "Who is responsible for overall SOA quality?" The individual services in the service inventory certainly have their place but who ensures the consistency and the framework to make SOA quality happen?

There are two sides to the SOA-quality coin. The first is in creating well designed and implemented service and applications in the first place. The second is in providing a resolution mechanism when the inevitable happens. There will be problems between consumers and services no matter what. Plan for the inevitable when consumers and providers need to find runtime problems and avoid figure pointing.

Lifecycle governance is the constraint framework to connect loosely coupled development processes. Extending current notions of lifecycle governance to include the provision of a quality framework lets consumers and providers manage the quality concerns of the other. High-quality service components are necessary but insufficient for healthy and holistic service-oriented architectures.

References
From a talk given by Amr Elssamadisy on SOA, Agile, and TDD at SD West:

"Here's the problem in a nutshell. Agile requires that you fix what you break. So if you change the service your team is writing, you must fix whatever that change breaks in the clients using that service. You're responsible for your own messes. SOA, on the other hand, usually assumes that there will be no collective code ownership. So you can't fix someone else's code. What's needed is a way for the team developing the service to run a test to see if their change will break any clients.

For that to work, though, the service team's test suite needs to know what breaks the clients. If they don't own the code then how does this happen? By sharing tests. Everyone shares all their tests with everyone else. That way the client's tests become part of the server team's regression test suite throughout the development process."

• http://en.wikipedia.org/wiki/Software_development_life_cycle
• Ambler's Agile SDLC:
www.ambysoft.com/essays/agileLifecycle.html
• Ambler's XP Lifecycle:
www.agilemodeling.com/essays/agileModelingXPLifecycle.htm#Figure1XPProjectLifecycle
• Ambler's AUP:
www.ambysoft.com/unifiedprocess/agileUP.html
• ErlsTop8SOAPitfalls:
www.infoq.com/articles/Top-8-SOA-Adoption-Pitfalls
• AgileManifesto: http://agilemanifesto.org/
• DCOM-COM with a Longer Wire:
http://msdn2.microsoft.com/en-us/library/ms809320.aspx