Web services allow for the delivery of SOAP messages over any protocol. A common misconception is that all SOAP messages must be transmitted over HTTP. While that approach is useful in many cases, there are situations where it makes sense to use alternatives. This paper investigates situations where HTTP does not scale sufficiently for enterprise Web service deployments and looks at available alternatives.

HTTP and Scalability
HTTP was designed for serving Web pages under the assumption that the protocol would only be required to send a request and receive a response. This paradigm has worked very well for the World Wide Web and has been ubiquitously accepted as its standard protocol. When a person makes a request on an interactive Web site, they are typically interacting with an application server tier (J2EE, .NET, or scripting languages such as Python and Perl). Each person is running a browser client and doing only one thing at a time on that particular site. In a typical enterprise, however, an application server fronts a number of back-office systems that provide critical business services. The application server usually supports a number of users concurrently. This implies that it typically needs to make a number of concurrent requests on those back-office systems.

Many people believe that a move to service-oriented architecture (SOA) implies a move to SOAP/HTTP as the ubiquitous protocol throughout the enterprise. What few seem to realize, however, is that the SOAP/HTTP approach has inherent scalability limitations under certain circumstances. Simple Web browsers have been the de facto HTTP client to date, and they are in essence single-threaded clients as far as the server is concerned, making only one request at a time over a given connection. This has created a perception that HTTP can be scaled as needed. To date, it has been scaled only for communication between browsers and application/Web servers, typically through clustering, replication, and the use of hardware load balancers. Unfortunately, scaling communications between an application server and back-office services cannot be solved satisfactorily using the same techniques.

For example, assume we have an online banking system with support for up to 4000 concurrent users. The Web tier comprises a cluster of application server instances behind a hardware HTTP load balancer. In order to fulfill the online banking business function, there are three Unix-hosted services and a mainframe-hosted service utilized by the application server. In a world where SOAP/HTTP is the only protocol, the application server will have to support an incoming connection from the browser, and one additional connection out to each of the four back-office services for each concurrent user. This is because HTTP demands that you wait for a response before you send your next request over that same connection. It has no concept of a request identifier, which is a core requirement to enable connection sharing.

One could of course just serialize the requests, awaiting each response before sending the next request on each given connection. However, this is a waste of resources because the back-office server is not doing anything with this connection until the response is sent back to the client application, and most back-office systems have the capacity to handle multiple concurrent requests.

Interleaving requests over a single connection would be the ideal. It would allow an enterprise to achieve the same level of concurrency while using fewer resources. One would send a number of requests to a server over a single connection and receive responses as they become available. The client can correlate responses based on a request identifier. This would allow responses to be returned as soon as they are ready (which may differ from the order in which requests were sent). Unfortunately, the HTTP specification forbids such interleaving.

The obvious conclusion is that a standards-based protocol that allows for request interleaving is needed. This would allow the sharing of a single connection between the application server and each back-office system. In the example previously outlined, if the application server had an upper limit of 1000 connections it can open at once (file descriptor limit), in our SOAP/HTTP world, each application server would be limited to concurrently supporting only 1000/5, or 200, clients. A typical workaround for this problem is to add application servers. If enough are added to support 1000 clients, the problems propagate into the back-office servers, which are now maxed out on the number of connections they can keep open. Creating pools of back-office server instances is prohibitively expensive, especially if they are hosted on a mainframe.

This problem has been solved in the past with connection concentrators, but because we cannot interleave HTTP POST requests, HTTP-based communication cannot be concentrated. Clearly, HTTP is not capable of scaling in such an environment.

HTTP 1.1 supports a feature known as "request pipelining." Pipelining allows a client to send multiple requests over a given connection without having to wait for each response. However, it is not as useful as interleaving, as the HTTP/1.1 specification (see the first entry in the References section) mandates that: A client that supports persistent connections MAY "pipeline" its requests (i.e., send multiple requests without waiting for each response). A server MUST send its responses to those requests in the same order that the requests were received."

Pipelining was designed to streamline the downloading of elements within Web pages over the Internet, supporting only HTTP request types that may be reissued without any change to the server state (idempotent requests). The HTTP/1.1 specification is very clear about this point: "Clients SHOULD NOT pipeline requests using non-idempotent methods or non-idempotent sequences of methods (see section 9.1.2)."

Web services, on the other hand, typically use the HTTP POST request type, which can be non-idempotent. Therefore, pipelining cannot be used.

Roy T. Fielding (the primary architect of the HTTP/1.1 protocol) spoke at ApacheCon November 2002, about a new protocol that he is working on called "Waka." In his presentation (see the second entry in the References section) he described Waka as "…a new protocol to solve HTTP's current problems in a generic way."

He went on to mention support for interleaved data and metadata delivery. Waka has not yet been fully specified, so the details on how Waka intends to support interleaving are not yet available. You can track the progress of Waka at the project Web site (see the third entry in the References section). At this time there are no implementations of Waka available.

• Figure 1
• Figure 2