Resource

State

Since REST Web Services ride the ubiquitous HTTP protocol and piggyback on the Web infrastructure, their presence is omnipotent and can be polymorphic in nature. This can be explained by the various number of REST-style services that currently exist on the Web. These include a client/server style, stateless and cached with a uniform interface, even code on-demand that is exemplified by Java applets.

You can locate resources by a Universal Resource Identifier (URI), and the operations that might be performed against those resources are defined by the HTTP specification. The core operations include:

• GET - This operation returns a state representation of the identified resource. You can determine the state by a number of contextual factors such as who is submitting the request, the parameters of the operation (passed in as either HTTP headers or query string parameters), and the current session state maintained by the service provider.
• POST - This operation performs some form of application-specific update to the identified resource. The behavior of this operation is completely up to the service implementing it. The data returned by this operation is also completely up to the application.
• PUT - This operation creates a new resource at an identified location (URI). The operation input must include a state representation of the resource. It is up to the service to create a resource based on that state representation.
• DELETE - The DELETE operation destroys a resource at the identified location (URI).

We'll look briefly at the various aspects of testing Web Services in general and then how it's done for REST-based services.

Functional Testing or Black Box Testing
This testing is carried out to ensure that the Web Service functions per the design and the requirements. These only deal with proposed inputs and expected outputs. Service test cases are generally mapped to business use cases that outline all the permutations and combinations of inputs and outputs.

REST-based Web Services are pretty unique since they basically correspond to the various operations outlined above. We suggest a couple of different possible scenarios to address the challenges involved.

The application to be tested could have an implementation of strict REST-based Web Services. We'll elaborate on this further with an example, say a product catalog in which items can be created, viewed, deleted, and updated. This is pretty similar to the CRUD operations that would happen for the items in the underlying database.

• When a new item is created it is achieved by the PUT operation and it in turn creates a new URI like www.xyz.com/items/itemnew01
• When an item is to be viewed it is achieved by the GET operation by using the URI www.xyz.com/items/showitem?itemid=I001. This returns the URI www.xyz.com/items/itemnew01
• When an item has to be edited it is achieved by the POST operation by using the URI www.xyz.com/items/itemid=I001
• When an item has to be deleted it is achieved by the DELETE operation by using the URI www.xyz.com/items/itemid=I001 In such cases, while the former can be functionally tested by normal Web site testing mechanisms and we can use the normal Web testing tools that are available today.

This case is a bit more complex. Currently there aren't many tools available that can cater to this case.

Currently most testers build up their test cases and the input data in Excel sheets and verify it against the expected output XML. It probably makes sense to build a small tool that can do the following:

• Read the input parameters from the Excel sheet
• Invoke the appropriate service.
• Store the response provided by the service
• Compare the response against the expected response using tools like XMLDiff and provide reports

Hence non-functional testing is as significant as functional testing.

Security Testing
As previously mentioned, Web Services are software components that could be exposed over the Internet. In such scenarios security becomes a primary concern, Authentication and authorization concerns have to be tested thoroughly. In most scenarios for REST-based services, authentication is taken care of by using the HTTPS protocol. Most enterprises prefer to add an additional layer of authentication using either basic HTTP authentication or a complex token-based security mechanism. Basic authentication generally deals with access control lists. So testing should cover the aspects of "malicious testing," where the system is tested for controlled access by a specific and ratified set of users.

Another common security lapse in a REST Web Service is its vulnerability to SQL injection attacks. The similarity between the various HTTP requests to the CRUD database operations is indicated in the section above. It leaves a poorly designed Web Service highly vulnerable to SQL injection attacks. So the test strategy should contain test cases to cover this angle as well.

Amazon's private "shared-secret" token is a complex implementation of security that uses HMAC signing to show proof of possession of the public token. Although this kind of security implementation in REST Web Services diffuses the basic advantages of REST by reinventing the wheel, this security model exists. Testing these security models is a lot more complex and is more along the lines of testing a PKI infrastructure including the token issuing authority. Authorization is generally designed using filters in combination with the resource control lists. Although this is no different from standard authorization testing mechanisms, it's imperative that the testing strategy includes tests for authorization. This ensures that a user can access only what he's supposed to access.

Another common area of testing is the audit trail of requests and responses. Since most REST Web Services are GET-based or based on information retrieval rather than altering the state of the resource on the server, it becomes very important to audit these requests. So the testing strategy should cover the audit trail mechanisms of these Web Services too.

Regression Testing
A regression test is normally a cut-down version of a functional test. Its aim is to ensure that the Web Service is still working between builds or releases. It assumes that some area of functionality worked in the past and its job is to check that it still does. However, versioning is difficult to achieve transparently in REST-based Web Services since each resource has its own representation. So if a resource changed in different builds or releases, it would have a new URI. However, less strict versions of REST Web Services allow a pure XML over HTTP and POST/PUT to be used interchangeably. They also have a version field either as part of the payload or query string. This lets them introduce versioning from a deployment perspective so different versions of the same service can co-exist while using the same URI. Although this violates basic principles of Roy Fielding's theory, unfortunately there are a lot many implementations out there. Hence regression tests need to account for versioning test cases when multiple versions of the same service will co-exist in the application ecosystem.

Load Testing
The aim of load and stress testing is to find out how well your Web Service scales as the number of clients accessing it increases. The load testing is generally done to find the acceptable benchmarks associated with the Web Service and identify the Service Level Agreements (SLAs) that can be specified with this service. These benchmarks then become the SLAs of the service. Load tests for REST-based Web Services are primarily based on the same concept as load testing Web applications or Web sites. However there are subtle differences that have to be looked at carefully. REST Web Services are XML-based services. So while you get the flexibility of XML, you also get the baggage that comes with it. XML is bulky and takes up more bandwidth. Large XML messages tend to take up lots of memory. These XML messages also have to be parsed and using DOM parsers for large XML documents often leads to low memory situations. So it's imperative that the load testing strategy includes testing the performance of the services under an extreme load, particularly large XML payloads.

Conclusion
We have articulated the basic architectural Web Service paradigms, namely SOAP-style Web Services and REST-style Web Services. We looked at various aspects of testing REST-based Web Services. It helps to develop a testing strategy or methodology for REST-based services. We also mooted the idea of designing a tool based on available out-of-the-box components to test the services. We concluded that testing REST-based services is reasonably easier than SOAP-based services, where additional testing of standards, interoperability, and versioning is required. Testing is also required to see if the SOAP-based Web Services adhere to WS-I standards.

References