Scalability
The Transactions per Second (TPS) achieved by a Web Service is a good indicator of its scalability. The average TPS of the five test cases are available in Figure 6. From Figure 6, it's inferred that XFire with JibX was able to handle approximately 564 TPS (on average), about 14% more than its nearest competitor, Axis2 with JibX. Axis 1.4 scaled the worst, achieving only about 177 TPS on average.

Speed
The Average Transaction Response Time is a good indicator of the server-side speed of a particular combination, as it indicates how quickly the server could process the client's request. From the summary graph in Figure 7, it's inferred that XFire with JibX had the smallest average Transaction Response Time, followed by the two Axis2-based applications.

Throughput
Throughput is a measure of performance in terms of how much data the service was able to push to the client in one second. As this figure can be a function of both the number of requests the service can handle per second, as well as the speed with which it can process a client's request, it offers a very comprehensive view of the overall performance of the test cases. From Figure 8, it can be seen that XFire with JibX achieved the greatest throughput, indicating that it offers the best overall performance.

Ease of Use
•  Axis2 with ADB: Axis Data Binding is well integrated with the Axis2 Engine - development using this option is very quick - once the WSDL of the service is available, the WSDL2JAVA tool will generate all classes and stubs required for both the service and a client (except the business logic, of course!). The documentation on Apache's website is detailed and easy to follow.
•  Axis2 with JibX: Axis2 supports JibX, although, by the Axis2 team's own admission, the support seems to be incomplete. Axis2's WSDL2JAVA tool supports generating client- and server- side stubs for JibX binding, however, it doesn't support code-generation for a Web Service from a given schema. Although there are tools available to handle some of the code generation, when this was written, they were only mature enough to handle simple cases; nonetheless, they did provide a good starting point. However, the entire process seemed more cumbersome than using XFire.
•  XFire with JibX: XFire supports JibX binding reasonably well, although the current version of XFire WSGen tool can't generate client- and server-side stubs with JibX binding given a WSDL, it supports code generation for a Web Service from a schema. Moreover, it's fairly straightforward to write a client for such a service, even without a client-side stub. At the time of writing, the documentation wasn't very detailed, but had an easy-to-follow example. Overall, XFire with JibX seemed easier to use and develop than its Axis2 counterpart. However the auto-generated WSDL in XFire JibX combination remains one of the major drawbacks since it didn't confirm to WS-I when this article was written.
•  Axis 1.4: Axis 1.4 is currently one of the most widely used Web Services platforms on the enterprise Web Service landscape. The platform has good support for WSDL first development in terms of both server-side and client-side artifact generation. However server-side code generated using a WDL first development forces a user to populate business logic inside tool-generated code. The platform has limited support for directly exposing existing Java classes as Web Services. If one has to expose existing Java classes as Web Services, developers often generate WSDL from the Java class and subsequently generate server-side artifacts and deployment descriptors using WSDL2JAVA and subsequently map existing business classes to implementation classes generated by the tool.
•  XFire with JaxB: XFire provides good support for Jaxb binding, it comes with both schema first development and code first generation of a Web Service. However Web Services generated using the schema first approach show significantly better performance compared to code first generated Web Services. XFire comes with a WSGen tool that can be used to generate client- and server-side stubs from a WSDL and also supports a WSDL first development.

Conclusion
By analyzing the data collected in the course of this study, several conclusions were reached.

First, it seems obvious that the performance of third-generation SOAP engines is significantly better than that of its first- and second-generation counterparts.

It's concluded that the JibX Binding Framework offers the best performance and a great deal of flexibility through its binding definition file. However, defining bindings for complicated objects can become very cumbersome and time-consuming.

It's also concluded that XFire with JibX offers the best marshalling and un-marshalling performance, and is only marginally slower than Axis2 with JibX in round-trip performance, which can be largely attributed to the slower client-side processing by the XFire-based JibX client. XFire also seems to offer the best scalability, speed, and throughput among all test cases. Moreover, the ease of development using JibX binding with XFire over Axis2 with JibX makes this an attractive option.

However, it's worth noting that, if ease of use and development is of essence, Axis2 with ADB offers the best balance between ease and performance - Axis2 with ADB is extremely easy to use (however, it's less flexible than JibX) and offers very decent performance.

REFERENCES
•  Web Services @ W3C: www.w3.org/2002/ws/
•  "XMill - An Efficient Compressor for XML": www.liefke.com/hartmut/xmill/xmill.html
•  Paul Sandoz, Alessando Triglia & Santiago Pericas-Geertsen. "Fast Infoset":
http://java.sun.com/developer/technicalArticles/xml/fastinfoset/
•  Dr. Srinivas Padmanabhuni, Bijoy Majumdar, Ujval Mysore, Vikram Sitaram. "XML Binding Frameworks in the Context of Service-Oriented Architecture": http://soa.sys-con.com/read/114130.htm
•  WebServices - SOAP: http://xml.apache.org/soap
•  "Apache Axis": http://ws.apache.org/axis/
•  "Codehaus XFire": http://xfire.codehaus.org
•  "Apache Axis2/Java: Next Generation Web Services": http://ws.apache.org/axis2/
•  Why StAX? , http://java.sun.com/webservices/docs/1.6/tutorial/doc/SJSXP2.html
•  Common DOM API: http://java.sun.com/j2se/1.4.2/docs/guide/plugin/dom/index.html
•  API overview: http://java.sun.com/webservices/jaxp/dist/1.1/docs/tutorial/overview/3_apis.html
•  "Axis2/Java-JibX Integration with Axis2":
http://ws.apache.org/axis2/1_2/jibx/jibx-codegen-integration.html#intro
•  "Codehaus XFire- Client and Server Stub generation form WSDL":
http://xfire.codehaus.org/Client+and+Server+Stub+Generation+from+WSDL
(All Web sites were accessed on June 26, 2007.)