Feature

Digg This!

Web services is an emerging Internet programming paradigm that enables the remote invocation of software objects among heterogeneous systems over the Internet. There are two sides to Web services programming. On one side is the creation and deployment of Web service objects onto Internet servers; on the other side is the consumption of Web service objects in client programs or server applications. Due to the language-neutrality of the Web services paradigm, Web services can be created or consumed with different kinds of programming languages, including scripting languages such as Perl, Python, Tcl, and PHP. In this article, I'll explain and demonstrate the use of scripting to rapidly create and readily consume Web services over the Internet.

Web Services Background
Generally, the concept of "Web services" represents an enabling architecture for wide-area distributed computing among heterogeneous systems over the Internet. In this architecture, a program running on one machine can directly invoke methods in software objects running on other machines over the Internet. These software objects are called Web services. Specifically, a Web service is a collection of data and logic packaged as a single entity with defined interfaces to methods supporting data access and logic invocation. What does "Web services" mean to programmers? To server-side software developers, it means that business data and logic can be conveniently published to the network as a service for remote access and invocation by client-side programs. For example, a bank can publish a software object to supply real-time exchange-rate information to end users' financial software. To client-side application programmers, it means that additional functionality can be integrated into application programs by programmatically accessing data and invoking logic exposed on remote software objects. For example, an expense report tool can make use of the bank's exchange-rate object to convert monetary values from US$ to CDN$.

Figure 1 shows a conceptual model of the Web services architecture. It consists of the following entities:

• Service Producer: The business unit that builds software objects, publishes their interface, and deploys them as Web services on Internet servers
• Service Consumer: The end user who runs software programs that remotely access data and invoke logic from Web services launched by service producers
• Service Registry: The middleman that maintains an online directory for service producers to list and advertise their Web services and for service consumers to discover and look up Web services

Underlying the Web services architecture is a set of well-defined open standards that facilitate communications and interactions among the entities. These standards include:

• Web Services Definition Language (WSDL): An XML-based language, WSDL provides a structured way for service producers to describe the functionality, specify the interfaces, and identify the network address of their Web services. The WSDL description of a Web service contains all the information needed by a service consumer to invoke the methods supported by the Web service.
• Simple Object Access Protocol (SOAP): Also XML-based, SOAP is a messaging protocol designed to transport service requests and responses between a client program and a Web service object across the Internet.
• Universal Description, Discovery, and Integration (UDDI): UDDI defines a framework to enable the establishment of service registries to provide distributed directory service to the producers and consumers of Web services. It includes a common set of SOAP-based APIs to support registration and advertisement of Web services by service producers, and to facilitate the searching and lookup of Web services by service consumers.

In our example, the bank that builds and deploys the exchange-rate object acts as the service producer. It's responsible for generating the proper WSDL file and registering the Web service at an UDDI server - the service registry. The expense report tool that consumes the bank's exchange-rate object is the service consumer. The programmer of this tool queries the UDDI server to discover the service, retrieves the WSDL file, and extracts information required to invoke the Web service over SOAP.

Programming Web Services
There are two veins in the development of Web services software. First, service producers have to build and deploy software objects as Web services. We call this service creation. Second, service consumers have to write code in application programs to generate SOAP requests and process SOAP responses to and from Web services. We call this service consumption.

Web services is language-neutral, meaning that server objects can be written in any programming language, independent of the implementation of the client programs, and, vice versa, client programs can be written in any languages independent of the implementation of the server objects. So programmers of Web services have quite a few choices when it comes to selecting a programming language to create or consume Web services. On one end of the spectrum is the family of "low-level" languages such as C++ and Java. On the other end of the spectrum is the family of high-level scripting languages such as Perl, Python, Tcl, Ruby, and even PHP.

Unlike C++ and Java, scripting languages are either weakly typed or type-less, with little or no provision for complex data structures. Besides, scripts are usually interpreted rather than compiled. As such, scripting codes might not execute as fast as compiled codes. Yet scripting languages have the following advantages:

• Ease of use: The learning curve is gentle and scripts are easy to debug.
• Rapid implementation: Scripts take much less time to write and debug than compiled programs.
• Compact code: A single line of scripting code often performs the work of many lines of low-level codes.
• Portability: Scripts can often be run on different platforms with no modification or minimal porting effort.

Traditionally, scripting languages have been most useful for text processing, data extraction, and file manipulation, as well as automating administrative system tasks on UNIX platforms. With the advent of the Internet, a majority of the CGI codes on Web servers are implemented in scripting languages, taking advantage of their rich set of library functions for database access and system services. Statistically, more than 50% of the Internet's Web pages are dynamically generated by Perl scripts.

There are good reasons for adopting scripting languages in the creation and consumption of Web services. Most compellingly, Web services software can be programmed, tested, debugged, and prototyped much more quickly with scripting languages than with system languages. This helps shorten the development cycle and speeds up time-to-deploy. On the consumption side, scripting can be used as a macrolanguage to integrate and glue together Web services objects in application programs. On the creation side, there exists a huge installed base of server-side Web scripts that can be redeployed as Web services with minor changes. Finally, since scripting languages are often the "native-tongue" of the current generation of Internet programmers, it will be natural for them to use scripting when it comes to programming Web services.

Creating Web Services
The creation and deployment of script-based Web services normally entails the following steps:

Step 1: Create the implementation file
Each Web service will be implemented in a single file containing the source code for the data and methods supported by the service. In Perl, the implementation file is nothing more than a package of subroutines; in Python, it's simply a module of functions. To illustrate, let's consider the creation of a Web service called "Advogato_Query." This Web service provides information based on content retrieved from www.advogato.com, a journalistic Web site for open-source developers. Listings 1 and 2 show skeletons of "Advogato_Query" written in Perl and in Python. (The code for this article can be found on the Web site at www.sys-con.com/webservices/sourcec.cfm.) Three methods are supported by "Advogato_Query." They are:

1. GetCurrentArticles(): Returns the titles of the top five current articles
2. GetArticleDetails(article_title): Returns the content of an article titled article_title
3. GetArticleList(start_date, end_date): Returns the list of articles posted between start_date and end_date, inclusively

Step 2: Provide the interface definition Because scripting languages are weakly typed, it's necessary to accompany a Web service script with an interface definition. Otherwise, the SOAP server wouldn't be able to properly and accurately perform checking, matching, and conversion of data types on the input/output parameters during invocation of the script. The interface definition should contain the names of all the exposed methods, plus the names and types of the input and output arguments for each method. Other SOAP-related metadata, such as namespace information, may be declared as well. Currently, there's no standard interface definition language (IDL) for script-based objects. As an example, consider the IDL used by the SOAP server built in PerlEx, a Perl Accelerator available from ActiveState Corp. PerlEx's SOAP IDL, called Universal Interface Specification (UIS), has a C-like syntax. Using UIS, you can define the three interfaces of the "Advogato_Query" Web service as follows:

struct dateStruct {
int32 year;
int32 month;
int32 day;
}

def str_array = ustring[];

interface Advogato_Query {
static str_array GetCurrentArticles();
static ustring GetArticleDetails( ustring article_title );
static str_array GetArticleList( dateStruct start_date,
dateStruct end_date );
};

For the PerlEx SOAP server, the interface definition is either prefixed to the implementation file of the Web service or supplied in a separate description file.

Step 3: Deploy the Web service
This refers to the installation of the Web service script on a SOAP server. The procedures are server-specific. In the case of the PerlEx SOAP server, a Web service is deployed by manually copying its implementation file (and interface definition file, if applicable) to a designated directory on the server. Other systems might support remote deployment such that scripts can be downloaded to the SOAP server over the Web.

Step 4: Generate the WSDL file
A WSDL file should be created for each deployed Web service script. It can be written manually by using a text editor or XML editor. Software tools also exist that can be used to automatically generate a large part of the WSDL file based on the script's interface definition. The WSDL file for "Advogato_ Query" shown in Listing 3 indicates that the service is hosted at webservices.activestate.com.

Step 5: Register the Web service
To make a newly deployed Web service known to the community, the service producer should register it with an online Web service directory. Currently, several such directories are available, including www.xmethods.com and www.webservices.org. Service registration at these portals is accomplished by completing and submitting an online registration form. For example, www.xmethods's.com registration form is available from www.xmethods. com/service. It requires the registrant to provide certain information, including service description, SOAP endpoint URL, method names, and WSDL URL. In the future, when UDDI becomes standardized and widely deployed, new Web service listings will be entered into UDDI registries via a standard process.

This completes the process of service creation using scripting languages. Let's see how a deployed Web service can be consumed by scripting codes over the Internet.

Consuming Web Services
As depicted in Figure 2, Web services can be consumed by (1) client-side applications in a two-tier architecture or (2) server-side programs in a three-tier architecture. For instance, the exchange-rate service mentioned earlier can be utilized in GUI applications designed to do financial calculations, or it can be invoked by CGI programs running on online merchants' Web servers.

For the purpose of our discussion, suppose we want to add multilanguage capability into a Perl- or Python-based lightweight e-mail client (e.g., Pail [http://pail.sourceforge.net/], Pmail [www.scottbender.net/pmail]) so that it can display the e-mail message in a foreign language. We can take advantage of Web services to realize this functionality.

Step 1: Discover Web services
First we have to find and identify a Web service that meets our application's requirements. One way to do service discovery is to simply do a keyword-based search using a general-purpose search engine such as www.google.com. A better way is to browse through the listings or query the directories maintained by several Web services portals, including www.xmethods.com and www.webservices.org. In the future, when UDDI infrastructures become available, service discovery can be done more systematically and efficiently by using a UDDI browser that directly queries UDDI registries. In fact, several experimental UDDI registries have already been launched for testing and trial purposes (two examples: < a href = "http://test.uddi.microsoft.com" target="_new">http://test.uddi.microsoft.com and www-3.ibm.com/services/uddi/testregistry/protect/registry.html).

For now, if we point our Web browser to www.xmethods.com, it will display a table of Web services available for public use. Clicking on any one of the entries in the table will bring us to the Web service's main page, which contains a description of the service as well as a pointer to the service's WSDL file. One of the services listed in the table is called "BabelFish." Driven by AltaVista's BabelFish translation engine, this Web service supports a method that translates an input text string from one language to another. It's exactly what our multilanguage e-mail client can use!

Step 2: Extract call parameters The next step is to determine the programming parameters needed to invoke the Web service. These parameters are:

• Method names
• Name and type of the input/output argument for each methods
• SOAP endpoint URL
• SOAP namespace
• Soapaction

Usually they are specified on the Web service's main page. If not, they can be extracted directly from the WSDL description of the Web service. For example, from the WSDL file of "BabelFish" (www.xmethods.net/sd/2001/BabelFishService.wsdl), we can obtain the following information:

• Method Name: BabelFish
• Input 1: name = translationmode, type = string
• Input 2: name = sourcedata, type = string
• Output: name = return, type string
• SOAP endpoint URL = http://services .xmethods.net:80/perl/soaplite.cgi
• SOAP namespace = urn:xmethodsBabel Fish
• Soapaction = "urn:xmethodsBabelFish# BabelFish"

Step 3: Invoke Web services
Now we're ready to do the coding. There are three approaches to programmatically consuming Web services in scripting languages. In the first approach, native code is written to assemble the SOAP requests and transmit the request to the SOAP server and then to receive the SOAP responses and parse the responses to retrieve the results. This is called "protocol-level programming." For example, we can write a function, called translate(), to invoke "BabelFish" in Perl as seen in Listing 4. To do the same in Python, see Listing 5.

Coding at this level requires the programmer to have a good know-how of the SOAP specifications, and be proficient in network programming. Besides, the process can be tedious and error-prone, especially when dealing with array and complex data types.

The second approach is called library-assisted programming. It relies on the use of an external SOAP library to handle the low-level tasks of marshalling and unmarshalling SOAP calls. Table 1 lists some of the publicly available SOAP libraries for various scripting languages. Using the SOAP::Lite package, we can rewrite the translate() subroutine in Perl as:

sub translate
{
my $mode = shift;
my $text = shift;

use SOAP::Lite
on_action => sub {sprintf '"urn:xmethodsBabelFish#BabelFish"'},
uri => 'urn:xmethodsBabelFish',
proxy => 'http://services.xmethods.net:80/perl/soaplite.cgi';

my $dat1 = SOAP::Data -> name('translationmode' => $mode);
my $dat2 = SOAP::Data
->name('sourcedata' => $text);
my $interface = new SOAP::Lite;
my $translated_text = $interface-> BabelFish($dat1, $dat2)->result;

return $translated_text;
}

Similarly, we may use the soaplib.py library to invoke the "BabelFish" service in Python (see Listing 6).

Library-assisted programming saves programmers from the trouble of assembling and disassembling SOAP messages. The task of invoking a Web service method is reduced to making several library calls.

Further simplification is achieved with the third approach, proxy-assisted programming. In this approach, Web services are invoked with the help of an intelligent proxy module that has built-in WSDL parsing capability. Effectively, the proxy module provides a wrapper layer above the SOAP library to completely hide the under-the-hood SOAP mechanics from the programmer. It also supports an object-oriented syntax so that programs can invoke remote Web services as if they are local objects. ActiveState's "WebService" module, available soon for Perl and Python, is an example of an intelligent proxy module for Web services invocation. Also in this category is the PHP-based SOAP4X toolkit, downloadable from http://dietrich.ganx4.com/soapx4/.

To invoke a Web service, a program only needs to supply a WSDL pointer to the proxy module's object factory function. This will create a proxy object that automatically inherits the interfaces of the Web service, and is directly callable by the program based on the methods and arguments specified in the WSDL file. For example, using ActiveState's forthcoming WebService module, we can implement the translate() subroutine in Perl as seen in Listing 7. In Python, the code will be written as shown in Listing 8.

Besides ease of use and code compactness, proxy modules may provide built-in support for call timeout, asynchronous invocation, and exception handling. When a program calls a Web service method asynchronously, the call returns immediately so that the program can perform other functions while the SOAP transaction is processed in the background. When the SOAP response arrives, the program will be notified to retrieve the result. This is especially useful in multitasking or GUI-based programs that cannot tolerate blocking. It saves the programmers from the unenviable task of spawning separate threads or forking separate processes to handle Web services requests and responses.

Putting It Together
To tie all the concepts together, let's consider the creation of a compound Web service. Unlike the basic Web services we've examined so far, a compound Web service is itself a consumer of other Web services (see Figure 3).

Our compound Web service is called "Advogato_In_French." It supports a single method that returns a list of Advogato headlines in translated text. Listing 9 shows the Perl implementation of this Web service.

At the top of the script is the UIS-based interface definition of the service. The code makes use of ActiveState's WebService proxy module to invoke two Web Services. From the first Web service ("Advogato_Query"), it obtains the Advogato headlines in English. The second Web service ("BabelFish") is called upon to translate the headlines from English into French. Shown below is a simple Python script written to call this Perl-based Web service.

import WebService
news = [];

advogato_in_french_service = WebService.new(
"http://webservices.activestate.com/wsdl/advogato_in_french.wsdl" )
news = advogato_in_french_service.GetArticleList()

for x in news: print x

Summary
In this article, we've walked through five easy steps leading to the deployment and publishing of scripts as Web services. We've also reviewed the three different approaches to invoking Web services in scripting codes. Especially simple and elegant is the method of "proxy-assisted programming." In the near future, new features in IDEs, such as Komodo, will further simplify the process of creating and consuming Web services with scripting languages. As the need for rapid development and prototyping rises, scripting will play a prominent role in both server-side and client-side Web services programming.