Welcome!

Microservices Expo Authors: Liz McMillan, Zakia Bouachraoui, Elizabeth White, Pat Romanski, Yeshim Deniz

Related Topics: Microservices Expo

Microservices Expo: Article

Semantic Discovery for Web Services

Semantic Discovery for Web Services

The Web services vision of loosely coupled interaction between components, programs, and applications is already beginning to create impressive efficiencies of scale in business integration. The notion of a Web service registry such as UDDI is helping to turn this vision into a reality. Despite this, however, the comprehensive adoption of a truly dynamic discovery of Web services may still be delayed.

The Challenge of Dynamic Web Services Discovery and Usage
Currently, the majority of Web services are developed for internal enterprise use. Most existing projects are experimental, while a few early applications already in use provide highly focused functionality for solving specific business and technology problems. For these applications, where all users are within the same enterprise or project team, manual discovery and coding of clients is the most efficient approach.

Web services appear poised to become more widely adopted in coming years, allowing much broader intra- and interenterprise integration. Increasingly, developers will require automated systems for service discovery, enabling further Web services interaction with even less human effort. UDDI exists precisely for this reason. However, unless the service's client knows the exact form and meaning of a service's WSDL in advance, the combination of UDDI with WSDL and coarse-grained business descriptions is not enough to allow fully automated service discovery and usage.

Semantic Confusion
When semantics for input and output parameters are weakly defined, increased numbers of Web services in a given functional domain increase the difficulty of accessing each service. Semantic inconsistency can occur in several ways:

  • Poorly defined semantics
  • Shared syntax, different semantics
  • Shared semantics, different syntax
These problems often combine and result in overlapping and conflicting syntax and semantics.

Poorly Defined Semantics
Today service semantics are often left unstated and are guessed at or interchanged by humans. Without sufficient description, a service is virtually useless to anyone but the original developer.

Let's look at a simple example, a tax calculation service, which has a single, synchronous operation. This operation receives an input message with floating-point amount and string zip, and an output message with floating-point rate and integer tax (see Figure 1).

 

This is a very simple Web service, yet confusion arises without a formal specification of semantics. For example, what units go into the amount element? It happens to be U.S. dollars. While this might be obvious to the service provider who intended this service for U.S. consumption, potential service clients from other countries need to at least be aware of the expectations of this service.

Shared Syntax, Different Semantics
Unspecified differences in semantics can cause confusion between services. This causes problems when a client is coded to use one service, but tries to use another service with different semantics. As long as the syntax (as represented, for example, by WSDL message definitions) is the same, the call to the service will succeed, but it will return unexpected results.

Two tax calculation services may receive similar inputs and respond with similar outputs, but one calculates income tax while the other calculates sales tax.

The client must be aware of the precise functionality each Web service provides before calling it. Even where WSDL is available, semantics are not expressed precisely in the service interface alone, but rather must be learned by the developer working on the client side.

Shared Semantics, Different Syntax
Likewise, two Web services may have shared semantics (functionality) but different syntax. For instance, two services may calculate sales tax; in the first service, rate is a multiplier - a 5.5% rate would be given as 0.55 - while in the other, it is a percentage, with the same value given as 5.5%. Multiplier and percentage are semantically identical, as long as a factor of 100 can be taken into account in transforming one to the other.

The zip element, which allows tax to be calculated based on the legal jurisdiction, is a five-digit U.S. Postal Service ZIP code in one service. Yet a client application may have been coded to work with another service that has the same functionality but instead expects a nine-digit ZIP code. We know that we can derive the short code by truncating the long one, but software does not know this unless it is formally encoded.

Where services have similar functionality but different syntax, it's often quite easy to convert between the two. However, this is only possible if the meaning of each input and output parameter is understood precisely.

A Semantic Approach to Service Discovery
Solving these problems requires a semantic approach to the dynamic discovery and interoperability of new Web services, building on lessons learned as part of the implementation of the Semantic Web, a visionary idea of dynamic interoperability of computer systems over the Internet. Tim Berners-Lee, inventor of the World Wide Web, has been promoting his vision for several years (see Resources). The key ingredient of the semantic approach is the formal capture of the Web service's interface by reference to an agreed-upon business-oriented vocabulary (semantic model), described in more detail later.

Although most writings on the Semantic Web do not explicitly mention SOAP, WSDL, and UDDI, the vision of the Semantic Web is best implemented with Web services. In this article, besides presenting a practical solution to a practical problem faced by Web service developers today, we hope to show how Web services fit into the vision of the Semantic Web.

The architecture for Web service discovery through UDDI, or other Web services registries such as ebXML, breaks down interaction stages into distinct roles, including Service Provider, Registry, and Client. To add semantics to these interactions, no change is required for the UDDI Registry, so roles in the semantic approach can be broken down into Provider and Client. Of the two, the Provider has the greatest burden, providing more detailed information about the meaning of the service. The developer on the client side must also change his or her workflow to use the semantic approach. No changes are needed to the discovery process are needed, but with the use of a semantic description, the client can discover the service semantically, and then apply transformations to adapt the interface of the service to the interface expected by using existing Client software.

Despite these changes, the semantic approach simply requires the formalization of existing development steps; all the semantic steps must in any case be done, albeit informally, in spreadsheets or word-processor documents. For a service to be used, the Provider must somehow tell the Client what the service "means," and the Client must understand what it means and adapt the expectations of the client software to the actual Service interface.

The Provider's Role:
Semantic Registration

The Service Provider
Development Process

  • Use an ontology to model the real-world concepts related to the service's functionality.
  • Create a WSDL document for the Web service.
  • Map elements of the WSDL messages to semantic concepts, saving mappings in RDF.
  • Register the model, WSDL, and mappings to the UDDI Registry.

    The role of the Service Provider is to formally represent the service's semantics in a machine-readable way in a UDDI Registry.

    Model Real-World Concepts
    To enable semantic queries, a Service Provider starts by modeling the semantics of the real-world concepts associated with the service. Such a model is based on the principles of ontology, the science of meaning. A semantic model based on ontology includes:

  • Classes: Sets of real-life entities with some characteristics in common, and the generalization/specialization (inheritance) relationships between them
  • Properties: Relate these classes to each other
  • Business Rules: Indicate constraints on and relationships between properties

    Part of the model for our example is depicted in Figure 2, while a fuller version is available as Listing 1 (for space reasons the code for Listing 1 is online at www.sys-con.com/webservices/sourcec.cfm).

     

    OWL (Web Ontology Language) is a markup language semantics that enables ontology sharing via the Web and is the W3C's XML-based working draft for a standard (see Listing 1). It is an updated version of DAML+OIL (DARPA Agent Markup Language + Ontology Inference Language) and continues much of the work done previously on RDF (Resource Description Framework). The OWL representation can be registered easily in a standard UDDI Registry as a tModel, as we will explain later.

    The semantic model represents the meaning of a service's functionality. Achieving consensus in ontology can be difficult. But even before standards emerge for the ontologies of specific vertical business areas, the Service Provider can register a semantic model, which the Client can then inspect and use for look-up. Where heterogeneous semantic models must be reconciled, they can be integrated by merging them into a larger model that includes synonymous classes and properties.

    With a formal semantic model in place, it doesn't matter if the service works with dollars or euros, percentages or multipliers, as long as the service's functionality is encoded in the ontology. With formalisms in place, such syntactic differences are easily overcome through XSLT transformations that convert from one syntax to another.

    The ontological model does not yet represent the service itself, since specific input and output parameters are not encoded. However, it does represent the critical concepts needed to understand the service's functionality.

    Create a WSDL Document for the Web Services
    Next, the Provider creates the WSDL, which defines the Service's syntax by specifying the structure of the input and output messages, along with aspects of the service's runtime bindings. Creating a WSDL document and registering it to a UDDI Registry is a best practice even if the semantic approach is not used. A WSDL document for this service is shown in Listing 2.

    Map Elements from the WSDL Messages to Semantic Concepts
    The next stage is to express the meaning of the WSDL by mapping the operations, along with the schemas of the WSDL input and output messages or signatures, to the semantic model.

    In some cases, we map a schema element into an indirect property, which is a series of several properties, to express semantic concepts precisely.

    In our example, we map the schema element rate in the output structure to the rate property of class Tax, linked to the multiplier property of class Rate. In this way, we have encoded the precise meaning of this output parameter, stating that the rate is expressed as a multiplier rather than as a percentage. A Client looking for a percentage can later transform multiplier to percentage by using the business rule formally encoded in the semantic model: multiplier = percentage/100.

    Likewise, we map the zip schema element to the jurisdiction property of class Tax linked to the zipCodes property of class Jurisdiction, linked yet again with fiveDigit property of class ZipCode. In this way, we state that the tax is to be calculated in a given jurisdiction (city or state), that the jurisdiction is characterized by its ZIP codes, and that the ZIP code is expressed in its five-digit form.

    Figure 3 shows a subset of these mappings graphically, while Table 1 lists the mappings in full.

     

     

    Mappings can be expressed in an XML document formatted in the RDF. RDF is built of triplets expressing a relationship of the form subject-predicate-object. For example, we would express a mapping as the following triplet: 1) "element rate", 2) "maps to", and 3) "property rate of class Tax".

    Register the Model, WSDL, and Mappings to the UDDI Registry
    The last stage is to register these elements to a UDDI Registry. The concepts in the WSDL, the semantic model, and the mappings between them fit quite naturally into UDDI as tModels. In fact, it is already considered a best practice to store WSDL in this way; the other artifacts are stored in the same way (see Figure 4).

     

    The semantic approach to UDDI requires registering more artifacts to the registry than the nonsemantic approach. Fortunately, however, the UDDI object model is flexible enough to allow this sort of usage since the UDDI API has been designed with batch-processing capabilities, allowing the registering or querying of multiple tModels with a single remote call.

    The Client's Role:
    Semantic Discovery

  • Identify the required functionality.
  • Query UDDI, locate service, and retrieve all semantic and interface descriptions.
  • Create transformations to use the service from the Client.
  • Access the Web service using the WSDL document.

    The Web service is now ready for semantic discovery. The Client needs to discover a service that meets the required business needs and then calls this service.

    Identify the Required Functionality
    To identify the required service functionality, the Client first discovers the agreed-upon semantic model using UDDI and loads it over standard HTTP. The techniques to do so resemble the use of UDDI for finding any other resource. The Client simply locates the OWL document representing the semantic model by finding the appropriate tModel based on industry sector and general taxonomies.

    Even when not standardized, the semantic model is general enough to cover a wide variety of services for a given industry vertical or horizontal sector. For example, the model depicted above (see Figure 2 and Listing 1) would be part of a much larger industry-standard model for financial services. Where multiple "standard" models exist, they can easily be linked together through ontological properties and inheritance in order to form a larger, more inclusive model.

    In the non-semantic approach, the industry definitions and taxonomies jointly express a kind of semantics. But the Client must still know exactly what service is needed and exactly what input and output messages to look for in the registry. In the semantic approach, the Client first accesses the general service category through taxonomies; then, by referencing the functionality of the service in meaningful semantic classes such as SalesTax, ZipCode, and Jurisdiction, the Client finds the service that approximates its precise needs.

    Query UDDI, Locate Service, and Retrieve all Semantic and Interface Descriptions
    Having identified the relevant ontological concepts in the semantic model, the Client now navigates the mappings that link the model to the required WSDL files. The Client discovers a WSDL file with a function whose input and output schemas are mapped to the input and output concepts.

    In our example, a suitable Web service is any WSDL with the correct semantics:

  • The input-message schema includes elements expressing the ontological properties amount and zip.
  • The output-message schema includes elements expressing the ontological properties tax and rate.

    The Client identifies these semantic values and uses the mappings to find identifiers for the services that provide the functionality. The Client can then directly discover these services through UDDI.

    Create Transformations to Use the Service from the Client
    The Client may have been coded to pass input and expect output with a certain syntax, while the service expects input and returns output with another syntax. For example, the Client may pass a ZIP code as a nine-digit string, while the service expects a five-digit string; likewise, the service may return a tax rate as a multiplier such as 0.055, while the Client expects a percentage such as 5.5.

    This problem can be resolved through semantics. In a simpler but manual approach, the developer can code the transformation from one to the other using XSLT. With semantics in hand, the meaning of the parameters is clear and such transformation code is easy to develop. In a more advanced and dynamic approach, semantically based software can automatically generate XSLT to convert from the XML of one schema to the XML of another schema, based on the shared semantics of the schemas. This is suitable for WSDL operations whose parameters are specified as "literal." A third approach combines semantics with the flexibility of DII (Dynamic Invocation Interface): the client adapts its input messages to the needs of the service's interface at runtime. This is suitable to parameters specified as "encoded."

    Access the Web Service
    With a WSDL document available to fully characterize the Web service and transformations in place to adapt between the Client and the Server, the Client can now call on the Web service directly, just as with any WSDL.

    Conclusion
    In the context of UDDI, WSDL alone cannot deliver the vision of loosely coupled, dynamically interoperating Web services, and so cannot ensure that the current growth in Web services adoption will continue. Without a semantic infrastructure to form the backbone of the UDDI Registry, semantic inconsistency will prevent the large-scale adoption of active Web services adoption. The ongoing decentralization of Web services will make this problem increasingly acute in the coming few years. Formalization of the semantics in Web services is a step toward automated UDDI discovery and the fulfillment of the Semantic Web vision.

    Resources

  • UDDI for Python (UDDI4Py): www.alphaworks.ibm.com/tech/uddi4py
  • Microsoft UDDI Software Development Kit: http://uddi.microsoft.com/developer/default.aspx
  • Java API for XML Registries (JAXR): http://java.sun.com/xml/jaxr/
  • UDDI Homepage: www.uddi.org
  • UDDI4J (Open Source UDDI Client API in Java): www.uddi4j.org
  • "Using WSDL in a UDDI Registry: www.oasis-open.org/committees/uddi-spec/doc/bp/uddi-spec -tc-bp-using-wsdl-v108-20021110.htm
  • Resource Description Framework: www.w3.org/RDF
  • "Semantic Web Activity (W3C)": www.w3.org/2001/sw/
  • Web Ontology Language (OWL) Reference: www.w3.org/TR/owl-ref
  • Berners-Lee, Tim; Hendler, James . and Lassila, Ora. "The Semantic Web." www.sciam.com/2001/0501issue/0501berners-lee.html
  • Grove, Andy. "Understanding UDDI tModels and Taxonomies," Web Services Journal (Vol. 1, issue 2; November 2001).
  • Januszewski, Karsten. "UDDI and WSDL: Natural Companions," Web services Journal (Vol. 1, issue 3; December 2001).
  • Ogbuji, Uche. "Supercharging WSDL with RDF." www-106.ibm.com/developerworks/library/ws-rdf/?dwzone=ws

    SIDE BAR

    WSDL Message Syntax

    WSDL allows SOAP input and output to be specified in two different ways: as "literal" values or "encoded" as function parameters. In the former approach, XML documents are passed; the latter approach resembles function calls as seen in programming languages. Either the "literal" or "encoded" approach can be used with the technique described here. Examples are given with the "literal" approach.

  • More Stories By Joshua Fox

    Joshua Fox is a senior software architect at Unicorn Solutions (www.unicorn.com), developing semantic information management systems for the enterprise. He has experience developing large-scale clustered Java systems for Internet collaboration and multimedia delivery, and has published and lectured widely. He can be reached at [email protected]

    More Stories By Joram Borenstein

    Joram Borenstein is a senior marketing manager at Unicorn Solutions (www.unicorn.com), working on data semantics technologies for enterprises worldwide. His previous experience includes managing the rollout of content management software platforms. He has published and lectured extensively. He can be reached at [email protected]

    Comments (0)

    Share your thoughts on this story.

    Add your comment
    You must be signed in to add a comment. Sign-in | Register

    In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.


    Microservices Articles
    The now mainstream platform changes stemming from the first Internet boom brought many changes but didn’t really change the basic relationship between servers and the applications running on them. In fact, that was sort of the point. In his session at 18th Cloud Expo, Gordon Haff, senior cloud strategy marketing and evangelism manager at Red Hat, will discuss how today’s workloads require a new model and a new platform for development and execution. The platform must handle a wide range of rec...
    When building large, cloud-based applications that operate at a high scale, it’s important to maintain a high availability and resilience to failures. In order to do that, you must be tolerant of failures, even in light of failures in other areas of your application. “Fly two mistakes high” is an old adage in the radio control airplane hobby. It means, fly high enough so that if you make a mistake, you can continue flying with room to still make mistakes. In his session at 18th Cloud Expo, Lee A...
    In his general session at 19th Cloud Expo, Manish Dixit, VP of Product and Engineering at Dice, discussed how Dice leverages data insights and tools to help both tech professionals and recruiters better understand how skills relate to each other and which skills are in high demand using interactive visualizations and salary indicator tools to maximize earning potential. Manish Dixit is VP of Product and Engineering at Dice. As the leader of the Product, Engineering and Data Sciences team at D...
    Lori MacVittie is a subject matter expert on emerging technology responsible for outbound evangelism across F5's entire product suite. MacVittie has extensive development and technical architecture experience in both high-tech and enterprise organizations, in addition to network and systems administration expertise. Prior to joining F5, MacVittie was an award-winning technology editor at Network Computing Magazine where she evaluated and tested application-focused technologies including app secu...
    Containers and Kubernetes allow for code portability across on-premise VMs, bare metal, or multiple cloud provider environments. Yet, despite this portability promise, developers may include configuration and application definitions that constrain or even eliminate application portability. In this session we'll describe best practices for "configuration as code" in a Kubernetes environment. We will demonstrate how a properly constructed containerized app can be deployed to both Amazon and Azure ...
    Modern software design has fundamentally changed how we manage applications, causing many to turn to containers as the new virtual machine for resource management. As container adoption grows beyond stateless applications to stateful workloads, the need for persistent storage is foundational - something customers routinely cite as a top pain point. In his session at @DevOpsSummit at 21st Cloud Expo, Bill Borsari, Head of Systems Engineering at Datera, explored how organizations can reap the bene...
    Using new techniques of information modeling, indexing, and processing, new cloud-based systems can support cloud-based workloads previously not possible for high-throughput insurance, banking, and case-based applications. In his session at 18th Cloud Expo, John Newton, CTO, Founder and Chairman of Alfresco, described how to scale cloud-based content management repositories to store, manage, and retrieve billions of documents and related information with fast and linear scalability. He addresse...
    SYS-CON Events announced today that DatacenterDynamics has been named “Media Sponsor” of SYS-CON's 18th International Cloud Expo, which will take place on June 7–9, 2016, at the Javits Center in New York City, NY. DatacenterDynamics is a brand of DCD Group, a global B2B media and publishing company that develops products to help senior professionals in the world's most ICT dependent organizations make risk-based infrastructure and capacity decisions.
    Discussions of cloud computing have evolved in recent years from a focus on specific types of cloud, to a world of hybrid cloud, and to a world dominated by the APIs that make today's multi-cloud environments and hybrid clouds possible. In this Power Panel at 17th Cloud Expo, moderated by Conference Chair Roger Strukhoff, panelists addressed the importance of customers being able to use the specific technologies they need, through environments and ecosystems that expose their APIs to make true ...
    In his keynote at 19th Cloud Expo, Sheng Liang, co-founder and CEO of Rancher Labs, discussed the technological advances and new business opportunities created by the rapid adoption of containers. With the success of Amazon Web Services (AWS) and various open source technologies used to build private clouds, cloud computing has become an essential component of IT strategy. However, users continue to face challenges in implementing clouds, as older technologies evolve and newer ones like Docker c...