Microservices Expo Authors: Liz McMillan, Pat Romanski, Carmen Gonzalez, Elizabeth White, Jason Bloomberg

Related Topics: Microservices Expo

Microservices Expo: Article

Semantic Discovery for Web Services

Semantic Discovery for Web Services

The Web services vision of loosely coupled interaction between components, programs, and applications is already beginning to create impressive efficiencies of scale in business integration. The notion of a Web service registry such as UDDI is helping to turn this vision into a reality. Despite this, however, the comprehensive adoption of a truly dynamic discovery of Web services may still be delayed.

The Challenge of Dynamic Web Services Discovery and Usage
Currently, the majority of Web services are developed for internal enterprise use. Most existing projects are experimental, while a few early applications already in use provide highly focused functionality for solving specific business and technology problems. For these applications, where all users are within the same enterprise or project team, manual discovery and coding of clients is the most efficient approach.

Web services appear poised to become more widely adopted in coming years, allowing much broader intra- and interenterprise integration. Increasingly, developers will require automated systems for service discovery, enabling further Web services interaction with even less human effort. UDDI exists precisely for this reason. However, unless the service's client knows the exact form and meaning of a service's WSDL in advance, the combination of UDDI with WSDL and coarse-grained business descriptions is not enough to allow fully automated service discovery and usage.

Semantic Confusion
When semantics for input and output parameters are weakly defined, increased numbers of Web services in a given functional domain increase the difficulty of accessing each service. Semantic inconsistency can occur in several ways:

  • Poorly defined semantics
  • Shared syntax, different semantics
  • Shared semantics, different syntax
These problems often combine and result in overlapping and conflicting syntax and semantics.

Poorly Defined Semantics
Today service semantics are often left unstated and are guessed at or interchanged by humans. Without sufficient description, a service is virtually useless to anyone but the original developer.

Let's look at a simple example, a tax calculation service, which has a single, synchronous operation. This operation receives an input message with floating-point amount and string zip, and an output message with floating-point rate and integer tax (see Figure 1).


This is a very simple Web service, yet confusion arises without a formal specification of semantics. For example, what units go into the amount element? It happens to be U.S. dollars. While this might be obvious to the service provider who intended this service for U.S. consumption, potential service clients from other countries need to at least be aware of the expectations of this service.

Shared Syntax, Different Semantics
Unspecified differences in semantics can cause confusion between services. This causes problems when a client is coded to use one service, but tries to use another service with different semantics. As long as the syntax (as represented, for example, by WSDL message definitions) is the same, the call to the service will succeed, but it will return unexpected results.

Two tax calculation services may receive similar inputs and respond with similar outputs, but one calculates income tax while the other calculates sales tax.

The client must be aware of the precise functionality each Web service provides before calling it. Even where WSDL is available, semantics are not expressed precisely in the service interface alone, but rather must be learned by the developer working on the client side.

Shared Semantics, Different Syntax
Likewise, two Web services may have shared semantics (functionality) but different syntax. For instance, two services may calculate sales tax; in the first service, rate is a multiplier - a 5.5% rate would be given as 0.55 - while in the other, it is a percentage, with the same value given as 5.5%. Multiplier and percentage are semantically identical, as long as a factor of 100 can be taken into account in transforming one to the other.

The zip element, which allows tax to be calculated based on the legal jurisdiction, is a five-digit U.S. Postal Service ZIP code in one service. Yet a client application may have been coded to work with another service that has the same functionality but instead expects a nine-digit ZIP code. We know that we can derive the short code by truncating the long one, but software does not know this unless it is formally encoded.

Where services have similar functionality but different syntax, it's often quite easy to convert between the two. However, this is only possible if the meaning of each input and output parameter is understood precisely.

A Semantic Approach to Service Discovery
Solving these problems requires a semantic approach to the dynamic discovery and interoperability of new Web services, building on lessons learned as part of the implementation of the Semantic Web, a visionary idea of dynamic interoperability of computer systems over the Internet. Tim Berners-Lee, inventor of the World Wide Web, has been promoting his vision for several years (see Resources). The key ingredient of the semantic approach is the formal capture of the Web service's interface by reference to an agreed-upon business-oriented vocabulary (semantic model), described in more detail later.

Although most writings on the Semantic Web do not explicitly mention SOAP, WSDL, and UDDI, the vision of the Semantic Web is best implemented with Web services. In this article, besides presenting a practical solution to a practical problem faced by Web service developers today, we hope to show how Web services fit into the vision of the Semantic Web.

The architecture for Web service discovery through UDDI, or other Web services registries such as ebXML, breaks down interaction stages into distinct roles, including Service Provider, Registry, and Client. To add semantics to these interactions, no change is required for the UDDI Registry, so roles in the semantic approach can be broken down into Provider and Client. Of the two, the Provider has the greatest burden, providing more detailed information about the meaning of the service. The developer on the client side must also change his or her workflow to use the semantic approach. No changes are needed to the discovery process are needed, but with the use of a semantic description, the client can discover the service semantically, and then apply transformations to adapt the interface of the service to the interface expected by using existing Client software.

Despite these changes, the semantic approach simply requires the formalization of existing development steps; all the semantic steps must in any case be done, albeit informally, in spreadsheets or word-processor documents. For a service to be used, the Provider must somehow tell the Client what the service "means," and the Client must understand what it means and adapt the expectations of the client software to the actual Service interface.

The Provider's Role:
Semantic Registration

The Service Provider
Development Process

  • Use an ontology to model the real-world concepts related to the service's functionality.
  • Create a WSDL document for the Web service.
  • Map elements of the WSDL messages to semantic concepts, saving mappings in RDF.
  • Register the model, WSDL, and mappings to the UDDI Registry.

    The role of the Service Provider is to formally represent the service's semantics in a machine-readable way in a UDDI Registry.

    Model Real-World Concepts
    To enable semantic queries, a Service Provider starts by modeling the semantics of the real-world concepts associated with the service. Such a model is based on the principles of ontology, the science of meaning. A semantic model based on ontology includes:

  • Classes: Sets of real-life entities with some characteristics in common, and the generalization/specialization (inheritance) relationships between them
  • Properties: Relate these classes to each other
  • Business Rules: Indicate constraints on and relationships between properties

    Part of the model for our example is depicted in Figure 2, while a fuller version is available as Listing 1 (for space reasons the code for Listing 1 is online at www.sys-con.com/webservices/sourcec.cfm).


    OWL (Web Ontology Language) is a markup language semantics that enables ontology sharing via the Web and is the W3C's XML-based working draft for a standard (see Listing 1). It is an updated version of DAML+OIL (DARPA Agent Markup Language + Ontology Inference Language) and continues much of the work done previously on RDF (Resource Description Framework). The OWL representation can be registered easily in a standard UDDI Registry as a tModel, as we will explain later.

    The semantic model represents the meaning of a service's functionality. Achieving consensus in ontology can be difficult. But even before standards emerge for the ontologies of specific vertical business areas, the Service Provider can register a semantic model, which the Client can then inspect and use for look-up. Where heterogeneous semantic models must be reconciled, they can be integrated by merging them into a larger model that includes synonymous classes and properties.

    With a formal semantic model in place, it doesn't matter if the service works with dollars or euros, percentages or multipliers, as long as the service's functionality is encoded in the ontology. With formalisms in place, such syntactic differences are easily overcome through XSLT transformations that convert from one syntax to another.

    The ontological model does not yet represent the service itself, since specific input and output parameters are not encoded. However, it does represent the critical concepts needed to understand the service's functionality.

    Create a WSDL Document for the Web Services
    Next, the Provider creates the WSDL, which defines the Service's syntax by specifying the structure of the input and output messages, along with aspects of the service's runtime bindings. Creating a WSDL document and registering it to a UDDI Registry is a best practice even if the semantic approach is not used. A WSDL document for this service is shown in Listing 2.

    Map Elements from the WSDL Messages to Semantic Concepts
    The next stage is to express the meaning of the WSDL by mapping the operations, along with the schemas of the WSDL input and output messages or signatures, to the semantic model.

    In some cases, we map a schema element into an indirect property, which is a series of several properties, to express semantic concepts precisely.

    In our example, we map the schema element rate in the output structure to the rate property of class Tax, linked to the multiplier property of class Rate. In this way, we have encoded the precise meaning of this output parameter, stating that the rate is expressed as a multiplier rather than as a percentage. A Client looking for a percentage can later transform multiplier to percentage by using the business rule formally encoded in the semantic model: multiplier = percentage/100.

    Likewise, we map the zip schema element to the jurisdiction property of class Tax linked to the zipCodes property of class Jurisdiction, linked yet again with fiveDigit property of class ZipCode. In this way, we state that the tax is to be calculated in a given jurisdiction (city or state), that the jurisdiction is characterized by its ZIP codes, and that the ZIP code is expressed in its five-digit form.

    Figure 3 shows a subset of these mappings graphically, while Table 1 lists the mappings in full.



    Mappings can be expressed in an XML document formatted in the RDF. RDF is built of triplets expressing a relationship of the form subject-predicate-object. For example, we would express a mapping as the following triplet: 1) "element rate", 2) "maps to", and 3) "property rate of class Tax".

    Register the Model, WSDL, and Mappings to the UDDI Registry
    The last stage is to register these elements to a UDDI Registry. The concepts in the WSDL, the semantic model, and the mappings between them fit quite naturally into UDDI as tModels. In fact, it is already considered a best practice to store WSDL in this way; the other artifacts are stored in the same way (see Figure 4).


    The semantic approach to UDDI requires registering more artifacts to the registry than the nonsemantic approach. Fortunately, however, the UDDI object model is flexible enough to allow this sort of usage since the UDDI API has been designed with batch-processing capabilities, allowing the registering or querying of multiple tModels with a single remote call.

    The Client's Role:
    Semantic Discovery

  • Identify the required functionality.
  • Query UDDI, locate service, and retrieve all semantic and interface descriptions.
  • Create transformations to use the service from the Client.
  • Access the Web service using the WSDL document.

    The Web service is now ready for semantic discovery. The Client needs to discover a service that meets the required business needs and then calls this service.

    Identify the Required Functionality
    To identify the required service functionality, the Client first discovers the agreed-upon semantic model using UDDI and loads it over standard HTTP. The techniques to do so resemble the use of UDDI for finding any other resource. The Client simply locates the OWL document representing the semantic model by finding the appropriate tModel based on industry sector and general taxonomies.

    Even when not standardized, the semantic model is general enough to cover a wide variety of services for a given industry vertical or horizontal sector. For example, the model depicted above (see Figure 2 and Listing 1) would be part of a much larger industry-standard model for financial services. Where multiple "standard" models exist, they can easily be linked together through ontological properties and inheritance in order to form a larger, more inclusive model.

    In the non-semantic approach, the industry definitions and taxonomies jointly express a kind of semantics. But the Client must still know exactly what service is needed and exactly what input and output messages to look for in the registry. In the semantic approach, the Client first accesses the general service category through taxonomies; then, by referencing the functionality of the service in meaningful semantic classes such as SalesTax, ZipCode, and Jurisdiction, the Client finds the service that approximates its precise needs.

    Query UDDI, Locate Service, and Retrieve all Semantic and Interface Descriptions
    Having identified the relevant ontological concepts in the semantic model, the Client now navigates the mappings that link the model to the required WSDL files. The Client discovers a WSDL file with a function whose input and output schemas are mapped to the input and output concepts.

    In our example, a suitable Web service is any WSDL with the correct semantics:

  • The input-message schema includes elements expressing the ontological properties amount and zip.
  • The output-message schema includes elements expressing the ontological properties tax and rate.

    The Client identifies these semantic values and uses the mappings to find identifiers for the services that provide the functionality. The Client can then directly discover these services through UDDI.

    Create Transformations to Use the Service from the Client
    The Client may have been coded to pass input and expect output with a certain syntax, while the service expects input and returns output with another syntax. For example, the Client may pass a ZIP code as a nine-digit string, while the service expects a five-digit string; likewise, the service may return a tax rate as a multiplier such as 0.055, while the Client expects a percentage such as 5.5.

    This problem can be resolved through semantics. In a simpler but manual approach, the developer can code the transformation from one to the other using XSLT. With semantics in hand, the meaning of the parameters is clear and such transformation code is easy to develop. In a more advanced and dynamic approach, semantically based software can automatically generate XSLT to convert from the XML of one schema to the XML of another schema, based on the shared semantics of the schemas. This is suitable for WSDL operations whose parameters are specified as "literal." A third approach combines semantics with the flexibility of DII (Dynamic Invocation Interface): the client adapts its input messages to the needs of the service's interface at runtime. This is suitable to parameters specified as "encoded."

    Access the Web Service
    With a WSDL document available to fully characterize the Web service and transformations in place to adapt between the Client and the Server, the Client can now call on the Web service directly, just as with any WSDL.

    In the context of UDDI, WSDL alone cannot deliver the vision of loosely coupled, dynamically interoperating Web services, and so cannot ensure that the current growth in Web services adoption will continue. Without a semantic infrastructure to form the backbone of the UDDI Registry, semantic inconsistency will prevent the large-scale adoption of active Web services adoption. The ongoing decentralization of Web services will make this problem increasingly acute in the coming few years. Formalization of the semantics in Web services is a step toward automated UDDI discovery and the fulfillment of the Semantic Web vision.


  • UDDI for Python (UDDI4Py): www.alphaworks.ibm.com/tech/uddi4py
  • Microsoft UDDI Software Development Kit: http://uddi.microsoft.com/developer/default.aspx
  • Java API for XML Registries (JAXR): http://java.sun.com/xml/jaxr/
  • UDDI Homepage: www.uddi.org
  • UDDI4J (Open Source UDDI Client API in Java): www.uddi4j.org
  • "Using WSDL in a UDDI Registry: www.oasis-open.org/committees/uddi-spec/doc/bp/uddi-spec -tc-bp-using-wsdl-v108-20021110.htm
  • Resource Description Framework: www.w3.org/RDF
  • "Semantic Web Activity (W3C)": www.w3.org/2001/sw/
  • Web Ontology Language (OWL) Reference: www.w3.org/TR/owl-ref
  • Berners-Lee, Tim; Hendler, James . and Lassila, Ora. "The Semantic Web." www.sciam.com/2001/0501issue/0501berners-lee.html
  • Grove, Andy. "Understanding UDDI tModels and Taxonomies," Web Services Journal (Vol. 1, issue 2; November 2001).
  • Januszewski, Karsten. "UDDI and WSDL: Natural Companions," Web services Journal (Vol. 1, issue 3; December 2001).
  • Ogbuji, Uche. "Supercharging WSDL with RDF." www-106.ibm.com/developerworks/library/ws-rdf/?dwzone=ws


    WSDL Message Syntax

    WSDL allows SOAP input and output to be specified in two different ways: as "literal" values or "encoded" as function parameters. In the former approach, XML documents are passed; the latter approach resembles function calls as seen in programming languages. Either the "literal" or "encoded" approach can be used with the technique described here. Examples are given with the "literal" approach.

  • More Stories By Joshua Fox

    Joshua Fox is a senior software architect at Unicorn Solutions (www.unicorn.com), developing semantic information management systems for the enterprise. He has experience developing large-scale clustered Java systems for Internet collaboration and multimedia delivery, and has published and lectured widely. He can be reached at [email protected]

    More Stories By Joram Borenstein

    Joram Borenstein is a senior marketing manager at Unicorn Solutions (www.unicorn.com), working on data semantics technologies for enterprises worldwide. His previous experience includes managing the rollout of content management software platforms. He has published and lectured extensively. He can be reached at [email protected]

    Comments (0)

    Share your thoughts on this story.

    Add your comment
    You must be signed in to add a comment. Sign-in | Register

    In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.

    Microservices Articles
    Modern software design has fundamentally changed how we manage applications, causing many to turn to containers as the new virtual machine for resource management. As container adoption grows beyond stateless applications to stateful workloads, the need for persistent storage is foundational - something customers routinely cite as a top pain point. In his session at @DevOpsSummit at 21st Cloud Expo, Bill Borsari, Head of Systems Engineering at Datera, explored how organizations can reap the bene...
    "NetApp's vision is how we help organizations manage data - delivering the right data in the right place, in the right time, to the people who need it, and doing it agnostic to what the platform is," explained Josh Atwell, Developer Advocate for NetApp, in this SYS-CON.tv interview at 20th Cloud Expo, held June 6-8, 2017, at the Javits Center in New York City, NY.
    The Jevons Paradox suggests that when technological advances increase efficiency of a resource, it results in an overall increase in consumption. Writing on the increased use of coal as a result of technological improvements, 19th-century economist William Stanley Jevons found that these improvements led to the development of new ways to utilize coal. In his session at 19th Cloud Expo, Mark Thiele, Chief Strategy Officer for Apcera, compared the Jevons Paradox to modern-day enterprise IT, examin...
    In his session at 20th Cloud Expo, Mike Johnston, an infrastructure engineer at Supergiant.io, discussed how to use Kubernetes to set up a SaaS infrastructure for your business. Mike Johnston is an infrastructure engineer at Supergiant.io with over 12 years of experience designing, deploying, and maintaining server and workstation infrastructure at all scales. He has experience with brick and mortar data centers as well as cloud providers like Digital Ocean, Amazon Web Services, and Rackspace. H...
    Skeuomorphism usually means retaining existing design cues in something new that doesn’t actually need them. However, the concept of skeuomorphism can be thought of as relating more broadly to applying existing patterns to new technologies that, in fact, cry out for new approaches. In his session at DevOps Summit, Gordon Haff, Senior Cloud Strategy Marketing and Evangelism Manager at Red Hat, will discuss why containers should be paired with new architectural practices such as microservices ra...
    In his session at 20th Cloud Expo, Scott Davis, CTO of Embotics, discussed how automation can provide the dynamic management required to cost-effectively deliver microservices and container solutions at scale. He also discussed how flexible automation is the key to effectively bridging and seamlessly coordinating both IT and developer needs for component orchestration across disparate clouds – an increasingly important requirement at today’s multi-cloud enterprise.
    The Software Defined Data Center (SDDC), which enables organizations to seamlessly run in a hybrid cloud model (public + private cloud), is here to stay. IDC estimates that the software-defined networking market will be valued at $3.7 billion by 2016. Security is a key component and benefit of the SDDC, and offers an opportunity to build security 'from the ground up' and weave it into the environment from day one. In his session at 16th Cloud Expo, Reuven Harrison, CTO and Co-Founder of Tufin, ...
    DevOps is often described as a combination of technology and culture. Without both, DevOps isn't complete. However, applying the culture to outdated technology is a recipe for disaster; as response times grow and connections between teams are delayed by technology, the culture will die. A Nutanix Enterprise Cloud has many benefits that provide the needed base for a true DevOps paradigm. In their Day 3 Keynote at 20th Cloud Expo, Chris Brown, a Solutions Marketing Manager at Nutanix, and Mark Lav...
    Many organizations are now looking to DevOps maturity models to gauge their DevOps adoption and compare their maturity to their peers. However, as enterprise organizations rush to adopt DevOps, moving past experimentation to embrace it at scale, they are in danger of falling into the trap that they have fallen into time and time again. Unfortunately, we've seen this movie before, and we know how it ends: badly.
    TCP (Transmission Control Protocol) is a common and reliable transmission protocol on the Internet. TCP was introduced in the 70s by Stanford University for US Defense to establish connectivity between distributed systems to maintain a backup of defense information. At the time, TCP was introduced to communicate amongst a selected set of devices for a smaller dataset over shorter distances. As the Internet evolved, however, the number of applications and users, and the types of data accessed and...