Microservices Expo Authors: Liz McMillan, Pat Romanski, Carmen Gonzalez, Elizabeth White, Jason Bloomberg

Related Topics: Microservices Expo

Microservices Expo: Article

Integrating CICS Applications as Web Services

Integrating CICS Applications as Web Services

Web services promise to lower the costs of integration and help legacy applications retain their value. This article explains how you can use them to integrate mainframe CICS applications with other enterprise applications.

Web services are platform-independent interfaces that allow communication with other applications using standards-based Internet technologies, such as HTTP and XML. They provide an opportunity for organizations to reduce the costs and complexities of application integration inside the firewall and create new possibilities for legacy applications to participate in e-business. One problem with traditional integration techniques is the proliferation of point-to-point communication and data conversions that must change as new applications are integrated or data formats change. The problem quickly gets complicated when you add business partners, subsidiaries, mergers, or acquisitions to the integration mix. Web services simplify integration by reducing the number of APIs to one (SOAP) and the number of data formats to one (XML). While there is often variation in the syntax of the XML payload, such as the FinXML, FIXML, OFX, and IFX standards used within financial services, the fundamental knowledge and skills needed to work with each of these standards is the same.

Web services are standards based and platform independent. Independent standards bodies control the direction of the core components, rather than individual corporations pushing their own technologies. By using standards-based technologies and widely available skill sets, Web services allow companies to develop flexible integration solutions that can evolve as needs change.

Web services sound promising for new applications that support built-in Internet technologies, but what about legacy applications, like CICS? How can they benefit from Web services integration and what does it take to integrate legacy applications as Web services?

CICS Applications and Web Services
IBM's CICS (Customer Information Control System) is a family of application servers that provides online transaction management and connectivity for legacy applications. Some simple facts about CICS demonstrate the importance of integrating CICS with business-critical initiatives that involve Web services.

  • 30 years and $1 trillion invested in CICS applications (IDC)
  • 20,000+ CICS/ mainframe licenses worldwide
  • 14,000+ CICS customers worldwide
  • Used by 490+ of IBM's top 500 customers
  • 30 million end users of CICS applications
  • 150,000+ concurrent users/system
  • 5,000 CICS software packages from 2,000 ISVs
  • 950,000 programmers earn their living from CICS
  • CICS handles more than 30 billion transactions/day valued at over $1 trillion/week

    Figure 1 shows a high-level taxonomy of CICS applications.


    CICS represents the broadest category of mainframe applications. CICS transactions fall into two subcategories: "visual" and "nonvisual." A "visual" transaction is one that expresses a presentation interface to an end user at a terminal. You could also refer to a "visual" transaction as a "terminal-oriented" transaction. In contrast, "nonvisual" transactions don't interact with an end user. Instead, another program invokes these transactions. (This type of transaction is also referred to as a "COMMAREA transaction" because the input/output parameters are passed to/from the transaction using an area of storage referred to as the "communication area," or COMMAREA.)

    The nature of CICS applications makes them complementary to Web services technologies. Nonvisual COMMAREA applications take one request and return data to the requesting application in a single step. This maps well with the Web services model because a single SOAP request would yield the required host data. Meanwhile, the majority of visual applications use a component of CICS called Basic Mapping Support (BMS). BMS essentially handles the presentation logic of the transaction and relieves the application developer of having to encode and decode 3270 terminal data streams. BMS expresses fields and data as name/value pairs, which can be converted to XML for consumption by Web services.

    While mainframes continue to be the most reliable and scalable platforms for handling large amounts of data and large numbers of transactions, many organizations want to move to newer technology and stop relying on the dwindling numbers of COBOL developers and system administrators with mainframe skills. However, in most cases these same organizations realize that their legacy applications "work", their business processes rely on them, and that making changes to decades of business logic is both time-consuming and risky. Moreover, integrating applications is usually cheaper than rewriting them to operate on other platforms. Now by enabling your legacy applications as Web services you can deliver integration projects faster and cheaper because your legacy application groups and Internet groups use their existing knowledge to integrate the applications.

    Integration Models
    There are two basic models for integrating CICS applications as Web services, both of which include the use of adapters. The differences between these models depend upon where the Web services exist, how they operate under the covers, and the types of applications you want to integrate. In this article, we will refer to these models as connectors and gateways.

  • Connectors run on the mainframe and can use native interfaces that permit seamless integration with the target application.
  • Gateways run off the mainframe on middle-tier servers and often use traditional methods, such as screen-scraping (that is, capturing legacy data based on row/column coordinates of the application screen).

    In the purest sense, adapters might seem antithetical to the Web services model because the target application is not itself acting as a Web service. Nevertheless, when you face the daunting task of rewriting trillions of lines of code and millions of legacy COBOL/Assembler applications, the need for adapters becomes apparent. The question then becomes "what type of adapter do you want to use?"

    The choice between using connectors or using gateways often depends upon the types of applications you need to integrate. Integration vendors have traditionally provided facilities that allow you to use gateways to access CICS. These gateways commonly use screen-scraping techniques. However, with the release of CICS Transaction Server, IBM began providing facilities that allow the use of connectors to access legacy applications, so you can choose between connector and gateway models based on your needs.

    The recent availability of connectors that support both visual (terminal-oriented) and nonvisual (COMMAREA) CICS application types allows remote applications to invoke almost any CICS application as a web service. Because most shops have a mix of CICS application types, companies should seek out this kind of connector to avoid multiple software licenses and additional training on how to integrate the different application types within your organization.

    In the connector and gateway examples below, we focus on using Web services with visual (terminal-oriented) applications because they are typically more difficult to integrate than nonvisual (COMMAREA) applications.

    Connector Model for Web Services
    Connectors allow you to transform your legacy applications into Web services without requiring the use of additional hardware, without changes to the legacy application, and without falling back upon brittle techniques like screen scraping. Compared to gateways, connectors yield better performance by running on the host, and more reliable operation due to the elimination of the many layers data must pass through due to screen-scraping (Figure 3 details these layers). Connectors also provide enhanced access to application information such as state and error codes. This information is lost when data is sent to a terminal emulation technology such as a 3270 emulation client.

    Figure 2 shows the basic model for accessing legacy applications as Web services through connector technologies. It illustrates a Web services architecture: a Provider that supplies the Web service, a Requester that uses a Web service, and a Broker that finds Providers for Requesters. In this model, the legacy application is the Provider. The following steps represent how to find and use a Web service connector (steps 1-3 are optional).
    1.   The Provider uploads a WSDL specification to publish its Web service with a Broker.
    2.   The Requester (usually a Java or .NET application) queries the Broker for a Web service by name or category.
    3.   The Broker selects a Provider and returns the Provider information to the Requester.
    4.   The Requester uses the information from the Broker to format and send a SOAP message to the service provider.
    5.   The Provider returns a SOAP/XML response to the Requester with the legacy application data enclosed.


    Gateway Model for Web Services
    Unlike connectors, gateways typically run on a physical or logical middle tier. Where the gateway runs is important because there are so few options for accessing the host from the middle-tier servers, which means gateways usually involve some form of screen-scraping. The solution is tightly coupled in that the integration is between the gateway and a specific application. Any changes to the application will break the integration.

    When gateways communicate with terminal-oriented legacy applications they open a terminal session with the legacy application, send a request to the application, receive the terminal datastream, use HLLAPI to capture the screen data, process the screen data, convert the contents to XML, and ship the XML document to the requester. (A variation of the gateway model is to use FEPI on the mainframe instead of a middle-tier terminal emulation client. This simply moves the middle tier onto the mainframe.) The most common components of the gateway model appear in Figure 3.


    Gateways allow you to get an application into the Web services mix, but screen scraping creates performance bottlenecks and multiple points of failure between the legacy application and the Web service. For this reason, gateways are best for short-term projects, either as a transition to using connectors or as a stopgap measure during application reengineering or platform migration.

    Figure 4 shows the basic model for accessing legacy applications indirectly using a gateway technology such as a screen scraper.


    Again, the diagram in Figure 4 illustrates a Web services architecture: a Provider that supplies the Web service, a Requester that uses a Web service, and a Broker that finds Providers for Requesters. The legacy application is not itself a Web service, but is accessed by an off-host Provider. The following steps represent how to find and use a Web service gateway (steps 1-3 are optional).
    1.   The Provider uploads a WSDL specification to publish its Web service with a Broker.
    2.   The Requester (usually a Java or .NET application) queries the Broker for a Web service by name or category.
    3.   The Broker selects a Provider and returns the Provider information to the Requester.
    4.   The Requester uses the information from the Broker to format and send a SOAP message to the Provider.
    5.   The Provider starts an emulation session and conducts a series of transactions with the legacy application to collect the requested data.
    6.   The Provider converts the legacy data stream into an appropriate SOAP/XML response and returns the response to the Requester with the legacy application data enclosed.

    Dynamic Connectors vs Code Generators
    As we have seen, connectors provide several advantages over gateways when it comes to Web services integration. How those connectors operate can be just as important. Products that implement the connector model for Web services usually fall into two camps: those that use code generation and those that are dynamic. Code generators have been around for some time and require developers to adhere to a fairly structured process. In the case of CICS applications, something like the following takes place:
    1.   Download program source code, copybooks, or screen maps to a workstation running a proprietary tool.
    2.   Use a proprietary tool to generate Web services "wrapper" code.
    3.   Upload generated code to the mainframe.
    4.   Compile, install, and test the generated code.
    5.   Repeat this process for each program and, again, whenever a program changes.

    While generated connectors can jump-start the initial development process, they create several maintenance problems. First, the integration developer is generating code off the host and uploading the code to the mainframe. In most cases, this developer will know a lot about Web services and XML, but will know little about the mainframe. Thus, any changes to the mainframe application require coordination between the Web developer, the application developer, and the CICS administrator. Second, generated connectors create "net new code" that must be managed. Changes to either the legacy application or to the Web service will require repetition of the code generation process to keep the generated code in sync with the integrated application. Without proper management, you are likely to drown in a sea of generated code.

    These issues led to the development of dynamic connectors. Dynamic connectors operate with little or no configuration and changes to legacy applications are automatically incorporated into the SOAP/XML output. In many cases there is no configuration required, while some cases may require a single-step process to specify Web service information for each application. As a result, there is no generated code and there is a clear division of labor: the CICS administrator installs the connector and the Web developer simply invokes the connector as a Web service.

    The subsystem under which your legacy application runs determines your top-level integration choices. IBM's CICS Transaction Server includes facilities that allow third-party vendors to create connectors that can immediately enable legacy applications as Web services. These facilities provide additional benefits over gateways, such as improved performance and increased stability compared to their screen-scraping counterparts. By using the same industry-standard technologies as Web services, some connectors make it possible for applications to transparently invoke CICS transactions within a Web services architecture and receive the resulting data as well-formed XML. For organizations that want to retain the value of their CICS applications, the combination of XML-enabling connectors and Web services offers a practical and powerful integration solution.

    Web services are not a trend, but an industry-wide movement that can provide a long-term solution for companies that want to integrate legacy applications and data with new e-business processes. In the end, companies need to assess the value of the data contained in their CICS applications. Most companies have already determined that such data is highly valuable and they are looking for ways to preserve their investments. Given that recent surveys show the top strategic priorities of CIOs and CTOs are integrating systems and processes, the use of Web services for legacy integration will grow rapidly.

  • More Stories By Russ Teubner

    During his 23-year career, Russ has focused much of his creative energy on solving the difficult problems associated with the integration of legacy systems with technologies that have emerged more recently, such as XML. He currently is CEO of HostBridge Technology.

    ------ Publications ------
    Enterprise Systems Journal
    z/Series Journal

    Comments (1)

    Share your thoughts on this story.

    Add your comment
    You must be signed in to add a comment. Sign-in | Register

    In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.

    Microservices Articles
    Modern software design has fundamentally changed how we manage applications, causing many to turn to containers as the new virtual machine for resource management. As container adoption grows beyond stateless applications to stateful workloads, the need for persistent storage is foundational - something customers routinely cite as a top pain point. In his session at @DevOpsSummit at 21st Cloud Expo, Bill Borsari, Head of Systems Engineering at Datera, explored how organizations can reap the bene...
    "NetApp's vision is how we help organizations manage data - delivering the right data in the right place, in the right time, to the people who need it, and doing it agnostic to what the platform is," explained Josh Atwell, Developer Advocate for NetApp, in this SYS-CON.tv interview at 20th Cloud Expo, held June 6-8, 2017, at the Javits Center in New York City, NY.
    The Jevons Paradox suggests that when technological advances increase efficiency of a resource, it results in an overall increase in consumption. Writing on the increased use of coal as a result of technological improvements, 19th-century economist William Stanley Jevons found that these improvements led to the development of new ways to utilize coal. In his session at 19th Cloud Expo, Mark Thiele, Chief Strategy Officer for Apcera, compared the Jevons Paradox to modern-day enterprise IT, examin...
    In his session at 20th Cloud Expo, Mike Johnston, an infrastructure engineer at Supergiant.io, discussed how to use Kubernetes to set up a SaaS infrastructure for your business. Mike Johnston is an infrastructure engineer at Supergiant.io with over 12 years of experience designing, deploying, and maintaining server and workstation infrastructure at all scales. He has experience with brick and mortar data centers as well as cloud providers like Digital Ocean, Amazon Web Services, and Rackspace. H...
    Skeuomorphism usually means retaining existing design cues in something new that doesn’t actually need them. However, the concept of skeuomorphism can be thought of as relating more broadly to applying existing patterns to new technologies that, in fact, cry out for new approaches. In his session at DevOps Summit, Gordon Haff, Senior Cloud Strategy Marketing and Evangelism Manager at Red Hat, will discuss why containers should be paired with new architectural practices such as microservices ra...
    In his session at 20th Cloud Expo, Scott Davis, CTO of Embotics, discussed how automation can provide the dynamic management required to cost-effectively deliver microservices and container solutions at scale. He also discussed how flexible automation is the key to effectively bridging and seamlessly coordinating both IT and developer needs for component orchestration across disparate clouds – an increasingly important requirement at today’s multi-cloud enterprise.
    The Software Defined Data Center (SDDC), which enables organizations to seamlessly run in a hybrid cloud model (public + private cloud), is here to stay. IDC estimates that the software-defined networking market will be valued at $3.7 billion by 2016. Security is a key component and benefit of the SDDC, and offers an opportunity to build security 'from the ground up' and weave it into the environment from day one. In his session at 16th Cloud Expo, Reuven Harrison, CTO and Co-Founder of Tufin, ...
    DevOps is often described as a combination of technology and culture. Without both, DevOps isn't complete. However, applying the culture to outdated technology is a recipe for disaster; as response times grow and connections between teams are delayed by technology, the culture will die. A Nutanix Enterprise Cloud has many benefits that provide the needed base for a true DevOps paradigm. In their Day 3 Keynote at 20th Cloud Expo, Chris Brown, a Solutions Marketing Manager at Nutanix, and Mark Lav...
    Many organizations are now looking to DevOps maturity models to gauge their DevOps adoption and compare their maturity to their peers. However, as enterprise organizations rush to adopt DevOps, moving past experimentation to embrace it at scale, they are in danger of falling into the trap that they have fallen into time and time again. Unfortunately, we've seen this movie before, and we know how it ends: badly.
    TCP (Transmission Control Protocol) is a common and reliable transmission protocol on the Internet. TCP was introduced in the 70s by Stanford University for US Defense to establish connectivity between distributed systems to maintain a backup of defense information. At the time, TCP was introduced to communicate amongst a selected set of devices for a smaller dataset over shorter distances. As the Internet evolved, however, the number of applications and users, and the types of data accessed and...