Welcome!

Microservices Expo Authors: Liz McMillan, Pat Romanski, Elizabeth White, Mehdi Daoudi, Yeshim Deniz

Related Topics: Microservices Expo

Microservices Expo: Article

Unlock the Power of the Mainframe

Unlock the Power of the Mainframe

Over many decades, IT organizations have invested billions of dollars in business logic and data housed in mainframes and within the CICS environment. Despite the age of these systems, the mainframe and CICS continue to provide the most scalable, reliable, and cost-effective platform upon which to conduct critical business operations.

Yet expanding the ROI of these systems by leveraging more modern technologies, such as Web services, continues to be a challenge.

Today's business environment requires an IT solution that will transform existing, rigid systems into flexible building blocks to enable a component-based, process-centric, and cross-platform integration environment. By adding an integration platform based on a service-oriented architecture, organizations can unlock critical mainframe resources and leverage data and business logic as services, allowing an organization to create a more flexible, dynamic, and agile infrastructure for competing in the 21st century.

This article presents the notion of CICS as an integration hub based on a component-based, service-oriented architecture supporting Web services. The challenges presented by most current approaches to implementing Web services to integrate with the mainframe are examined and contrasted with supporting Web services natively in CICS.

Among the technical issues discussed are:

  • Transforming CICS resources into XML via an XML SQL metaphor
  • Dynamically generating WSDL that effectively exposes Web services and subsequently accessing these services as SOAP-enabled Web services
  • Building composite applications or business processes that are composed of multiple CICS and non-CICS resources, and accessing the newly created business process as a Web service executing natively in CICS

    Web Services Brings Ubiquity to Integration
    The benefits of Web services have been touted for a few years now. Standards for communications (TCP/IP and HTTP), message exchange (SOAP), data formatting (XML), and means of describing the access (WSDL) to services have emerged. Many vendor products support standards for creating Web services requestors and providers, and some tools can even generate code from WSDL for service requestors to access Web services.

    The supporting runtime infrastructure and integration capabilities between Web services and legacy applications and data are the focus of this article. The standards don't deal with what is behind a Web service, nor should they. But behind the curtain lurks the dirty work that has to be done to make Web services real integration.

     

    Service-Oriented Architecture: Bridging CICS with Web Services
    Web services simply move the ability to create services into a standards-based approach that is independent of hardware/operating system platform and program language. Simply using Web services technologies to continue creating custom, point-to-point integration linkages produces just another tangled web of issues to address.

    The real power of a service-oriented architecture is to think in terms beyond individual application-to-application linkages and more in terms of generic services that can be constructed for any and all applications to leverage. This is the key to effectively realizing the maximum benefit of Web services.

    Typical Web Services Approach
    Typical Web services products run on an outboard processor from the mainframe/CICS perspective, usually on a Unix or Windows server platform.

    The ability to create Web services to wrap mainframe resources is limited by the ability of the chosen tool to integrate with the various types of resources found in CICS or accessible from CICS, including:

    • COMMAREA programs
    • VSAM files
    • 3270 screens (BMS and others)
    • DB2 tables and stored procedures
    • IMS data and programs
    • MQSeries-enabled applications
    Not all vendor products provide the ability to access all of these resources and some are accessed using the same old tired EAI techniques of the past behind a facade of Web services.

    There is still quite a bit of separation between the mainframe/CICS resources and the Web service provider platform.The issues in this type of an approach include:

    • Security
    • Transactional semantics
    • Change management
    • Availability and scalability
    • Cultural differences
    Security
    Authentication and access control are critical to the security of CICS resources, as well as for accounting purposes. What capabilities are offered to allow for users or programs to be authenticated to the back-end resources? HTTPS can be leveraged. Digital certificates can be crafted so that the identity of the Web service requestor will correspond to RACF userids, but the two security mechanisms must be integrated.

    Transactional Semantics
    CICS is a transaction manager. Many line-of-business systems depend upon transactional capabilities from order entry to order fulfillment and shipping to billing and so on. Virtually all financial institutions from banks to brokerage firms to insurance companies - depend upon CICS. ATM networks are controlled by CICS-based systems, whose reach encompasses the ATM terminal. It's important to understand the demarcation points of where transactions begin and end, and the impact on the end-to-end system. What happens if the communication link between the Web/application server or integration broker and the mainframe has a problem? How did the failure impact the Web service? Did the failure occur before initiating any CICS transactions or after? If the request got to CICS, did the CICS transaction complete? How is the Web service requestor or provider to know?

    Change Management
    All of the integration techniques of the past caused problems whenever something changed in the legacy application and data:

  • Changes to 3270 screens broke any screen- scraping integration
  • Printer output changes impacted applications tapping into printer drops
  • Changes to data needed to be reflected in RPC client and server stub - requiring regeneration, recompilation, and redistribution to any client applications
  • Changes to fixed message formats needed to be reflected in changes to code when applications were integrated using messaging and queuing

    XML and Web services help in that additional XML tags and data can be added to a SOAP message body. Only those applications that need the new tags and data need to change.

    Although changes can be isolated due to the loose coupling between Web service providers and requestors and the ability of Web service requestors to dynamically react to changes via WSDL, there is still an issue. An enormous problem common among the typical approaches to integrating the mainframe with Web services is managing changes to the underlying resources and the downstream impact to the Web services that have been created to wrap them. Because most approaches call for a Web/application server or integration broker running on a platform other than the mainframe, changes must be coordinated between the two platforms and typically between two organizations.

    Availability and Scalability
    Utilizing a separate platform to host a Web/application server and/or gateway requires two layers of systems that must be highly available and scalable and must also deal with potential communication failures between the two layers as well. This makes for a very complex operational situation. Clearly, tools and techniques exist to address the availability and scalability of Web servers that can also be applied to Web/application servers hosting Web services. And just as clearly, the mainframe/CICS environment also provides tools and technologies to address high availability and scalability of transactional systems. But they are two distinct environments and sets of tools/ technologies solving the same basic problem in two places. These tools and technologies don't integrate with each other. This requires two skill sets on the part of operational personnel and complicates troubleshooting. Furthermore, the communications links between the two environments must also mesh well with the high availability/scalability solutions on either endpoint.

    Web Services and CICS:
    Can These Cultures Coexist?

    The predominant approaches to constructing Web services rely upon two fairly recent technologies - Sun Microsystems' Java 2 Platform Enterprise Edition (J2EE) and Microsoft's .NET. Virtually all vendor products require the use of a Windows or Unix-based server. Many require some form of a gateway and/or third-party adapters to interface to the mainframe.

    The result is still two independent camps, much like in past EAI efforts, that require cooperation and communication. Worse yet is the ownership issue. Whichever group "owns" the integration between the two environments is critical to the operations team. One of the greatest challenges to any EAI effort between the mainframe and other platforms is communication and cooperation among the different development and support organizations.

    The Solution: CICS as the Web Services Hub
    To overcome these issues, as well as provide for a more robust integration environment, CICS itself should host the Web services provider and integration layer. Furthermore, the task of creating and managing services should be performed by the same organization that maintains and manages the legacy applications and data being wrapped to ensure consistency and eliminate communications problems typically found in most integration scenarios.

    This approach offers a single point of control for both the Web services and the CICS resources behind them. The Web service provider is, in effect, also the CICS resource provider. When problems arise, there is no issue over who owns the problem. And because there are no longer two distinct environments separated by a network, problem determination is simpler.

    All sorts of operational issues are eliminated. Another layer of hardware/software to maintain and manage is removed, reducing operational costs. The same management tools currently being employed to support CICS can also be used to deal with the Web services also running in CICS. No retraining of operational staff is necessary.

    The Web service provider, by virtue of the fact that it is under the control of CICS, immediately inherits all the benefits of CICS, including transactional semantics, availability, and scalability. Security can be more tightly integrated with CICS from the Web service provider as well.

     

    Putting the Pieces Together
    CICS offers a feature known as CICS Web Support (CWS), which allows a Web browser to connect directly to CICS to invoke a CICS transaction using the EXEC CICS LINK call and passing data via the COMMAREA. EBCDIC/ASCII translations are automatically handled. And decode/encode functions may be plugged in to manipulate the data being passed between the Web interface and COMMAREA of the invoked transaction. All the hooks necessary to support Web services via HTTP exist in the CICS environment.

    When combined with the capability to form Sysplex and CICSplex configurations, CICS Web Support provides a highly available and scalable platform for creating Web services second to none.

    What Is Needed?
    Additional capabilities are needed to enable CICS to operate as a Web services provider and integration hub:

  • A mechanism for translating native data formats and record layouts from/to XML
  • The ability to deal with SOAP messaging
  • A repository to define services and mappings as meta-data
  • The ability to dynamically generate WSDL for a given service based upon the meta-data
  • The ability to access data and programs using a structured approach

    Each item should be controlled under the auspices of CICS, easily understood by the mainframe/CICS Systems Programmer, and easily linked with CICS resources.

    XML Translation
    The ability to translate between XML and native data types (COBOL, DB2, etc.) and record formats (DB2 tables, VSAM files) is necessary. An XML parser is available for COBOL. However, there are a number of issues involved:

  • Requires an upgrade to COBOL 3
  • Only supports reading XML, not creating XML
  • Efficiency concerns exist
  • Cumbersome for COBOL programmers to work with

    SOAP Messaging
    Decoding/encoding of SOAP message formats for both inbound and reply messages requires the use of a SOAP processor.

    Repository
    A repository of metadata should be maintained to define Web services and their mappings to various CICS resources.

    Dynamically Generate WSDL
    With the Web services defined via metadata in a repository accessible via CICS, it's easy to configure the Web service to respond to inquiries and generate WSDL on the fly to specify available services and interfaces.

    Accessing Programs and Data Using a Structured Approach
    Simply providing a CICS-resident means of supporting Web services is necessary but not sufficient to allow IT organizations to realize the benefits of Web services/CICS integration.

    For simple resources such as VSAM files and DB2 tables, one recommendation is to offer an SQL-like mechanism to define the methods or operations that may be invoked on the service. The methods should include select(), insert(), update(), and delete(). Programs, including DB2 stored procedures, can be executed using the executeProcedure() method.

    By defining a standard set of methods, it becomes easier to generate code to support the service level interfaces, especially when combined with the ability to map XML to/from native CICS structures.

    Setting the Stage for Composite Services and Beyond
    Once various CICS-accessible resources have been wrapped as Web services, they may also be invoked within CICS itself, thereby creating the opportunity for realizing composite services at the source. Furthermore, with the appropriate building blocks in CICS, business processes can be constructed and exposed as Web services as well.

    Conclusion
    This article explains the benefits and advantages of supporting Web services directly inside CICS instead of the traditional approach most vendors have taken by providing this support on a middle tier. A number of vendors are moving in this strategic direction - one of which is IBM, the CICS provider. IBM's traditional approach to providing Web services access to CICS resources is via the WebSphere Application Server and the CICS Transaction Gateway. IBM recently announced a technology preview that supports SOAP enablement directly from CICS Transaction Server 2.2. While not a complete implementation of a Web services integration platform as prescribed in this article, it is a first step toward leveraging CICS as the hub for Web services and validates the approach taken in this article.

    References

  • Gisolfi, Dan. "Use Web services to integrate with OS390/CICS." IBM eServer Developer Domain: www-1.ibm.com/servers/esdd/articles/ os390/index.html.
  • SOAP for CICS: www-3.ibm.com/software/htp/cics/soap
  • More Stories By Paul Roth

    Paul Roth is CTO of CommerceQuest. he is a leading advocate within the industry for open standards, e-business, and business process integration. Paul serves as the company's technology evangelist, speaking on the technical issues surrounding business integration across and inside enterprises.

    Comments (0)

    Share your thoughts on this story.

    Add your comment
    You must be signed in to add a comment. Sign-in | Register

    In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.


    Microservices Articles
    The explosion of new web/cloud/IoT-based applications and the data they generate are transforming our world right before our eyes. In this rush to adopt these new technologies, organizations are often ignoring fundamental questions concerning who owns the data and failing to ask for permission to conduct invasive surveillance of their customers. Organizations that are not transparent about how their systems gather data telemetry without offering shared data ownership risk product rejection, regu...
    Containers and Kubernetes allow for code portability across on-premise VMs, bare metal, or multiple cloud provider environments. Yet, despite this portability promise, developers may include configuration and application definitions that constrain or even eliminate application portability. In this session we'll describe best practices for "configuration as code" in a Kubernetes environment. We will demonstrate how a properly constructed containerized app can be deployed to both Amazon and Azure ...
    DevOps is often described as a combination of technology and culture. Without both, DevOps isn't complete. However, applying the culture to outdated technology is a recipe for disaster; as response times grow and connections between teams are delayed by technology, the culture will die. A Nutanix Enterprise Cloud has many benefits that provide the needed base for a true DevOps paradigm. In their Day 3 Keynote at 20th Cloud Expo, Chris Brown, a Solutions Marketing Manager at Nutanix, and Mark Lav...
    The now mainstream platform changes stemming from the first Internet boom brought many changes but didn’t really change the basic relationship between servers and the applications running on them. In fact, that was sort of the point. In his session at 18th Cloud Expo, Gordon Haff, senior cloud strategy marketing and evangelism manager at Red Hat, will discuss how today’s workloads require a new model and a new platform for development and execution. The platform must handle a wide range of rec...
    The Internet of Things is clearly many things: data collection and analytics, wearables, Smart Grids and Smart Cities, the Industrial Internet, and more. Cool platforms like Arduino, Raspberry Pi, Intel's Galileo and Edison, and a diverse world of sensors are making the IoT a great toy box for developers in all these areas. In this Power Panel at @ThingsExpo, moderated by Conference Chair Roger Strukhoff, panelists discussed what things are the most important, which will have the most profound e...
    If your cloud deployment is on AWS with predictable workloads, Reserved Instances (RIs) can provide your business substantial savings compared to pay-as-you-go, on-demand services alone. Continuous monitoring of cloud usage and active management of Elastic Compute Cloud (EC2), Relational Database Service (RDS) and ElastiCache through RIs will optimize performance. Learn how you can purchase and apply the right Reserved Instances for optimum utilization and increased ROI.
    TCP (Transmission Control Protocol) is a common and reliable transmission protocol on the Internet. TCP was introduced in the 70s by Stanford University for US Defense to establish connectivity between distributed systems to maintain a backup of defense information. At the time, TCP was introduced to communicate amongst a selected set of devices for a smaller dataset over shorter distances. As the Internet evolved, however, the number of applications and users, and the types of data accessed and...
    Consumer-driven contracts are an essential part of a mature microservice testing portfolio enabling independent service deployments. In this presentation we'll provide an overview of the tools, patterns and pain points we've seen when implementing contract testing in large development organizations.
    In his session at 19th Cloud Expo, Claude Remillard, Principal Program Manager in Developer Division at Microsoft, contrasted how his team used config as code and immutable patterns for continuous delivery of microservices and apps to the cloud. He showed how the immutable patterns helps developers do away with most of the complexity of config as code-enabling scenarios such as rollback, zero downtime upgrades with far greater simplicity. He also demoed building immutable pipelines in the cloud ...
    You have great SaaS business app ideas. You want to turn your idea quickly into a functional and engaging proof of concept. You need to be able to modify it to meet customers' needs, and you need to deliver a complete and secure SaaS application. How could you achieve all the above and yet avoid unforeseen IT requirements that add unnecessary cost and complexity? You also want your app to be responsive in any device at any time. In his session at 19th Cloud Expo, Mark Allen, General Manager of...