Welcome!

Microservices Expo Authors: Elizabeth White, Pat Romanski, Dalibor Siroky, Scott Davis, Stackify Blog

Related Topics: Microservices Expo

Microservices Expo: Article

Toward Web Services Management Standards

Toward Web Services Management Standards

The work being done in WSDM will lay a firm foundation for effective distributed system management, both leveraging the unifying strengths of Web services in the solution itself and addressing the specific requirements for managing what are rapidly becoming the universal glue in enterprise system design.

The OASIS Web Services Distributed Management Technical Committee (WSDM TC) was chartered in March 2003 to recommend standards to address a problem that has been developing for many years. But with the widespread emergence of Web services and service-oriented architecture implementations in mainstream enterprise IT environments, the distributed systems management problem can no longer be ignored. The WSDM TC is focusing on two distinct tasks as it attempts to solve some pressing distributed system management problems.

The first activity area, called Management Using Web Services (MUWS) addresses the use of Web services technologies as the foundation of a modern distributed systems management framework - including using Web services to facilitate interactions between managed resources and management applications. The same characteristics that make Web services successful for application integration make them an excellent choice for use in solving the management integration problem - facilitating communications between managers and resources across numerous vendors, platforms, technologies, and topologies.

In addition to the use of Web services in the creation of a management framework, WSDM is addressing the specific requirements for managing Web services like any other IT resource. This activity is called Management of Web Services (MOWS). The manageability models that are being developed for Web services will be exposed using the techniques defined as part of the MUWS task.

Web Services Accelerate a Problem
For over two decades, there has been an evolution underway in the architecture of enterprise IT systems - a move away from big, unwieldy monolithic systems toward distributed architectures. Technologies such as DCE, CORBA, and DCOM were attempts to standardize mechanisms for software system interoperability. Client/server architectures - two-tier, three-tier, and then n-tier - were stepping stones along the path to what is rapidly emerging as the dominant form of enterprise IT system design - the service-oriented architecture (SOA).

The SOA is a distributed system made up of software components that interact by passing messages to discoverable service access points. The SOA stresses interoperability, location transparency, and loose coupling.

While most consider the SOA an evolutionary step in the long road toward enterprise-scale distributed computing, the rate of adoption of SOAs based on Web services technologies is nothing short of revolutionary. This revolution is being fueled by the unprecedented vendor support of Web services standards. Web services deliver the required messaging and discovery building blocks for enterprise SOA deployments.

As Web service-based SOAs spring up across the enterprise landscape, they are accelerating and exacerbating a systems management challenge that has been growing in urgency in parallel with the development of enterprise-scale distributed computing.

In a monolithic application (or tightly coupled, client/server application), application boundaries are relatively clear and fixed. In an SOA environment, applications cross system (and even organizational) boundaries, they overlap, and they can change over time (see Figure 1).

 

Managing these applications is a serious challenge - but an absolute requirement. Failure or change of a single application component can bring down numerous interdependent enterprise applications. The addition of new applications or components can overload existing components, causing unexpected degradation or failure of seemingly unrelated systems. Application performance depends on the combined performance of cooperating components and their interactions.

Effective systems and application management in a Web service-based SOA requires a management framework that is consistent across an increasingly heterogeneous set of participating component systems, while supporting complex aggregate (cross-component) management use cases, like service-level agreement enforcement and dynamic resource provisioning.

The rapid emergence of Web services and their acceleration of SOA adoption have made this need urgent - resulting in the charter of WSDM in March 2003.

Basic Structure and Components
Figure 2 highlights the basic structure and components of today's management frameworks. This generic model will be used to highlight the specific areas of management being addressed by WSDM and how they differ from traditional models.

 

At the bottom of the framework is a managed system that contains a collection of managed resources. A managed system typically contains multiple managed resources - for example, CPU, memory, disks, software components, and Web services. In an SOA, a key management framework requirement is the ability to understand the relationships between managed resources both within and across system boundaries. Management systems today do a relatively poor job of building, maintaining, and leveraging information about resource relationships - particularly across managed system boundaries.

Next up the stack is the management agent. This component models managed resources and represents these resources in interactions with a manager. The manager always addresses and interacts with the management agent. The agent then interacts with managed resources.

A management protocol is the communication channel between the manager and the management agent. A management protocol is usually a two-way communication channel that allows the flow of operation requests and responses (e.g. get, set, and stop). It may use the same channel for passing notifications of events of interest.

In the distributed systems management context, it is increasingly important for management agents (and supporting management protocols) to define and support active capabilities versus traditional passive capabilities. For example, rather than merely raising an alert when a given Web service is unable to meet the performance requirements of a given service-level agreement, the management framework should be able to take corrective action. This action could take the form of rerouting requests to a backup service that is less heavily loaded, or provisioning a new application server with an instance of the software providing the service if no backup is currently running and available.

The manager is an application that communicates with management agents via management protocols, gathering information about managed system and managed resource status and performance, and supporting specific management tasks (e.g., root cause failure analysis, SLA monitoring and reporting, and capacity planning). It is this set of supported tasks that should drive (and is driving in the case of WSDM) the requirements for the rest of the management framework.

Existing Standards and Solutions Do Not Address the Problem
System and application management considerations have always been secondary to the creation and introduction of new systems and applications.

Management standards and solutions typically trail the introduction of new approaches to IT system design. Over time, the result has been a proliferation of fractured, simplistic management standards and proprietary solutions that are usually targeted to solve point management problems. These point solutions have been specific to managed resources (e.g., storage, network infrastructure, software applications), management problems (e.g., failure detection and analysis, performance management, security policy enforcement), or specific vendor or technology solutions (J2EE application management, Windows server management).

In a world of complex distributed systems, these no longer suffice. To effectively manage a distributed application there must be a uniform mechanism for consistently managing its constituent components; understanding and managing component relationships; and managing the resulting aggregate system as a whole. In addition, emerging customer requirements for autonomic computing capabilities require management solutions to be far more active in nature - solving problems, not just identifying them. Management systems must be able to do all of these things for applications and systems across the enterprise, and across enterprise boundaries.

WSDM addresses many of these traditional management solution shortcomings (e.g., lack of integration across vendors and platforms, end-to-end information collection, agent dependency and passivity) through two distinct focus areas as highlighted in Figure 3.

 

Management Using Web Services (MUWS)
WSDM MUWS activity focuses on defining how Web services architecture and technologies can be used to manage any IT resource. In particular, MUWS will define how to describe the manageability capabilities of managed resources using WSDL documents.

An overarching requirement for MUWS is that it must use existing Internet infrastructure technologies and be compliant to the Web Services Architecture developed by the W3C WSA Working Group. Many current MUWS contributors were original contributors to W3C WSA activities - where management requirements and issues related to Web services were initially laid out.

Web services technologies offer substantial advantages in the context of distributed systems management. They are language and platform neutral. There is tremendous industry momentum and support across a wide variety of system offerings. They offer on-the-wire interoperability and resource discovery with extensibility, separation of interface and implementation, and a rich metadata mechanism. In short, they allow MUWS to concentrate on developing specifications for problems specific to distributed systems management that require this set of infrastructure capabilities, without reinventing them.

There are two activities MUWS is not engaging in.

MUWS is not creating Web services technologies outside the scope of management. Required platform technologies will be obtained from the Web services community in the form of existing standards, specifications, and best practices. Examples are security, policy, and reliable messaging.

MUWS is not creating a new model for representing managed resources. Rather, the requirement is that MUWS must be able to work with multiple, existing, domain-specific models. CIM from DTMF is the most prominent, but not the exclusive, model for this work. SNMP and OMI information models will also be considered. The objective is to bind a unifying layer on top of these various models to facilitate consistent management in a heterogeneous distributed environment. This on-the-wire unification of disparate resource models is the real power of Web services in the management context.

There is a variety of functionality requirements that the MUWS recommendation must address, including (note that these were preliminary as of July 2003):

  • Conformance and consistency with other management standards and existing managed resource models As noted above, MUWS will embrace and extend the usefulness of, versus replace, existing management models and frameworks. MUWS will allow Web service-based access to managed resources already instrumented for existing management technologies. This increases the number of managers that can manage these resources without requiring the managers to support each management technology independently. In particular, MUWS is coordinating with DMTF, GGF, and the W3C. The Distributed Management Task Force (DMTF) offers management models (CIM) that will need to be described as manageability interfaces in WSDL. The Global Grid Forum (GGF) is working on solving dynamic resource sharing and provisioning problems. The GGF Open Grid Services Infrastructure (OGSI) defines how to represent and access IT resources as stateful grid services. OGSI also has the Common Manageability Model (CMM) working group, which focuses on representing manageability using grid services. There is much synergy and experience to be shared among MUWS, OGSI and CMM. The WSDM TC is working with other OASIS technical committees, including the security TC and provisioning TC.
  • Wide scope of manageable resource support:
    MUWS will support the development of manageability interfaces for hardware and software resources, both physical and logical. A wide variety of management capabilities will be supported, including life cycle state and control, monitoring, and configuration management. Manageable resources can support these capabilities selectively and incrementally.
  • Distributed management: MUWS must enable management of highly distributed environments, including supporting occasionally connected resources, resource aggregation, hierarchical managers, and multiple managers.
  • Variety of message exchange patterns: Supporting both synchronous and asynchronous communication methods, one-way and two-way messages, and initiation of communication from either manager or managed resource (e.g. a manager queries for information using request-reply model, managed resource provides one-way event notification).
  • Discovery: MUWS manageable resources will be described through WSDL and XML Schema, making these resources discoverable like any other Web service.
  • Secure: MUWS will enable secure communication between manager and managed resources, and is being explicitly defined to work across enterprise boundaries to support cross-organization management scenarios. One of the advantages of relying on Web services as a foundation is that MUWS doesn't need to explicitly solve this problem. Rather, it can reference security technologies and standards already available for Web services.

    In addition to these functional requirements, MUWS must also enable interoperability, extensibility, scalability, usability, efficiency, and internationalization.

    Management of Web Services (MOWS)
    The WSDM MOWS activity is developing a model of a Web service as a manageable resource. The model requirements are being primarily driven by a set of management tasks that should be supported. These management tasks are numerous and include such varied activities as service metering, auditing, billing, performance profiling, SLA management, problem detection, root cause failure diagnosis, service deployment, and life cycle management. These management functionality requirements are the ultimate drivers of model requirements - they will highlight what Web service characteristics and management methods must be exposed to a manager. Explicit in MOWS is the requirement that it be described and accessible in a way consistent with MUWS.

    Like MUWS activities, MOWS aims to build on existing model frameworks (such as DMTF CIM) in its definition of the management model of a Web service, rather than reinventing a general managed resource object model scheme.

    In order to enable the expected management application use cases, the MOWS model will include (note: these were preliminary as of July 2003):

  • Identification: Each modeled Web service must have a unique identification, including a notion of version.
  • Life cycle/State: Expose the current state of a service and permit lifecycle management including the ability to start and stop a service.
  • Metrics: Must expose key operational metrics of a Web service, at the operation level, including such metrics as response time and throughput.
  • Configuration: Will support the ability to make specific configuration changes to a deployed Web service.
  • Relationships: Must permit the expression of relationships between and among Web services and other IT architectural elements and systems.
  • Types of resources: Clearly, Web services are manageable resources in the MOWS model, but additional considerations are being made for modeling the scope in which a given service is being leveraged - individual, composite, part of a long-running business process.
  • Extensibility: The model must be extensible and must permit discovery of supported management functionality in a given model instantiation.
  • Change description and notification: Will support the description of versions of Web services and notification of a change or impending change to the service interface or implementation.

    Conclusion
    The adoption of Web services technologies in the foundation of SOA implementations is well underway in the enterprise. This architectural approach to IT system design brings substantial and well-documented business benefits. Consistant with historical behavior patterns, management is a secondary consideration in many of these implementations.

    But in this architectural iteration, the costs associated with relegating management to the back seat are substantially higher than they have ever been. The nature of the SOA - complex system interdependencies - tends to magnify and spread system problems. What may have been a contained issue in a monolithic application environment, affecting a limited set of systems or business processes, can be a far-reaching and far more costly problem in an SOA environment.

    The initial fruits of these labors are due in January 2004 as the Web Services Distributed Management v1.0 Specification.

  • More Stories By Heather Kreger

    Heather Kreger is IBM’s lead architect for SOA Standards in the IBM Software Group, with 15 years of standards experience. She has led the development of standards for Web services, Management and Java in numerous industry standards groups including W3C, OASIS, DMTF, and The Open Group. Heather is the author of numerous articles and specifications, as well as the book “Java and JMX, Building Manageable Systems,” and most recently was co-editor of “Navigating the SOA Open Standards Landscape Around Architecture.”

    Comments (0)

    Share your thoughts on this story.

    Add your comment
    You must be signed in to add a comment. Sign-in | Register

    In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.


    @MicroservicesExpo Stories
    As DevOps methodologies expand their reach across the enterprise, organizations face the daunting challenge of adapting related cloud strategies to ensure optimal alignment, from managing complexity to ensuring proper governance. How can culture, automation, legacy apps and even budget be reexamined to enable this ongoing shift within the modern software factory? In her Day 2 Keynote at @DevOpsSummit at 21st Cloud Expo, Aruna Ravichandran, VP, DevOps Solutions Marketing, CA Technologies, was jo...
    You know you need the cloud, but you’re hesitant to simply dump everything at Amazon since you know that not all workloads are suitable for cloud. You know that you want the kind of ease of use and scalability that you get with public cloud, but your applications are architected in a way that makes the public cloud a non-starter. You’re looking at private cloud solutions based on hyperconverged infrastructure, but you’re concerned with the limits inherent in those technologies.
    The nature of test environments is inherently temporary—you set up an environment, run through an automated test suite, and then tear down the environment. If you can reduce the cycle time for this process down to hours or minutes, then you may be able to cut your test environment budgets considerably. The impact of cloud adoption on test environments is a valuable advancement in both cost savings and agility. The on-demand model takes advantage of public cloud APIs requiring only payment for t...
    The cloud era has reached the stage where it is no longer a question of whether a company should migrate, but when. Enterprises have embraced the outsourcing of where their various applications are stored and who manages them, saving significant investment along the way. Plus, the cloud has become a defining competitive edge. Companies that fail to successfully adapt risk failure. The media, of course, continues to extol the virtues of the cloud, including how easy it is to get there. Migrating...
    For DevOps teams, the concepts behind service-oriented architecture (SOA) are nothing new. A style of software design initially made popular in the 1990s, SOA was an alternative to a monolithic application; essentially a collection of coarse-grained components that communicated with each other. Communication would involve either simple data passing or two or more services coordinating some activity. SOA served as a valid approach to solving many architectural problems faced by businesses, as app...
    It has never been a better time to be a developer! Thanks to cloud computing, deploying our applications is much easier than it used to be. How we deploy our apps continues to evolve thanks to cloud hosting, Platform-as-a-Service (PaaS), and now Function-as-a-Service. FaaS is the concept of serverless computing via serverless architectures. Software developers can leverage this to deploy an individual "function", action, or piece of business logic. They are expected to start within milliseconds...
    Some journey to cloud on a mission, others, a deadline. Change management is useful when migrating to public, private or hybrid cloud environments in either case. For most, stakeholder engagement peaks during the planning and post migration phases of a project. Legacy engagements are fairly direct: projects follow a linear progression of activities (the “waterfall” approach) – change managers and application coders work from the same functional and technical requirements. Enablement and develo...
    Gone are the days when application development was the daunting task of the highly skilled developers backed with strong IT skills, low code application development has democratized app development and empowered a new generation of citizen developers. There was a time when app development was in the domain of people with complex coding and technical skills. We called these people by various names like programmers, coders, techies, and they usually worked in a world oblivious of the everyday pri...
    While some developers care passionately about how data centers and clouds are architected, for most, it is only the end result that matters. To the majority of companies, technology exists to solve a business problem, and only delivers value when it is solving that problem. 2017 brings the mainstream adoption of containers for production workloads. In his session at 21st Cloud Expo, Ben McCormack, VP of Operations at Evernote, discussed how data centers of the future will be managed, how the p...
    From manual human effort the world is slowly paving its way to a new space where most process are getting replaced with tools and systems to improve efficiency and bring down operational costs. Automation is the next big thing and low code platforms are fueling it in a significant way. The Automation era is here. We are in the fast pace of replacing manual human efforts with machines and processes. In the world of Information Technology too, we are linking disparate systems, softwares and tool...
    DevOps is good for organizations. According to the soon to be released State of DevOps Report high-performing IT organizations are 2X more likely to exceed profitability, market share, and productivity goals. But how do they do it? How do they use DevOps to drive value and differentiate their companies? We recently sat down with Nicole Forsgren, CEO and Chief Scientist at DORA (DevOps Research and Assessment) and lead investigator for the State of DevOps Report, to discuss the role of measure...
    Is advanced scheduling in Kubernetes achievable?Yes, however, how do you properly accommodate every real-life scenario that a Kubernetes user might encounter? How do you leverage advanced scheduling techniques to shape and describe each scenario in easy-to-use rules and configurations? In his session at @DevOpsSummit at 21st Cloud Expo, Oleg Chunikhin, CTO at Kublr, answered these questions and demonstrated techniques for implementing advanced scheduling. For example, using spot instances and co...
    DevOps is under attack because developers don’t want to mess with infrastructure. They will happily own their code into production, but want to use platforms instead of raw automation. That’s changing the landscape that we understand as DevOps with both architecture concepts (CloudNative) and process redefinition (SRE). Rob Hirschfeld’s recent work in Kubernetes operations has led to the conclusion that containers and related platforms have changed the way we should be thinking about DevOps and...
    "As we've gone out into the public cloud we've seen that over time we may have lost a few things - we've lost control, we've given up cost to a certain extent, and then security, flexibility," explained Steve Conner, VP of Sales at Cloudistics,in this SYS-CON.tv interview at 20th Cloud Expo, held June 6-8, 2017, at the Javits Center in New York City, NY.
    These days, APIs have become an integral part of the digital transformation journey for all enterprises. Every digital innovation story is connected to APIs . But have you ever pondered over to know what are the source of these APIs? Let me explain - APIs sources can be varied, internal or external, solving different purposes, but mostly categorized into the following two categories. Data lakes is a term used to represent disconnected but relevant data that are used by various business units wit...
    With continuous delivery (CD) almost always in the spotlight, continuous integration (CI) is often left out in the cold. Indeed, it's been in use for so long and so widely, we often take the model for granted. So what is CI and how can you make the most of it? This blog is intended to answer those questions. Before we step into examining CI, we need to look back. Software developers often work in small teams and modularity, and need to integrate their changes with the rest of the project code b...
    "I focus on what we are calling CAST Highlight, which is our SaaS application portfolio analysis tool. It is an extremely lightweight tool that can integrate with pretty much any build process right now," explained Andrew Siegmund, Application Migration Specialist for CAST, in this SYS-CON.tv interview at 21st Cloud Expo, held Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA.
    "Cloud4U builds software services that help people build DevOps platforms for cloud-based software and using our platform people can draw a picture of the system, network, software," explained Kihyeon Kim, CEO and Head of R&D at Cloud4U, in this SYS-CON.tv interview at 21st Cloud Expo, held Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA.
    Kubernetes is an open source system for automating deployment, scaling, and management of containerized applications. Kubernetes was originally built by Google, leveraging years of experience with managing container workloads, and is now a Cloud Native Compute Foundation (CNCF) project. Kubernetes has been widely adopted by the community, supported on all major public and private cloud providers, and is gaining rapid adoption in enterprises. However, Kubernetes may seem intimidating and complex ...
    DevOps is often described as a combination of technology and culture. Without both, DevOps isn't complete. However, applying the culture to outdated technology is a recipe for disaster; as response times grow and connections between teams are delayed by technology, the culture will die. A Nutanix Enterprise Cloud has many benefits that provide the needed base for a true DevOps paradigm. In their Day 3 Keynote at 20th Cloud Expo, Chris Brown, a Solutions Marketing Manager at Nutanix, and Mark Lav...