Welcome!

Microservices Expo Authors: Elizabeth White, Liz McMillan, Derek Weeks, Mehdi Daoudi, Don MacVittie

Related Topics: Microservices Expo

Microservices Expo: Article

Considering the SOA Reference Model

Part 2 - Design Pillars

(SYS-CON Media) - The main drivers for SOA-based architectures are to facilitate the manageable growth of large-scale enterprise systems, to facilitate Internet-scale provisioning and the use of services, and to reduce the cost of organization-to-organization cooperation - SOA RM

When approaching a SOA implementation, I would like to consider two fundamental questions that many developers ask:
1)  What's the difference between service-oriented and service-based architectures?
2)  What special architecture elements are defined by the SOA RM?

In my opinion, the answer to the first is in the difference between the words oriented and based. I believe that smart IT organizations offer a lot of services already because the technical benefits of services have been well known for a while. However, the applications based on these services are still monolithic and don't provide convenient ways to implement business change. SOA, by contrast, elevates services to the level of first-class citizens by associating them (well, some of them) with business responsibilities. That is, business tasks can finally surface in the form of aggregated services that are open to business changes by design. SOA RM stresses a new role and meaning for services and outlines that:
1)  "The central focus of Service Oriented Architecture is the task or business function" versus both IT task/function and business operation automation task. This means that SOA primarily targets actual business services and processes instead of just improving existing (obsolete or legacy) applications.
2)  SOA service may be MUCH richer than a technical service because SOA service is supposed to run in the "execution context." The latter is defined as the set of technical and business elements that form a path between those with needs and those with capabilities. The "business elements" open absolutely new sphere of conditions, particularly business conditions that a regular technical service doesn't deal with.
3)  SOA "reflects business-motivating considerations" expressed "in service descriptions and service interfaces", where the service description can be expressed in the form of a contract. The service contract includes policies that "MAY also express business-oriented policies - such as hours of business, return policies, and so on".

If you elaborate on the business aspect of SOA, you'll get a much different picture of the service realm with different purposes, use, governance, maintenance, and, finally, architectural concept than we have in a "regular" service-based architecture. Besides these, I don't see any real differences. The service serves; the questions are how, where, why, when, to whom, and if it's needed after all.

The second question is trickier. The SOA RM standard doesn't articulate much on architecture elements. Instead, it defines SOA services and their relationship in new way that allows for addressing actual business services and processes by assigning business responsibilities to the services directly. That is, the standard defines what an IT service has to be oriented to in order to provide for IT agility to the business. I understand that the intention of SOA RM is giving us the definitions of the service and the "associates" that require a business service-oriented infrastructure. This infrastructure is the very architecture meant by the reference model.

So, since we're going to discuss SOA design pillars, let's list some commonly used elements of SOA infrastructure that include:

  • a SOA Service Communication Framework
  • a SOA Service Composition Framework
  • a SOA Service Transaction Framework
  • a SOA Service Security Framework
  • User Interfaces (including human UI) and
  • a SOA Service Component Adapter Framework
Elements of SOA infrastructure are reviewed in many technical publications; we will use them for references without deeper discussion.

Going forward, let's notice that some pillars relate to the SOA service design while others relate to the service interactions with other services and its consumers. Pillars 1 to 5 have ordered dependencies, the rest are applicable to the entire design process. Plus we'll use terms that have been defined and discussed in part one of this article, so you might look through it.

Service Design Pillars
Pillar 1
Considering Business Services selected for the implementation in SOA identify technical "vertical" and "horizontal" services. A vertical service implements one or a few business tasks while a horizontal service implements a supporting SOA infrastructure task, e.g. a Service Repository or Service contract management function. Traditional technical services like security, login, and the like also fit into the horizontal services category.

While the principle of separation of concerns is fully applicable here, the tasks implemented by the same vertical service have to provide "cohesive collections of behaviors" from the business perspective. Plus technical services have to be reasonably coarse-grain in their interfaces (to minimize the network round-trips and future modernizations in production caused by new technologies) but still provide flexibility. Nevertheless, one has to expect a certain amount of exploration and refactoring in this area of the service lifecycle.

When identifying horizontal services, first consider scalability, reliability, performance, expendability, and manageability - all together; consider programming convenience second. Services that are too generic usually don't provide the best performance just as too detailed ones cause problems in aggregation, management, and maintenance in light of frequent changes in the execution context.

Pillar 2
Considering business units of work, identify business data and its flows. Take into account two orthogonal sets of requirements - the presentation layer and the persistence layer requirements. Some units of work can require compositions of tem, so watch for a clear information transition with layered data format mappings. Also consider combinations of the business and just the technical data in the flows. That is, define metadata for the business and technical data upfront and operate at the metadata level (UML and WS-CDL can help).

Metadata is an important SOA element. Metadata provides the necessary data abstraction and allows for a data-quality controls. An XML Schema is one possible metadata definition and control instrument. This doesn't mean that only XML format is recommended. In some cases, especially for synchronous short-running transactions, programmatic Data Objects may be used for performance improvement. However, the Data Objects better be derived or generated from the XML Schema, which would provide metadata enforcement (related utilities are available in Java and C# and are known as XML binding).

Pillar 3
Define and lock down service interfaces and interfaces to the service resources such as persistence storage. Design the interface as business-oriented coarse-grained so your consumers can avoid frequent refactoring.

Best practices recommend avoiding RPC-like style in service interfaces. Web Services, RPC encoding, and SOAP-encoded arrays, in particular, seriously restrict service interoperability; consider a document/literal style instead. For non-XML cases, use a container style of interface to pass data structures and "exchanged" operations (aka commands in the Command Pattern).

It's important to design service interfaces with an eye to security, transaction, manageability, error handling, versioning, and interoperability from the beginning. Adding any of these things later is either extremely expensive (in effort, time, and money) or practically impossible.

For example, security is very important these days because of multiple technology threats and industrial/corporate security policies. Transaction support is dictated by its potential participation in service compositions. Manageability and error handling are core to any business service and process. Interoperability is one of the most critical factors in the SOA service lifecycle. That is, use of specific programming platform constructs can significantly reduce the utility of a service and lead to its refusal.

In spite of any ideas about keeping the service interface immutable, in real life variations in service invocation always happen over time. Anyway, a service interface isn't the only one to constitute a SOA service; there are also possibilities of SOA service behavior changes that affect service consumers. That is, the service has to be versioned. The versioning may be expressed via the interface and/or provided by a service registry. Only one condition has to be preserved in this regard - no service parts including the interface may be versioned separately from the service itself for service consumers.

Pillar 4
A service access protocol may be considered a part of the service interface definition or a separate issue. The spectrum of communication protocols used in SOA is wide including TCP/IP, HTTP/HTTPS, SMTP, and IIOP.

The designer has to choose communication protocol(s) for every SOA service. When considering a protocol, one has to take into account: 1) invocation model - synchronous or asynchronous; 2) the amount of transferred data; 3) performance; 4) extendibility; 5) scalability; 6) reliability; 7) transaction; and 8) manageability.

Protocols are also important for communicating with service resources. It's better to use standardized protocols where possible. For example, current best practice refers to the Web Services Invocation Framework (WSIF) as a binding bridge between Web Service's WSDL and a particular service resource protocol while Java components rely on JBI.

The right protocol is a balanced choice. For example, HTTP/SOAP-based protocol is slower than IIOP but it's more extendible; TCP/IP is more reliable than UDP but it's slower. Don't forget that one service may have more than one communication channel. For example, the same session EJB can be reached through RMI-over-IIOP and SOAP-over-HTTP/S.

The pioneers of SOA implementations have recommend doing a "protocol performance POC" for particular kinds of tasks. That is, the earlier in the design process a performance test gives the results the better for the entire project because refactoring in this area means costly redesign from the scratch.

Pillar 5
Define the SOA Service contract and Execution Context. One of the most important parts of the SOA service design is the definition of the SOA service contract template. While SOA RM simply says that "a contract...represents an agreement by two or more parties. Like policies, agreements are also about the conditions of using a service; they may constrain the expected real-world effects of using a service," the SOA service contract may be relatively complex. The contract, preferably written in XML, defines among other things:

  • runtime policies/constraints
  • runtime functional service parameters exposed to the service consumers; these parameters can be monitored and controlled
  • non-functional service parameters like scheduled service availability
  • business functional characteristics promised to service consumers like gathering audit information on service invocation events (the characteristics should be evidential and measurable)
  • all service interfaces and communication protocols exposed to a particular consumer
  • definitions or references to the Change Control procedures related to the service
  • service version information
  • references to the external (to your organization) SOA services invoked by the service. It's important for the business to know what other SOA services are involved because some political and business considerations may cause certain constraints on such involvement
  • some custom or business domain specific parameters if needed
The content of a SOA service contract isn't standardized. However it's the only document that's used by a cautious consumer to decide whether your service is the one that meets his needs. It is also the document that defines your liability as a service provider to your consumers.

Runtime policies better be derived from the SOA service contract. The policies have to follow standards wherever possible. As SOA RM defines it, "A service contract is a measurable assertion that governs the requirements and expectations of two or more parties. Unlike policy enforcement, which is usually the responsibility of the policy owner, contract enforcement may involve resolving disputes between the parties to the contract. The resolution of such disputes may involve appeals to higher authorities."

The SOA service implementation process treats the service contract as an objective set of development tasks. Not all implementation tasks ought to be expressed in the service contract, but only those that get visibility to the service consumers. Other tasks are the subject of regular application requirements.

Another fundamental characteristic of a SOA service is the execution context. According to SOA RM, the "Execution context is central to many aspects of a service interaction." The context includes concepts such as the SOA service contract, and behavioural and information models. The same service can be viewed and act differently in different execution contexts. For example, a logging service may store data "in the clear" when serving load-balancing calculations but it has to encrypt data for financial risk calculations (i.e., financial regulations constitute the execution context).

Pillar 6
Develop for security. SOA security isn't included in the SOA RM. Nonetheless it's obvious that the model's "real-world effects" can't survive without a business trust, i.e., without service security. We don't recommend developing a special implementation for each service. Instead use corporate security services at the level of service communication and service components, e.g., security policy syndication services.

Since a lot of security threats come from inside an organization, perimeter protection (external firewalls) isn't enough. We can outline the minimal SOA security requirements as 1) end-user bi-directional authentication with the service, and 2) inter-service bi-directional authentication. End-user and inter-service authorizations are much desired but may be optional in particular cases (e.g., a consumer may access everything after he's confirmed as a legitimate user). Other security measures are optional as well and depend on the nature of business data and processes.

Traditional security means like PKI, Kerberos, SSL, two-way SSL, user name and password tokens serve well in SOA. The service-specific means relate to the XML security standards. They include XML Digital Signature, XML standards for PKI (XKMS), SAML (for propagating the subject and its credentials), XACML (for transferring security policies and policy validation controls), and other encryption techniques applied to the exchange messages and communication channels. The choice of security methods to be applied depends on corporate and industry policies and regulations.

The takeaway here is that whatever security means ought to be implemented, they have to be considered in the design from the beginning. It's almost impossible or extremely expensive to add security in later phases of development.

Pillar 7
Develop to support transactions. SOA RM recognizes the importance of service transaction as "higher-order attributes of services' process models" but doesn't detail the topic. In practice, SOA uses commonly accepted standards for transaction management on different platforms and adds cross-platform transaction management standards. Transactions can be used as in an aggregated service (choreographed aggregation) as in an orchestration of services reflecting fragments of Business Process Management (BPM). The orchestration is typically based on the BPEL standard that introduces short-running and long-running transactions.

A short-running transaction is usually a synchronous RPC-type invocation running according to the known rules of transaction isolation. Transaction standards for Web Services and messaging include WS-Transaction and WS Reliable Messaging (WS-RM). A short-running transaction is good for performance and commonly used for co-located services or in cases where failure is unlikely and error handling is simple. A long-running transaction is an asynchronous transaction where the transactional state may be persisted for long periods of time (days, months, years) and are restored after a certain event. This kind of transaction is quite useful in manual operations or human decision making in the automated business process. BPEL also introduces a transaction capable of "undoing" the mistaken results of the previous transaction by compensating them (like an overcharge refund).

Transactional behaviour has to be embedded in the SOA service from the design phase. This will help provide future service aggregation- and orchestration-based SOA solutions that might not be clear at the moment the service is initially designed.

Pillar 8
Design for interoperability. As mentioned, interoperability is a crucial and critical characteristic of a SOA. The SOA RM says, "The value of SOA is that it provides a simple scalable paradigm for organizing large networks of systems that require interoperability to realize the value inherent in the individual components."

The existing WS-I Basic Profile address most interoperability issues in Web Services. CORBA IDL mapping does the same for the services communicating via IIOP. Communication via HTTP is strictly defined and doesn't leave room for interoperability issues (except for JavaScript flavors) while SOAP-over-HTTP/S still has some mismatch problems in mapping to different programming languages.

The recommendation here is to put a lot of attention on the data types and naming convention policies (besides the protocols) on the both sides of the interface - for the requester and the provider components. The policies must enforce compatibility and interoperability. One possible mechanism for data type and naming space interoperability control is XML Schema or any other schemas (metadata) accepted by the requester and the provider.

Another best practice recommendation is the creation of a proxy (isolation mapping layer) around the interfaces. That is, the data formats passed through the service interface shouldn't be driven by internal data formats (for either requester or provider). If the evolution of either side causes data format changes, it's better to create object mapping in the proxy than to try to modify the interface.

Pillar 9
Use Rules for flexibility and easy-to-change implementation. Rule engines are just tools used in SOA infrastructure implementation and certainly not a part of the SOA RM. However, being combined with policies on one side and decision-making points on the other side, rule engines become quite powerful instruments for providing a high level of flexibility and quick adaptation to business changes. Leading SOA infrastructure/framework vendors include different rule engines into their business process design tools and Enterprise Service Bus solutions.

While rule engines are really convenient instruments, there's no compatibility between engines from different vendors; a rule engine locks you into a vendor. Anyway, not everything should be driven by rules - rule maintenance is expensive and requires special skills. That is, the designer and developer have to be reasonable in using rule engines.

For example, if the business logic introduces a point for choosing from a finite number of cases (while cases change seldom), using rules may be overkill. Another known risk in using rules is a potential rule contradiction. That is, a rule engine generally doesn't automatically validate if a new rule contradicts any existing rules and any permitted combinations of those rules. Without such validation, the new rule can bypass restrictive rules and expose prohibited information (e.g., a junior clerk gets access to a high-volume account). So when using a rule engine in SOA, proceed with caution.


More Stories By Michael Poulin

Michael Poulin works as an enterprise-level solution architect in the financial industry in the UK. He is a Sun Certified Architect for Java Technology, certified TOGAF Practitioner, and Licensed ZapThink SOA Architect. Michael specializes in distributed computing, SOA, and application security.

Comments (0)

Share your thoughts on this story.

Add your comment
You must be signed in to add a comment. Sign-in | Register

In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.


@MicroservicesExpo Stories
The dynamic nature of the cloud means that change is a constant when it comes to modern cloud-based infrastructure. Delivering modern applications to end users, therefore, is a constantly shifting challenge. Delivery automation helps IT Ops teams ensure that apps are providing an optimal end user experience over hybrid-cloud and multi-cloud environments, no matter what the current state of the infrastructure is. To employ a delivery automation strategy that reflects your business rules, making r...
As many know, the first generation of Cloud Management Platform (CMP) solutions were designed for managing virtual infrastructure (IaaS) and traditional applications. But that's no longer enough to satisfy evolving and complex business requirements. In his session at 21st Cloud Expo, Scott Davis, Embotics CTO, explored how next-generation CMPs ensure organizations can manage cloud-native and microservice-based application architectures, while also facilitating agile DevOps methodology. He expla...
The past few years have brought a sea change in the way applications are architected, developed, and consumed—increasing both the complexity of testing and the business impact of software failures. How can software testing professionals keep pace with modern application delivery, given the trends that impact both architectures (cloud, microservices, and APIs) and processes (DevOps, agile, and continuous delivery)? This is where continuous testing comes in. D
Modern software design has fundamentally changed how we manage applications, causing many to turn to containers as the new virtual machine for resource management. As container adoption grows beyond stateless applications to stateful workloads, the need for persistent storage is foundational - something customers routinely cite as a top pain point. In his session at @DevOpsSummit at 21st Cloud Expo, Bill Borsari, Head of Systems Engineering at Datera, explored how organizations can reap the bene...
"I focus on what we are calling CAST Highlight, which is our SaaS application portfolio analysis tool. It is an extremely lightweight tool that can integrate with pretty much any build process right now," explained Andrew Siegmund, Application Migration Specialist for CAST, in this SYS-CON.tv interview at 21st Cloud Expo, held Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA.
Admiral Calcote - also known as Lee Calcote (@lcalcote) or the Ginger Geek to his friends - gave a presentation entitled Characterizing and Contrasting Container Orchestrators at the 2016 All Day DevOps conference. Okay, he isn't really an admiral - nor does anyone call him that - but he used the title admiral to describe what container orchestrators do, relating it to an admiral directing a fleet of container ships. You could also say that they are like the conductor of an orchestra, directing...
The past few years have seen a huge increase in the amount of critical IT services that companies outsource to SaaS/IaaS/PaaS providers, be it security, storage, monitoring, or operations. Of course, along with any outsourcing to a service provider comes a Service Level Agreement (SLA) to ensure that the vendor is held financially responsible for any lapses in their service which affect the customer’s end users, and ultimately, their bottom line. SLAs can be very tricky to manage for a number ...
Our work, both with clients and with tools, has lead us to wonder how it is that organizations are handling compliance issues in the cloud. The big cloud vendors offer compliance for their infrastructure, but the shared responsibility model requires that you take certain steps to meet compliance requirements. Which lead us to start poking around a little more. We wanted to get a picture of what was available, and how it was being used. There is a lot of fluidity in this space, as in all things c...
Gaining visibility in today’s sprawling cloud infrastructure is complex and laborious, involving drilling down into tools offered by various cloud services providers. Enterprise IT organizations need smarter and effective tools at their disposal in order to address this pertinent problem. Gaining a 360 - degree view of the cloud costs requires collection and analysis of the cost data across all cloud infrastructures used inside an enterprise.
Some people are directors, managers, and administrators. Others are disrupters. Eddie Webb (@edwardawebb) is an IT Disrupter for Software Development Platforms at Liberty Mutual and was a presenter at the 2016 All Day DevOps conference. His talk, Organically DevOps: Building Quality and Security into the Software Supply Chain at Liberty Mutual, looked at Liberty Mutual's transformation to Continuous Integration, Continuous Delivery, and DevOps. For a large, heavily regulated industry, this task...
DevOps promotes continuous improvement through a culture of collaboration. But in real terms, how do you: Integrate activities across diverse teams and services? Make objective decisions with system-wide visibility? Use feedback loops to enable learning and improvement? With technology insights and real-world examples, in his general session at @DevOpsSummit, at 21st Cloud Expo, Andi Mann, Chief Technology Advocate at Splunk, explored how leading organizations use data-driven DevOps to clos...
The goal of Microservices is to improve software delivery speed and increase system safety as scale increases. Microservices being modular these are faster to change and enables an evolutionary architecture where systems can change, as the business needs change. Microservices can scale elastically and by being service oriented can enable APIs natively. Microservices also reduce implementation and release cycle time and enables continuous delivery. This paper provides a logical overview of the Mi...
The notion of improving operational efficiency is conspicuously absent from the healthcare debate - neither Obamacare nor the newly proposed GOP plan discusses the impact that a step-function improvement in efficiency could have on access to healthcare (through more capacity), quality of healthcare services (through reduced wait times for patients) or cost (through better utilization of scarce, expensive assets).
Gone are the days when application development was the daunting task of the highly skilled developers backed with strong IT skills, low code application development has democratized app development and empowered a new generation of citizen developers. There was a time when app development was in the domain of people with complex coding and technical skills. We called these people by various names like programmers, coders, techies, and they usually worked in a world oblivious of the everyday pri...
The “Digital Era” is forcing us to engage with new methods to build, operate and maintain applications. This transformation also implies an evolution to more and more intelligent applications to better engage with the customers, while creating significant market differentiators. In both cases, the cloud has become a key enabler to embrace this digital revolution. So, moving to the cloud is no longer the question; the new questions are HOW and WHEN. To make this equation even more complex, most ...
Some journey to cloud on a mission, others, a deadline. Change management is useful when migrating to public, private or hybrid cloud environments in either case. For most, stakeholder engagement peaks during the planning and post migration phases of a project. Legacy engagements are fairly direct: projects follow a linear progression of activities (the “waterfall” approach) – change managers and application coders work from the same functional and technical requirements. Enablement and develo...
SYS-CON Events announced today that Synametrics Technologies will exhibit at SYS-CON's 22nd International Cloud Expo®, which will take place on June 5-7, 2018, at the Javits Center in New York, NY. Synametrics Technologies is a privately held company based in Plainsboro, New Jersey that has been providing solutions for the developer community since 1997. Based on the success of its initial product offerings such as WinSQL, Xeams, SynaMan and Syncrify, Synametrics continues to create and hone in...
Kubernetes is an open source system for automating deployment, scaling, and management of containerized applications. Kubernetes was originally built by Google, leveraging years of experience with managing container workloads, and is now a Cloud Native Compute Foundation (CNCF) project. Kubernetes has been widely adopted by the community, supported on all major public and private cloud providers, and is gaining rapid adoption in enterprises. However, Kubernetes may seem intimidating and complex ...
You know you need the cloud, but you’re hesitant to simply dump everything at Amazon since you know that not all workloads are suitable for cloud. You know that you want the kind of ease of use and scalability that you get with public cloud, but your applications are architected in a way that makes the public cloud a non-starter. You’re looking at private cloud solutions based on hyperconverged infrastructure, but you’re concerned with the limits inherent in those technologies.
For DevOps teams, the concepts behind service-oriented architecture (SOA) are nothing new. A style of software design initially made popular in the 1990s, SOA was an alternative to a monolithic application; essentially a collection of coarse-grained components that communicated with each other. Communication would involve either simple data passing or two or more services coordinating some activity. SOA served as a valid approach to solving many architectural problems faced by businesses, as app...