Microservices Expo Authors: Elizabeth White, Mehdi Daoudi, Pat Romanski, Flint Brenton, Gordon Haff

Related Topics: Microservices Expo

Microservices Expo: Article

Considering the SOA Reference Model

Part 2 - Design Pillars

(SYS-CON Media) - The main drivers for SOA-based architectures are to facilitate the manageable growth of large-scale enterprise systems, to facilitate Internet-scale provisioning and the use of services, and to reduce the cost of organization-to-organization cooperation - SOA RM

When approaching a SOA implementation, I would like to consider two fundamental questions that many developers ask:
1)  What's the difference between service-oriented and service-based architectures?
2)  What special architecture elements are defined by the SOA RM?

In my opinion, the answer to the first is in the difference between the words oriented and based. I believe that smart IT organizations offer a lot of services already because the technical benefits of services have been well known for a while. However, the applications based on these services are still monolithic and don't provide convenient ways to implement business change. SOA, by contrast, elevates services to the level of first-class citizens by associating them (well, some of them) with business responsibilities. That is, business tasks can finally surface in the form of aggregated services that are open to business changes by design. SOA RM stresses a new role and meaning for services and outlines that:
1)  "The central focus of Service Oriented Architecture is the task or business function" versus both IT task/function and business operation automation task. This means that SOA primarily targets actual business services and processes instead of just improving existing (obsolete or legacy) applications.
2)  SOA service may be MUCH richer than a technical service because SOA service is supposed to run in the "execution context." The latter is defined as the set of technical and business elements that form a path between those with needs and those with capabilities. The "business elements" open absolutely new sphere of conditions, particularly business conditions that a regular technical service doesn't deal with.
3)  SOA "reflects business-motivating considerations" expressed "in service descriptions and service interfaces", where the service description can be expressed in the form of a contract. The service contract includes policies that "MAY also express business-oriented policies - such as hours of business, return policies, and so on".

If you elaborate on the business aspect of SOA, you'll get a much different picture of the service realm with different purposes, use, governance, maintenance, and, finally, architectural concept than we have in a "regular" service-based architecture. Besides these, I don't see any real differences. The service serves; the questions are how, where, why, when, to whom, and if it's needed after all.

The second question is trickier. The SOA RM standard doesn't articulate much on architecture elements. Instead, it defines SOA services and their relationship in new way that allows for addressing actual business services and processes by assigning business responsibilities to the services directly. That is, the standard defines what an IT service has to be oriented to in order to provide for IT agility to the business. I understand that the intention of SOA RM is giving us the definitions of the service and the "associates" that require a business service-oriented infrastructure. This infrastructure is the very architecture meant by the reference model.

So, since we're going to discuss SOA design pillars, let's list some commonly used elements of SOA infrastructure that include:

  • a SOA Service Communication Framework
  • a SOA Service Composition Framework
  • a SOA Service Transaction Framework
  • a SOA Service Security Framework
  • User Interfaces (including human UI) and
  • a SOA Service Component Adapter Framework
Elements of SOA infrastructure are reviewed in many technical publications; we will use them for references without deeper discussion.

Going forward, let's notice that some pillars relate to the SOA service design while others relate to the service interactions with other services and its consumers. Pillars 1 to 5 have ordered dependencies, the rest are applicable to the entire design process. Plus we'll use terms that have been defined and discussed in part one of this article, so you might look through it.

Service Design Pillars
Pillar 1
Considering Business Services selected for the implementation in SOA identify technical "vertical" and "horizontal" services. A vertical service implements one or a few business tasks while a horizontal service implements a supporting SOA infrastructure task, e.g. a Service Repository or Service contract management function. Traditional technical services like security, login, and the like also fit into the horizontal services category.

While the principle of separation of concerns is fully applicable here, the tasks implemented by the same vertical service have to provide "cohesive collections of behaviors" from the business perspective. Plus technical services have to be reasonably coarse-grain in their interfaces (to minimize the network round-trips and future modernizations in production caused by new technologies) but still provide flexibility. Nevertheless, one has to expect a certain amount of exploration and refactoring in this area of the service lifecycle.

When identifying horizontal services, first consider scalability, reliability, performance, expendability, and manageability - all together; consider programming convenience second. Services that are too generic usually don't provide the best performance just as too detailed ones cause problems in aggregation, management, and maintenance in light of frequent changes in the execution context.

Pillar 2
Considering business units of work, identify business data and its flows. Take into account two orthogonal sets of requirements - the presentation layer and the persistence layer requirements. Some units of work can require compositions of tem, so watch for a clear information transition with layered data format mappings. Also consider combinations of the business and just the technical data in the flows. That is, define metadata for the business and technical data upfront and operate at the metadata level (UML and WS-CDL can help).

Metadata is an important SOA element. Metadata provides the necessary data abstraction and allows for a data-quality controls. An XML Schema is one possible metadata definition and control instrument. This doesn't mean that only XML format is recommended. In some cases, especially for synchronous short-running transactions, programmatic Data Objects may be used for performance improvement. However, the Data Objects better be derived or generated from the XML Schema, which would provide metadata enforcement (related utilities are available in Java and C# and are known as XML binding).

Pillar 3
Define and lock down service interfaces and interfaces to the service resources such as persistence storage. Design the interface as business-oriented coarse-grained so your consumers can avoid frequent refactoring.

Best practices recommend avoiding RPC-like style in service interfaces. Web Services, RPC encoding, and SOAP-encoded arrays, in particular, seriously restrict service interoperability; consider a document/literal style instead. For non-XML cases, use a container style of interface to pass data structures and "exchanged" operations (aka commands in the Command Pattern).

It's important to design service interfaces with an eye to security, transaction, manageability, error handling, versioning, and interoperability from the beginning. Adding any of these things later is either extremely expensive (in effort, time, and money) or practically impossible.

For example, security is very important these days because of multiple technology threats and industrial/corporate security policies. Transaction support is dictated by its potential participation in service compositions. Manageability and error handling are core to any business service and process. Interoperability is one of the most critical factors in the SOA service lifecycle. That is, use of specific programming platform constructs can significantly reduce the utility of a service and lead to its refusal.

In spite of any ideas about keeping the service interface immutable, in real life variations in service invocation always happen over time. Anyway, a service interface isn't the only one to constitute a SOA service; there are also possibilities of SOA service behavior changes that affect service consumers. That is, the service has to be versioned. The versioning may be expressed via the interface and/or provided by a service registry. Only one condition has to be preserved in this regard - no service parts including the interface may be versioned separately from the service itself for service consumers.

Pillar 4
A service access protocol may be considered a part of the service interface definition or a separate issue. The spectrum of communication protocols used in SOA is wide including TCP/IP, HTTP/HTTPS, SMTP, and IIOP.

The designer has to choose communication protocol(s) for every SOA service. When considering a protocol, one has to take into account: 1) invocation model - synchronous or asynchronous; 2) the amount of transferred data; 3) performance; 4) extendibility; 5) scalability; 6) reliability; 7) transaction; and 8) manageability.

Protocols are also important for communicating with service resources. It's better to use standardized protocols where possible. For example, current best practice refers to the Web Services Invocation Framework (WSIF) as a binding bridge between Web Service's WSDL and a particular service resource protocol while Java components rely on JBI.

The right protocol is a balanced choice. For example, HTTP/SOAP-based protocol is slower than IIOP but it's more extendible; TCP/IP is more reliable than UDP but it's slower. Don't forget that one service may have more than one communication channel. For example, the same session EJB can be reached through RMI-over-IIOP and SOAP-over-HTTP/S.

The pioneers of SOA implementations have recommend doing a "protocol performance POC" for particular kinds of tasks. That is, the earlier in the design process a performance test gives the results the better for the entire project because refactoring in this area means costly redesign from the scratch.

Pillar 5
Define the SOA Service contract and Execution Context. One of the most important parts of the SOA service design is the definition of the SOA service contract template. While SOA RM simply says that "a contract...represents an agreement by two or more parties. Like policies, agreements are also about the conditions of using a service; they may constrain the expected real-world effects of using a service," the SOA service contract may be relatively complex. The contract, preferably written in XML, defines among other things:

  • runtime policies/constraints
  • runtime functional service parameters exposed to the service consumers; these parameters can be monitored and controlled
  • non-functional service parameters like scheduled service availability
  • business functional characteristics promised to service consumers like gathering audit information on service invocation events (the characteristics should be evidential and measurable)
  • all service interfaces and communication protocols exposed to a particular consumer
  • definitions or references to the Change Control procedures related to the service
  • service version information
  • references to the external (to your organization) SOA services invoked by the service. It's important for the business to know what other SOA services are involved because some political and business considerations may cause certain constraints on such involvement
  • some custom or business domain specific parameters if needed
The content of a SOA service contract isn't standardized. However it's the only document that's used by a cautious consumer to decide whether your service is the one that meets his needs. It is also the document that defines your liability as a service provider to your consumers.

Runtime policies better be derived from the SOA service contract. The policies have to follow standards wherever possible. As SOA RM defines it, "A service contract is a measurable assertion that governs the requirements and expectations of two or more parties. Unlike policy enforcement, which is usually the responsibility of the policy owner, contract enforcement may involve resolving disputes between the parties to the contract. The resolution of such disputes may involve appeals to higher authorities."

The SOA service implementation process treats the service contract as an objective set of development tasks. Not all implementation tasks ought to be expressed in the service contract, but only those that get visibility to the service consumers. Other tasks are the subject of regular application requirements.

Another fundamental characteristic of a SOA service is the execution context. According to SOA RM, the "Execution context is central to many aspects of a service interaction." The context includes concepts such as the SOA service contract, and behavioural and information models. The same service can be viewed and act differently in different execution contexts. For example, a logging service may store data "in the clear" when serving load-balancing calculations but it has to encrypt data for financial risk calculations (i.e., financial regulations constitute the execution context).

Pillar 6
Develop for security. SOA security isn't included in the SOA RM. Nonetheless it's obvious that the model's "real-world effects" can't survive without a business trust, i.e., without service security. We don't recommend developing a special implementation for each service. Instead use corporate security services at the level of service communication and service components, e.g., security policy syndication services.

Since a lot of security threats come from inside an organization, perimeter protection (external firewalls) isn't enough. We can outline the minimal SOA security requirements as 1) end-user bi-directional authentication with the service, and 2) inter-service bi-directional authentication. End-user and inter-service authorizations are much desired but may be optional in particular cases (e.g., a consumer may access everything after he's confirmed as a legitimate user). Other security measures are optional as well and depend on the nature of business data and processes.

Traditional security means like PKI, Kerberos, SSL, two-way SSL, user name and password tokens serve well in SOA. The service-specific means relate to the XML security standards. They include XML Digital Signature, XML standards for PKI (XKMS), SAML (for propagating the subject and its credentials), XACML (for transferring security policies and policy validation controls), and other encryption techniques applied to the exchange messages and communication channels. The choice of security methods to be applied depends on corporate and industry policies and regulations.

The takeaway here is that whatever security means ought to be implemented, they have to be considered in the design from the beginning. It's almost impossible or extremely expensive to add security in later phases of development.

Pillar 7
Develop to support transactions. SOA RM recognizes the importance of service transaction as "higher-order attributes of services' process models" but doesn't detail the topic. In practice, SOA uses commonly accepted standards for transaction management on different platforms and adds cross-platform transaction management standards. Transactions can be used as in an aggregated service (choreographed aggregation) as in an orchestration of services reflecting fragments of Business Process Management (BPM). The orchestration is typically based on the BPEL standard that introduces short-running and long-running transactions.

A short-running transaction is usually a synchronous RPC-type invocation running according to the known rules of transaction isolation. Transaction standards for Web Services and messaging include WS-Transaction and WS Reliable Messaging (WS-RM). A short-running transaction is good for performance and commonly used for co-located services or in cases where failure is unlikely and error handling is simple. A long-running transaction is an asynchronous transaction where the transactional state may be persisted for long periods of time (days, months, years) and are restored after a certain event. This kind of transaction is quite useful in manual operations or human decision making in the automated business process. BPEL also introduces a transaction capable of "undoing" the mistaken results of the previous transaction by compensating them (like an overcharge refund).

Transactional behaviour has to be embedded in the SOA service from the design phase. This will help provide future service aggregation- and orchestration-based SOA solutions that might not be clear at the moment the service is initially designed.

Pillar 8
Design for interoperability. As mentioned, interoperability is a crucial and critical characteristic of a SOA. The SOA RM says, "The value of SOA is that it provides a simple scalable paradigm for organizing large networks of systems that require interoperability to realize the value inherent in the individual components."

The existing WS-I Basic Profile address most interoperability issues in Web Services. CORBA IDL mapping does the same for the services communicating via IIOP. Communication via HTTP is strictly defined and doesn't leave room for interoperability issues (except for JavaScript flavors) while SOAP-over-HTTP/S still has some mismatch problems in mapping to different programming languages.

The recommendation here is to put a lot of attention on the data types and naming convention policies (besides the protocols) on the both sides of the interface - for the requester and the provider components. The policies must enforce compatibility and interoperability. One possible mechanism for data type and naming space interoperability control is XML Schema or any other schemas (metadata) accepted by the requester and the provider.

Another best practice recommendation is the creation of a proxy (isolation mapping layer) around the interfaces. That is, the data formats passed through the service interface shouldn't be driven by internal data formats (for either requester or provider). If the evolution of either side causes data format changes, it's better to create object mapping in the proxy than to try to modify the interface.

Pillar 9
Use Rules for flexibility and easy-to-change implementation. Rule engines are just tools used in SOA infrastructure implementation and certainly not a part of the SOA RM. However, being combined with policies on one side and decision-making points on the other side, rule engines become quite powerful instruments for providing a high level of flexibility and quick adaptation to business changes. Leading SOA infrastructure/framework vendors include different rule engines into their business process design tools and Enterprise Service Bus solutions.

While rule engines are really convenient instruments, there's no compatibility between engines from different vendors; a rule engine locks you into a vendor. Anyway, not everything should be driven by rules - rule maintenance is expensive and requires special skills. That is, the designer and developer have to be reasonable in using rule engines.

For example, if the business logic introduces a point for choosing from a finite number of cases (while cases change seldom), using rules may be overkill. Another known risk in using rules is a potential rule contradiction. That is, a rule engine generally doesn't automatically validate if a new rule contradicts any existing rules and any permitted combinations of those rules. Without such validation, the new rule can bypass restrictive rules and expose prohibited information (e.g., a junior clerk gets access to a high-volume account). So when using a rule engine in SOA, proceed with caution.

More Stories By Michael Poulin

Michael Poulin works as an enterprise-level solution architect in the financial industry in the UK. He is a Sun Certified Architect for Java Technology, certified TOGAF Practitioner, and Licensed ZapThink SOA Architect. Michael specializes in distributed computing, SOA, and application security.

Comments (0)

Share your thoughts on this story.

Add your comment
You must be signed in to add a comment. Sign-in | Register

In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.

@MicroservicesExpo Stories
"We started a Master of Science in business analytics - that's the hot topic. We serve the business community around San Francisco so we educate the working professionals and this is where they all want to be," explained Judy Lee, Associate Professor and Department Chair at Golden Gate University, in this SYS-CON.tv interview at 21st Cloud Expo, held Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA.
For over a decade, Application Programming Interface or APIs have been used to exchange data between multiple platforms. From social media to news and media sites, most websites depend on APIs to provide a dynamic and real-time digital experience. APIs have made its way into almost every device and service available today and it continues to spur innovations in every field of technology. There are multiple programming languages used to build and run applications in the online world. And just li...
There is a huge demand for responsive, real-time mobile and web experiences, but current architectural patterns do not easily accommodate applications that respond to events in real time. Common solutions using message queues or HTTP long-polling quickly lead to resiliency, scalability and development velocity challenges. In his session at 21st Cloud Expo, Ryland Degnan, a Senior Software Engineer on the Netflix Edge Platform team, will discuss how by leveraging a reactive stream-based protocol,...
The general concepts of DevOps have played a central role advancing the modern software delivery industry. With the library of DevOps best practices, tips and guides expanding quickly, it can be difficult to track down the best and most accurate resources and information. In order to help the software development community, and to further our own learning, we reached out to leading industry analysts and asked them about an increasingly popular tenet of a DevOps transformation: collaboration.
We call it DevOps but much of the time there’s a lot more discussion about the needs and concerns of developers than there is about other groups. There’s a focus on improved and less isolated developer workflows. There are many discussions around collaboration, continuous integration and delivery, issue tracking, source code control, code review, IDEs, and xPaaS – and all the tools that enable those things. Changes in developer practices may come up – such as developers taking ownership of code ...
The dynamic nature of the cloud means that change is a constant when it comes to modern cloud-based infrastructure. Delivering modern applications to end users, therefore, is a constantly shifting challenge. Delivery automation helps IT Ops teams ensure that apps are providing an optimal end user experience over hybrid-cloud and multi-cloud environments, no matter what the current state of the infrastructure is. To employ a delivery automation strategy that reflects your business rules, making r...
Cloud Governance means many things to many people. Heck, just the word cloud means different things depending on who you are talking to. While definitions can vary, controlling access to cloud resources is invariably a central piece of any governance program. Enterprise cloud computing has transformed IT. Cloud computing decreases time-to-market, improves agility by allowing businesses to adapt quickly to changing market demands, and, ultimately, drives down costs.
Modern software design has fundamentally changed how we manage applications, causing many to turn to containers as the new virtual machine for resource management. As container adoption grows beyond stateless applications to stateful workloads, the need for persistent storage is foundational - something customers routinely cite as a top pain point. In his session at @DevOpsSummit at 21st Cloud Expo, Bill Borsari, Head of Systems Engineering at Datera, explored how organizations can reap the bene...
How is DevOps going within your organization? If you need some help measuring just how well it is going, we have prepared a list of some key DevOps metrics to track. These metrics can help you understand how your team is doing over time. The word DevOps means different things to different people. Some say it a culture and every vendor in the industry claims that their tools help with DevOps. Depending on how you define DevOps, some of these metrics may matter more or less to you and your team.
"CA has been doing a lot of things in the area of DevOps. Now we have a complete set of tool sets in order to enable customers to go all the way from planning to development to testing down to release into the operations," explained Aruna Ravichandran, Vice President of Global Marketing and Strategy at CA Technologies, in this SYS-CON.tv interview at DevOps Summit at 21st Cloud Expo, held Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA.
"We are an integrator of carrier ethernet and bandwidth to get people to connect to the cloud, to the SaaS providers, and the IaaS providers all on ethernet," explained Paul Mako, CEO & CTO of Massive Networks, in this SYS-CON.tv interview at 21st Cloud Expo, held Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA.
"Grape Up leverages Cloud Native technologies and helps companies build software using microservices, and work the DevOps agile way. We've been doing digital innovation for the last 12 years," explained Daniel Heckman, of Grape Up in this SYS-CON.tv interview at 21st Cloud Expo, held Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA.
"NetApp's vision is how we help organizations manage data - delivering the right data in the right place, in the right time, to the people who need it, and doing it agnostic to what the platform is," explained Josh Atwell, Developer Advocate for NetApp, in this SYS-CON.tv interview at 20th Cloud Expo, held June 6-8, 2017, at the Javits Center in New York City, NY.
"Outscale was founded in 2010, is based in France, is a strategic partner to Dassault Systémes and has done quite a bit of work with divisions of Dassault," explained Jackie Funk, Digital Marketing exec at Outscale, in this SYS-CON.tv interview at 20th Cloud Expo, held June 6-8, 2017, at the Javits Center in New York City, NY.
"I focus on what we are calling CAST Highlight, which is our SaaS application portfolio analysis tool. It is an extremely lightweight tool that can integrate with pretty much any build process right now," explained Andrew Siegmund, Application Migration Specialist for CAST, in this SYS-CON.tv interview at 21st Cloud Expo, held Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA.
Let's do a visualization exercise. Imagine it's December 31, 2018, and you're ringing in the New Year with your friends and family. You think back on everything that you accomplished in the last year: your company's revenue is through the roof thanks to the success of your product, and you were promoted to Lead Developer. 2019 is poised to be an even bigger year for your company because you have the tools and insight to scale as quickly as demand requires. You're a happy human, and it's not just...
The enterprise data storage marketplace is poised to become a battlefield. No longer the quiet backwater of cloud computing services, the focus of this global transition is now going from compute to storage. An overview of recent storage market history is needed to understand why this transition is important. Before 2007 and the birth of the cloud computing market we are witnessing today, the on-premise model hosted in large local data centers dominated enterprise storage. Key marketplace play...
Cavirin Systems has just announced C2, a SaaS offering designed to bring continuous security assessment and remediation to hybrid environments, containers, and data centers. Cavirin C2 is deployed within Amazon Web Services (AWS) and features a flexible licensing model for easy scalability and clear pay-as-you-go pricing. Although native to AWS, it also supports assessment and remediation of virtual or container instances within Microsoft Azure, Google Cloud Platform (GCP), or on-premise. By dr...
With continuous delivery (CD) almost always in the spotlight, continuous integration (CI) is often left out in the cold. Indeed, it's been in use for so long and so widely, we often take the model for granted. So what is CI and how can you make the most of it? This blog is intended to answer those questions. Before we step into examining CI, we need to look back. Software developers often work in small teams and modularity, and need to integrate their changes with the rest of the project code b...
Kubernetes is an open source system for automating deployment, scaling, and management of containerized applications. Kubernetes was originally built by Google, leveraging years of experience with managing container workloads, and is now a Cloud Native Compute Foundation (CNCF) project. Kubernetes has been widely adopted by the community, supported on all major public and private cloud providers, and is gaining rapid adoption in enterprises. However, Kubernetes may seem intimidating and complex ...