Click here to close now.


MICROSERVICES Authors: Jason Bloomberg, Liz McMillan, Elizabeth White, XebiaLabs Blog, Michael Kanasoot

Related Topics: Cloud Expo, Java, MICROSERVICES, Open Source, Web 2.0, Apache

Cloud Expo: Article

The Power of Opacity in REST

There’s more to the opacity story than opaque URIs

Ever wonder how a sophisticated Web site works? Take Facebook, for example. You can view the source and you can hardly pick out any recognizable HTML, let alone divine how the wizards back at Facebook HQ get the site to work. Now, try viewing the source at a simpler Web site, like ZapThink’s. Sure enough, there’s HTML under the covers, but you still can’t tell from the file the Web server sends to your browser what’s going on behind the scenes (we use WordPress, in case you were wondering).

Put into RESTful terms, there is a separation between resource (e.g., the program running on the server) and the representation (e.g., the Web page it sends to your browser). In fact, this separation is a fundamental REST constraint which allows the resource to be opaque.

When people talk about opacity in the REST context, they are usually referring to Uniform Resource Indicators (URIs). You should be able to construct URIs however you like, the theory goes, and it’s up to the resource to figure out how to respond appropriately. In other words, it’s not up to the client to know how to provide specific instructions to the server, other than by clicking the hyperlinks the resource has previously provided to the client.

But there’s more to the opacity story than opaque URIs. Fundamentally, the client has no way of knowing anything at all about what’s really going on behind the scenes. The resource might be a file, a script, a container, an object, or some complicated combination of these and other kinds of things. There are two important lessons for the techies behind the curtain: first, don’t assume resources come in one flavor, and second, it’s important to understand the full breadth of capabilities and patterns that you can leverage when architecting or building resources. After all, anything you can give a URI to can be a resource.

Exploring the Power of Opacity
Let’s begin our exploration of opacity with HTTP’s POST method. Of the four primary HTTP methods (GET, POST, PUT, and DELETE), POST is the only one that’s not idempotent: in other words, not only does it change the state of the resource, but it does so in a way that calling it twice has a different effect than calling it once. In the RESTful context, you should use POST to initialize a resource. According to the HTTP spec, POST creates a subordinate resource, as the figure below illustrates:

In the interaction above, the client POSTs to the cart resource, which initializes a cart instance, names it “abcde,” and returns a hyperlink to that new subordinate resource to the client. In this context, subordinate means that the abcde comes after cart and a slash in the URI

Here’s the essential question: just what do cart and abcde represent on the server? cart looks like a directory and abcde looks like a file, given the pathlike structure of the URI. But we know that guess probably isn’t right, because POSTing to the cart resource actually created the abcde resource, which represents the cart instance. So could abcde be an object instance? Perhaps. The bottom line is you can’t tell, because as far as the client is concerned, it doesn’t matter. What matters is that the client now has one (or more) hyperlinks to its own cart that it can interact with via a uniform interface.

One way or the other, however, POST changes the state of the abcde cart instance, which requires a relatively onerous level of processing on the server. To lighten the future load on the server, thus improving its scalability, we may want to cache the representation the resource provides. Fortunately, REST explicitly supports cacheability, as the figure below illustrates:

In the pattern above, a gateway intermediary passes along the POST to the server, fetching a static representation it puts in its cache. As long as clients make requests that aren’t intended to change the state of the resource (namely, GETs), then serving up the cached copy is as good as passing along the request to the underlying resource, until the representation expires from the cache.

Opacity plays a critical role in this example as well, since saying the cached copy is just as good as a response directly from the resource is an example of opacity. As a result, the gateway is entirely transparent to the client, serving in the role of server in interactions with the client but in the role of client in interactions with the underlying server.

The limitation of the example above, of course, is the static nature of the cache. If the client wants to change the state of the resource (via PUT or another POST), then such a request would necessarily expire the cache, requiring the intermediary to pass the request along to the underlying server. In situations where the resource state changes frequently, therefore, caching is of limited value.

Opacity and RESTful Clouds
We can extend the pattern above to provide greater capabilities on the intermediary. In the example below, the intermediary is a full-fledged server in its own right, and the underlying server returns executable server scripts for the intermediary to execute on behalf of the underlying server. In other words, the intermediary caches representations that are themselves server programs (e.g., php scripts). Furthermore, these server scripts are prepopulated with any initial state data in response to the original POST from the client.

Increasing the sophistication of our cache would provide little value, however, if we didn’t have a better way of dealing with state information. Fortunately, REST grants our wishes in this case as well, because it enables us to separate resource state (maintained on the underlying server) from application state, which we can transfer to the client.

In the figure above, after the client has initialized the resource, it may wish to, say, update its cart. So, the user clicks a link that executes a PUT that sends the updated information, along with values from one or more hidden form fields to the intermediary. However, instead of updating resource state, the state information remains in the messages (both requests from the client and representations returned from the intermediary) as long as the client only executes idempotent requests. There is no need to update resource state in this situation, because the scripts on the intermediary know to pass along state information in hidden form fields, for example. When the cart process is complete and the user is ready to submit an order, only then does the client execute another POST, which the intermediary knows to pass along to the underlying server.

However, there’s no strict rule that says that the intermediary can only handle idempotent requests; you could easily put a script on it that would handle POSTs, and similarly, it might make sense to send an idempotent request like a DELETE along to the underlying server for execution. But on the other hand, the rule that the intermediary handles only the idempotent requests may be appropriate in your situation, because POST would then be the only method that could ever change state on the underlying server.

As we explained in an earlier ZapFlash, one of the primary benefits to following the pattern in the figure above is to support elasticity when you put the intermediary server in the Cloud. Because it is stateless, it doesn’t matter which virtual machine (VM) instance replies to any client request, and if a VM instance crashes, we can bootstrap its replacement without losing any state information. In other words, opacity is essential to both the elasticity and fault tolerance of the Cloud, and furthermore, following a RESTful approach provides that opacity.

The ZapThink Take
There’s one more RESTful pattern that ZapThink is particularly interested in: RESTful SOA, naturally. For this pattern we need another kind of intermediary: a RESTful SOA intermediary, in addition to the Cloud-based stateless server intermediary, or anything else we want to abstract for that matter. The figure below illustrates the RESTful SOA pattern.

The role of the RESTful SOA intermediary is to provide abstracted (in other words, opaque) RESTful Service endpoints that follow strict URI formatting rules. Furthermore, this intermediary must handle state information appropriately, that is, following a RESTful approach that transfers state information in messages. As a result, the SOA intermediary can support stateless message protocols for interactions with Service consumers while remaining stateless itself. Most ESBs maintain state, and therefore a RESTful SOA intermediary wouldn’t be a typical ESB, although it could certainly route messages to one.

So, which pattern is the best one? As we say in our Licensed ZapThink Architect (LZA) and Cloud Computing for Architects (CCA) courses, it depends. The architect is looking for the right tool for the job. You must understand the problem before recommending the appropriate solution. We cover REST-based SOA in our LZA course (coming to Johannesburg) and RESTful Clouds in the CCA course (coming to London, DC, and San Diego). See you there!

Image credit: Derek Keats

More Stories By Jason Bloomberg

Jason Bloomberg is the leading expert on architecting agility for the enterprise. As president of Intellyx, Mr. Bloomberg brings his years of thought leadership in the areas of Cloud Computing, Enterprise Architecture, and Service-Oriented Architecture to a global clientele of business executives, architects, software vendors, and Cloud service providers looking to achieve technology-enabled business agility across their organizations and for their customers. His latest book, The Agile Architecture Revolution (John Wiley & Sons, 2013), sets the stage for Mr. Bloomberg’s groundbreaking Agile Architecture vision.

Mr. Bloomberg is perhaps best known for his twelve years at ZapThink, where he created and delivered the Licensed ZapThink Architect (LZA) SOA course and associated credential, certifying over 1,700 professionals worldwide. He is one of the original Managing Partners of ZapThink LLC, the leading SOA advisory and analysis firm, which was acquired by Dovel Technologies in 2011. He now runs the successor to the LZA program, the Bloomberg Agile Architecture Course, around the world.

Mr. Bloomberg is a frequent conference speaker and prolific writer. He has published over 500 articles, spoken at over 300 conferences, Webinars, and other events, and has been quoted in the press over 1,400 times as the leading expert on agile approaches to architecture in the enterprise.

Mr. Bloomberg’s previous book, Service Orient or Be Doomed! How Service Orientation Will Change Your Business (John Wiley & Sons, 2006, coauthored with Ron Schmelzer), is recognized as the leading business book on Service Orientation. He also co-authored the books XML and Web Services Unleashed (SAMS Publishing, 2002), and Web Page Scripting Techniques (Hayden Books, 1996).

Prior to ZapThink, Mr. Bloomberg built a diverse background in eBusiness technology management and industry analysis, including serving as a senior analyst in IDC’s eBusiness Advisory group, as well as holding eBusiness management positions at USWeb/CKS (later marchFIRST) and WaveBend Solutions (now Hitachi Consulting).

@MicroservicesExpo Stories
Our guest on the podcast this week is Jason Bloomberg, President at Intellyx. When we build services we want them to be lightweight, stateless and scalable while doing one thing really well. In today’s cloud world, we’re revisiting what to takes to make a good service in the first place.microservices Listen in to learn why following “the book” doesn’t necessarily mean that you’re solving key business problems.
SYS-CON Events announced today that Cisco, the worldwide leader in IT that transforms how people connect, communicate and collaborate, has been named “Gold Sponsor” of SYS-CON's 16th International Cloud Expo®, which will take place on June 9-11, 2015, at the Javits Center in New York City, NY. Cisco makes amazing things happen by connecting the unconnected. Cisco has shaped the future of the Internet by becoming the worldwide leader in transforming how people connect, communicate and collaborat...
Hosted PaaS providers have given independent developers and startups huge advantages in efficiency and reduced time-to-market over their more process-bound counterparts in enterprises. Software frameworks are now available that allow enterprise IT departments to provide these same advantages for developers in their own organization. In his workshop session at DevOps Summit, Troy Topnik, ActiveState’s Technical Product Manager, will show how on-prem or cloud-hosted Private PaaS can enable organ...
SYS-CON Events announced today that MangoApps will exhibit at SYS-CON's 16th International Cloud Expo®, which will take place on June 9-11, 2015, at the Javits Center in New York City, NY., and the 17th International Cloud Expo®, which will take place on November 3–5, 2015, at the Santa Clara Convention Center in Santa Clara, CA. MangoApps provides private all-in-one social intranets allowing workers to securely collaborate from anywhere in the world and from any device. Social, mobile, and eas...
SYS-CON Events announced today that Solgenia will exhibit at SYS-CON's 16th International Cloud Expo®, which will take place on June 9-11, 2015, at the Javits Center in New York City, NY, and the 17th International Cloud Expo®, which will take place on November 3–5, 2015, at the Santa Clara Convention Center in Santa Clara, CA. Solgenia is the global market leader in Cloud Collaboration and Cloud Infrastructure software solutions. Designed to “Bridge the Gap” between Personal and Professional S...
When it comes to microservices there are myths and uncertainty about the journey ahead. Deploying a “Hello World” app on Docker is a long way from making microservices work in real enterprises with large applications, complex environments and existing organizational structures. February 19, 2015 10:00am PT / 1:00pm ET → 45 Minutes Join our four experts: Special host Gene Kim, Gary Gruver, Randy Shoup and XebiaLabs’ Andrew Phillips as they explore the realities of microservices in today’s IT worl...
OmniTI has expanded its services to help customers automate their processes to deliver high quality applications to market faster. Consistent with its focus on IT agility and quality, OmniTI operates under DevOps principles, exploring the flow of value through the IT delivery process, identifying opportunities to eliminate waste, realign misaligned incentives, and open bottlenecks. OmniTI takes a unique, value-centric approach by plotting each opportunity in an effort-payoff quadrant, then work...
Cloud computing is changing the way we look at IT costs, according to industry experts on a recent Cloud Luminary Fireside Chat panel discussion. Enterprise IT, traditionally viewed as a cost center, now plays a central role in the delivery of software-driven goods and services. Therefore, companies need to understand their cloud utilization and resulting costs in order to ensure profitability on their business offerings. Led by Bernard Golden, this fireside chat offers valuable insights on ho...
The world's leading Cloud event, Cloud Expo has launched Microservices Journal on the portal, featuring over 19,000 original articles, news stories, features, and blog entries. DevOps Journal is focused on this critical enterprise IT topic in the world of cloud computing. Microservices Journal offers top articles, news stories, and blog posts from the world's well-known experts and guarantees better exposure for its authors than any other publication. Follow new article posts on T...
Modern Systems announced completion of a successful project with its new Rapid Program Modernization (eavRPMa"c) software. The eavRPMa"c technology architecturally transforms legacy applications, enabling faster feature development and reducing time-to-market for critical software updates. Working with Modern Systems, the University of California at Santa Barbara (UCSB) leveraged eavRPMa"c to transform its Student Information System from Software AG's Natural syntax to a modern application lev...
For those of us that have been practicing SOA for over a decade, it's surprising that there's so much interest in microservices. In fairness microservices don't look like the vendor play that was early SOA in the early noughties. But experienced SOA practitioners everywhere will be wondering if microservices is actually a good thing. You see microservices is basically an SOA pattern that inherits all the well-known SOA principles and adds characteristics that address the use of SOA for distribut...
Microservice architectures are the new hotness, even though they aren't really all that different (in principle) from the paradigm described by SOA (which is dead, or not dead, depending on whom you ask). One of the things this decompositional approach to application architecture does is encourage developers and operations (some might even say DevOps) to re-evaluate scaling strategies. In particular, the notion is forwarded that an application should be built to scale and then infrastructure sho...
SYS-CON Events announced today the IoT Bootcamp – Jumpstart Your IoT Strategy, being held June 9–10, 2015, in conjunction with 16th Cloud Expo and Internet of @ThingsExpo at the Javits Center in New York City. This is your chance to jumpstart your IoT strategy. Combined with real-world scenarios and use cases, the IoT Bootcamp is not just based on presentations but includes hands-on demos and walkthroughs. We will introduce you to a variety of Do-It-Yourself IoT platforms including Arduino, Ras...
Our guest on the podcast this week is Jason Bloomberg, President at Intellyx. When we build services we want them to be lightweight, stateless and scalable while doing one thing really well. In today's cloud world, we're revisiting what to takes to make a good service in the first place. Listen in to learn why following "the book" doesn't necessarily mean that you're solving key business problems.
Microservices are the result of decomposing applications. That may sound a lot like SOA, but SOA was based on an object-oriented (noun) premise; that is, services were built around an object - like a customer - with all the necessary operations (functions) that go along with it. SOA was also founded on a variety of standards (most of them coming out of OASIS) like SOAP, WSDL, XML and UDDI. Microservices have no standards (at least none deriving from a standards body or organization) and can be b...
Right off the bat, Newman advises that we should "think of microservices as a specific approach for SOA in the same way that XP or Scrum are specific approaches for Agile Software development". These analogies are very interesting because my expectation was that microservices is a pattern. So I might infer that microservices is a set of process techniques as opposed to an architectural approach. Yet in the book, Newman clearly includes some elements of concept model and architecture as well as p...
SYS-CON Events announced today the DevOps Foundation Certification Course, being held June ?, 2015, in conjunction with DevOps Summit and 16th Cloud Expo at the Javits Center in New York City, NY. This sixteen (16) hour course provides an introduction to DevOps – the cultural and professional movement that stresses communication, collaboration, integration and automation in order to improve the flow of work between software developers and IT operations professionals. Improved workflows will res...
Microservices, for the uninitiated, are essentially the decomposition of applications into multiple services. This decomposition is often based on functional lines, with related functions being grouped together into a service. While this may sound a like SOA, it really isn't, especially given that SOA was an object-centered methodology that focused on creating services around "nouns" like customer and product. Microservices, while certainly capable of being noun-based, are just as likely to be v...
An explosive combination of technology trends will be where ‘microservices’ and the IoT Internet of Things intersect, a concept we can describe by comparing it with a previous theme, the ‘X Internet.' The idea of using small self-contained application components has been popular since XML Web services began and a distributed computing future of smart fridges and kettles was imagined long back in the early Internet years.
Even though it’s now Microservices Journal, long-time fans of SOA World Magazine can take comfort in the fact that the URL – – remains unchanged. And that’s no mistake, as microservices are really nothing more than a new and improved take on the Service-Oriented Architecture (SOA) best practices we struggled to hammer out over the last decade. Skeptics, however, might say that this change is nothing more than an exercise in buzzword-hopping. SOA is passé, and now that people are ...