Click here to close now.



Welcome!

Microservices Expo Authors: Liz McMillan, Elizabeth White, Carmen Gonzalez, Pat Romanski, Yeshim Deniz

Related Topics: @CloudExpo, Java IoT, Microservices Expo, Open Source Cloud, Agile Computing, Apache

@CloudExpo: Article

The Power of Opacity in REST

There’s more to the opacity story than opaque URIs

Ever wonder how a sophisticated Web site works? Take Facebook, for example. You can view the source and you can hardly pick out any recognizable HTML, let alone divine how the wizards back at Facebook HQ get the site to work. Now, try viewing the source at a simpler Web site, like ZapThink’s. Sure enough, there’s HTML under the covers, but you still can’t tell from the file the Web server sends to your browser what’s going on behind the scenes (we use WordPress, in case you were wondering).

Put into RESTful terms, there is a separation between resource (e.g., the program running on the server) and the representation (e.g., the Web page it sends to your browser). In fact, this separation is a fundamental REST constraint which allows the resource to be opaque.

When people talk about opacity in the REST context, they are usually referring to Uniform Resource Indicators (URIs). You should be able to construct URIs however you like, the theory goes, and it’s up to the resource to figure out how to respond appropriately. In other words, it’s not up to the client to know how to provide specific instructions to the server, other than by clicking the hyperlinks the resource has previously provided to the client.

But there’s more to the opacity story than opaque URIs. Fundamentally, the client has no way of knowing anything at all about what’s really going on behind the scenes. The resource might be a file, a script, a container, an object, or some complicated combination of these and other kinds of things. There are two important lessons for the techies behind the curtain: first, don’t assume resources come in one flavor, and second, it’s important to understand the full breadth of capabilities and patterns that you can leverage when architecting or building resources. After all, anything you can give a URI to can be a resource.

Exploring the Power of Opacity
Let’s begin our exploration of opacity with HTTP’s POST method. Of the four primary HTTP methods (GET, POST, PUT, and DELETE), POST is the only one that’s not idempotent: in other words, not only does it change the state of the resource, but it does so in a way that calling it twice has a different effect than calling it once. In the RESTful context, you should use POST to initialize a resource. According to the HTTP spec, POST creates a subordinate resource, as the figure below illustrates:

In the interaction above, the client POSTs to the cart resource, which initializes a cart instance, names it “abcde,” and returns a hyperlink to that new subordinate resource to the client. In this context, subordinate means that the abcde comes after cart and a slash in the URI http://example.com/cart/abcde.

Here’s the essential question: just what do cart and abcde represent on the server? cart looks like a directory and abcde looks like a file, given the pathlike structure of the URI. But we know that guess probably isn’t right, because POSTing to the cart resource actually created the abcde resource, which represents the cart instance. So could abcde be an object instance? Perhaps. The bottom line is you can’t tell, because as far as the client is concerned, it doesn’t matter. What matters is that the client now has one (or more) hyperlinks to its own cart that it can interact with via a uniform interface.

One way or the other, however, POST changes the state of the abcde cart instance, which requires a relatively onerous level of processing on the server. To lighten the future load on the server, thus improving its scalability, we may want to cache the representation the resource provides. Fortunately, REST explicitly supports cacheability, as the figure below illustrates:

In the pattern above, a gateway intermediary passes along the POST to the server, fetching a static representation it puts in its cache. As long as clients make requests that aren’t intended to change the state of the resource (namely, GETs), then serving up the cached copy is as good as passing along the request to the underlying resource, until the representation expires from the cache.

Opacity plays a critical role in this example as well, since saying the cached copy is just as good as a response directly from the resource is an example of opacity. As a result, the gateway is entirely transparent to the client, serving in the role of server in interactions with the client but in the role of client in interactions with the underlying server.

The limitation of the example above, of course, is the static nature of the cache. If the client wants to change the state of the resource (via PUT or another POST), then such a request would necessarily expire the cache, requiring the intermediary to pass the request along to the underlying server. In situations where the resource state changes frequently, therefore, caching is of limited value.

Opacity and RESTful Clouds
We can extend the pattern above to provide greater capabilities on the intermediary. In the example below, the intermediary is a full-fledged server in its own right, and the underlying server returns executable server scripts for the intermediary to execute on behalf of the underlying server. In other words, the intermediary caches representations that are themselves server programs (e.g., php scripts). Furthermore, these server scripts are prepopulated with any initial state data in response to the original POST from the client.

Increasing the sophistication of our cache would provide little value, however, if we didn’t have a better way of dealing with state information. Fortunately, REST grants our wishes in this case as well, because it enables us to separate resource state (maintained on the underlying server) from application state, which we can transfer to the client.

In the figure above, after the client has initialized the resource, it may wish to, say, update its cart. So, the user clicks a link that executes a PUT that sends the updated information, along with values from one or more hidden form fields to the intermediary. However, instead of updating resource state, the state information remains in the messages (both requests from the client and representations returned from the intermediary) as long as the client only executes idempotent requests. There is no need to update resource state in this situation, because the scripts on the intermediary know to pass along state information in hidden form fields, for example. When the cart process is complete and the user is ready to submit an order, only then does the client execute another POST, which the intermediary knows to pass along to the underlying server.

However, there’s no strict rule that says that the intermediary can only handle idempotent requests; you could easily put a script on it that would handle POSTs, and similarly, it might make sense to send an idempotent request like a DELETE along to the underlying server for execution. But on the other hand, the rule that the intermediary handles only the idempotent requests may be appropriate in your situation, because POST would then be the only method that could ever change state on the underlying server.

As we explained in an earlier ZapFlash, one of the primary benefits to following the pattern in the figure above is to support elasticity when you put the intermediary server in the Cloud. Because it is stateless, it doesn’t matter which virtual machine (VM) instance replies to any client request, and if a VM instance crashes, we can bootstrap its replacement without losing any state information. In other words, opacity is essential to both the elasticity and fault tolerance of the Cloud, and furthermore, following a RESTful approach provides that opacity.

The ZapThink Take
There’s one more RESTful pattern that ZapThink is particularly interested in: RESTful SOA, naturally. For this pattern we need another kind of intermediary: a RESTful SOA intermediary, in addition to the Cloud-based stateless server intermediary, or anything else we want to abstract for that matter. The figure below illustrates the RESTful SOA pattern.

The role of the RESTful SOA intermediary is to provide abstracted (in other words, opaque) RESTful Service endpoints that follow strict URI formatting rules. Furthermore, this intermediary must handle state information appropriately, that is, following a RESTful approach that transfers state information in messages. As a result, the SOA intermediary can support stateless message protocols for interactions with Service consumers while remaining stateless itself. Most ESBs maintain state, and therefore a RESTful SOA intermediary wouldn’t be a typical ESB, although it could certainly route messages to one.

So, which pattern is the best one? As we say in our Licensed ZapThink Architect (LZA) and Cloud Computing for Architects (CCA) courses, it depends. The architect is looking for the right tool for the job. You must understand the problem before recommending the appropriate solution. We cover REST-based SOA in our LZA course (coming to Johannesburg) and RESTful Clouds in the CCA course (coming to London, DC, and San Diego). See you there!

Image credit: Derek Keats

More Stories By Jason Bloomberg

Jason Bloomberg is the leading expert on architecting agility for the enterprise. As president of Intellyx, Mr. Bloomberg brings his years of thought leadership in the areas of Cloud Computing, Enterprise Architecture, and Service-Oriented Architecture to a global clientele of business executives, architects, software vendors, and Cloud service providers looking to achieve technology-enabled business agility across their organizations and for their customers. His latest book, The Agile Architecture Revolution (John Wiley & Sons, 2013), sets the stage for Mr. Bloomberg’s groundbreaking Agile Architecture vision.

Mr. Bloomberg is perhaps best known for his twelve years at ZapThink, where he created and delivered the Licensed ZapThink Architect (LZA) SOA course and associated credential, certifying over 1,700 professionals worldwide. He is one of the original Managing Partners of ZapThink LLC, the leading SOA advisory and analysis firm, which was acquired by Dovel Technologies in 2011. He now runs the successor to the LZA program, the Bloomberg Agile Architecture Course, around the world.

Mr. Bloomberg is a frequent conference speaker and prolific writer. He has published over 500 articles, spoken at over 300 conferences, Webinars, and other events, and has been quoted in the press over 1,400 times as the leading expert on agile approaches to architecture in the enterprise.

Mr. Bloomberg’s previous book, Service Orient or Be Doomed! How Service Orientation Will Change Your Business (John Wiley & Sons, 2006, coauthored with Ron Schmelzer), is recognized as the leading business book on Service Orientation. He also co-authored the books XML and Web Services Unleashed (SAMS Publishing, 2002), and Web Page Scripting Techniques (Hayden Books, 1996).

Prior to ZapThink, Mr. Bloomberg built a diverse background in eBusiness technology management and industry analysis, including serving as a senior analyst in IDC’s eBusiness Advisory group, as well as holding eBusiness management positions at USWeb/CKS (later marchFIRST) and WaveBend Solutions (now Hitachi Consulting).

@MicroservicesExpo Stories
Cloud Expo, Inc. has announced today that Andi Mann returns to 'DevOps at Cloud Expo 2016' as Conference Chair The @DevOpsSummit at Cloud Expo will take place on November 1-3, 2016, at the Santa Clara Convention Center in Santa Clara, CA. "DevOps is set to be one of the most profound disruptions to hit IT in decades," said Andi Mann. "It is a natural extension of cloud computing, and I have seen both firsthand and in independent research the fantastic results DevOps delivers. So I am excited t...
In the world of DevOps there are ‘known good practices’ – aka ‘patterns’ – and ‘known bad practices’ – aka ‘anti-patterns.' Many of these patterns and anti-patterns have been developed from real world experience, especially by the early adopters of DevOps theory; but many are more feasible in theory than in practice, especially for more recent entrants to the DevOps scene. In this power panel at @DevOpsSummit at 18th Cloud Expo, moderated by DevOps Conference Chair Andi Mann, panelists discusse...
When people aren’t talking about VMs and containers, they’re talking about serverless architecture. Serverless is about no maintenance. It means you are not worried about low-level infrastructural and operational details. An event-driven serverless platform is a great use case for IoT. In his session at @ThingsExpo, Animesh Singh, an STSM and Lead for IBM Cloud Platform and Infrastructure, will detail how to build a distributed serverless, polyglot, microservices framework using open source tec...
More and more companies are looking to microservices as an architectural pattern for breaking apart applications into more manageable pieces so that agile teams can deliver new features quicker and more effectively. What this pattern has done more than anything to date is spark organizational transformations, setting the foundation for future application development. In practice, however, there are a number of considerations to make that go beyond simply “build, ship, and run,” which changes ho...
Gartner is now treating algorithms like they are some kind of innovative addition to the modern digital discussion. Presumably the brilliant minds there have some novel insight into algorithms and, yes, the Algorithm Economy that CIOs should sit up and take notice of. Not only are algorithms nothing new, but much of what Gartner is saying about them is obvious. The bigger picture here is that software continues to improve, and enterprises are becoming increasingly software-driven, in part bec...
The Internet of Things is clearly many things: data collection and analytics, wearables, Smart Grids and Smart Cities, the Industrial Internet, and more. Cool platforms like Arduino, Raspberry Pi, Intel's Galileo and Edison, and a diverse world of sensors are making the IoT a great toy box for developers in all these areas. In this Power Panel at @ThingsExpo, moderated by Conference Chair Roger Strukhoff, panelists discussed what things are the most important, which will have the most profound...
The Internet of Things (IoT) is growing rapidly by extending current technologies, products and networks. By 2020, Cisco estimates there will be 50 billion connected devices. Gartner has forecast revenues of over $300 billion, just to IoT suppliers. Now is the time to figure out how you’ll make money – not just create innovative products. With hundreds of new products and companies jumping into the IoT fray every month, there’s no shortage of innovation. Despite this, McKinsey/VisionMobile data...
NHK, Japan Broadcasting, will feature the upcoming @ThingsExpo Silicon Valley in a special 'Internet of Things' and smart technology documentary that will be filmed on the expo floor between November 3 to 5, 2015, in Santa Clara. NHK is the sole public TV network in Japan equivalent to the BBC in the UK and the largest in Asia with many award-winning science and technology programs. Japanese TV is producing a documentary about IoT and Smart technology and will be covering @ThingsExpo Silicon Val...
SYS-CON Events announced today that Men & Mice, the leading global provider of DNS, DHCP and IP address management overlay solutions, will exhibit at SYS-CON's 18th International Cloud Expo®, which will take place on June 7-9, 2016, at the Javits Center in New York City, NY. The Men & Mice Suite overlay solution is already known for its powerful application in heterogeneous operating environments, enabling enterprises to scale without fuss. Building on a solid range of diverse platform support,...
Internet of @ThingsExpo, taking place June 7-9, 2016 at Javits Center, New York City and Nov 1-3, 2016, at the Santa Clara Convention Center in Santa Clara, CA, is co-located with the 18th International @CloudExpo and will feature technical sessions from a rock star conference faculty and the leading industry players in the world and ThingsExpo New York Call for Papers is now open.
SYS-CON Events announced today that Catchpoint Systems, Inc., a provider of innovative web and infrastructure monitoring solutions, has been named “Silver Sponsor” of SYS-CON's DevOps Summit at 18th Cloud Expo New York, which will take place June 7-9, 2016, at the Javits Center in New York City, NY. Catchpoint is a leading Digital Performance Analytics company that provides unparalleled insight into customer-critical services to help consistently deliver an amazing customer experience. Designed...
@DevOpsSummit taking place June 7-9, 2016 at Javits Center, New York City, and Nov 1-3, 2016, at the Santa Clara Convention Center in Santa Clara, CA, is co-located with the 18th International @CloudExpo and will feature technical sessions from a rock star conference faculty and the leading industry players in the world.
Cloud Expo, Inc. has announced today that Andi Mann returns to 'DevOps at Cloud Expo 2016' as Conference Chair The @DevOpsSummit at Cloud Expo will take place on June 7-9, 2016, at the Javits Center in New York City, New York. "DevOps is set to be one of the most profound disruptions to hit IT in decades," said Andi Mann. "It is a natural extension of cloud computing, and I have seen both firsthand and in independent research the fantastic results DevOps delivers. So I am excited to help the g...
Korean Broadcasting System (KBS) will feature the upcoming 18th Cloud Expo | @ThingsExpo in a New York news documentary about the "New IT for the Future." The documentary will cover how big companies are transmitting or adopting the new IT for the future and will be filmed on the expo floor between June 7-June 9, 2016, at the Javits Center in New York City, New York. KBS has long been a leader in the development of the broadcasting culture of Korea. As the key public service broadcaster of Korea...
SYS-CON Events announced today that Addteq will exhibit at SYS-CON's @DevOpsSummit at Cloud Expo New York, which will take place on June 7-9, 2016, at the Javits Center in New York City, NY. Addteq is one of the top 10 Platinum Atlassian Experts who specialize in DevOps, custom and continuous integration, automation, plugin development, and consulting for midsize and global firms. Addteq firmly believes that automation is essential for successful software releases. Addteq centers its products a...
In the rush to compete in the digital age, a successful digital transformation is essential, but many organizations are setting themselves up for failure. There’s a common misconception that the process is just about technology, but it’s not. It’s about your business. It shouldn’t be treated as an isolated IT project; it should be driven by business needs with the committed involvement of a range of stakeholders.
SYS-CON Events announced today that FalconStor Software® Inc., a 15-year innovator of software-defined storage solutions, will exhibit at SYS-CON's 18th International Cloud Expo®, which will take place on June 7-9, 2016, at the Javits Center in New York City, NY. FalconStor Software®, Inc. (NASDAQ: FALC) is a leading software-defined storage company offering a converged, hardware-agnostic, software-defined storage and data services platform. Its flagship solution FreeStor®, utilizes a horizonta...
With major technology companies and startups seriously embracing IoT strategies, now is the perfect time to attend @ThingsExpo 2016 in New York and Silicon Valley. Learn what is going on, contribute to the discussions, and ensure that your enterprise is as "IoT-Ready" as it can be! Internet of @ThingsExpo, taking place Nov 3-5, 2015, at the Santa Clara Convention Center in Santa Clara, CA, is co-located with 17th Cloud Expo and will feature technical sessions from a rock star conference faculty ...
SYS-CON Events announced today that Column Technologies will exhibit at SYS-CON's @DevOpsSummit at Cloud Expo, which will take place on June 7-9, 2016, at the Javits Center in New York City, NY. Established in 1998, Column Technologies is a global technology solutions provider with over 400 employees, headquartered in the United States with offices in Canada, India, and the United Kingdom. Column Technologies provides “Best of Breed” technology solutions that automate the key DevOps principal...
SYS-CON Events announced today that SoftLayer, an IBM Company, has been named “Gold Sponsor” of SYS-CON's 18th Cloud Expo, which will take place on June 7-9, 2016, at the Javits Center in New York, New York. SoftLayer, an IBM Company, provides cloud infrastructure as a service from a growing number of data centers and network points of presence around the world. SoftLayer’s customers range from Web startups to global enterprises.