Welcome!

Microservices Journal Authors: Pat Romanski, Plutora Blog, Liz McMillan, Lori MacVittie, Carmen Gonzalez

Related Topics: Microservices Journal

Microservices Journal: Article

Large Scale B2B via Web Services

Scaling mission-critical Web service for hundreds of partners

Over the past few years Web services have grown from a conceptual draft based on the ideal of cross-platform programmatic interoperability to a formal specification and a vision of grand-scale distributed system architectures. Today we are putting that vision to work and designing systems with service-oriented architectures that are critical to our businesses. Amazon.com has been one of the forerunners in the adoption and promotion of Web services. Consistently striving to remain on the forefront of technology, Amazon has thoroughly embraced Web services - so much so in fact, that you can tie some serious revenue numbers directly to the external use of Amazon's Web services. For the remainder of this article I will outline for you just one of the avenues Amazon has taken to leverage Web services to generate new business.

Most of us are familiar with Amazon.com and have come to notice that other companies offer many of the products sold on Amazon; it isn't uncommon at all to browse direct product offerings from dozens of merchants such as Bombay and Sharper Image on Amazon.com. In fact third-party sellers play a critical role for Amazon in terms of breadth and depth in product offering, but how does the product information of these third-party sellers get there? Also, how are business-critical order, inventory, and price data exchanged reliably and in a timely manner for hundreds of merchants utilizing various platforms and technologies?

The answer: Amazon has developed a highly scalable SOA platform that leverages industry-standard technologies and incorporates the ability to conduct secure transactions, thus effectively allowing third-party sellers to fully control product listings directly on Amazon.com. A true service-oriented architecture allows merchants the ability to put their products in front of millions of Amazon customers; in return Amazon captures a commission on the sale and everyone is happy. The challenge of building a platform to handle hundreds of business partners with millions of transactions took some serious considerations into account.

Knowing that business partners would be utilizing various technologies to programmatically exchange data required not only a standard for the data format but also a security standard for the data transmission as well. By relying on XML schemas to define the structure of the documents to be exchanged (HTTPS as the transport protocol, and SwA [SOAP with Attachments] as the communication protocol), those key transport issues are addressed. Amazon publishes to its third-party sellers the XSD specification to each message Amazon sends and receives. A third-party seller has only to implement the XSD, extract their product data, transform it per the XSD, and post it to Amazon with a SOAP message over HTTPS.

Another design consideration involved the assumption that business partners would have a myriad of system designs that if addressed individually would require custom data definitions, connectors, and message exchange choreography for each partner. This forced the idea that the Amazon platform must be agnostic with regard to partner systems; nothing better lends itself to being agnostic than a service-oriented architecture. In some cases the disparity in partner systems can be so extreme that a system component such as fulfillment, which is critical to business operations, can be owned by a separate company at a different physical location. By encapsulating services in the Amazon interface the partner has significant flexibility in their integration design and can easily incorporate disparate systems.

Because real dollars are impacted with every message exchange, all message exchanges had to be secured and guaranteed. All exchanges utilize HTTPS and require three credentials for validation against an existing account. In order to ensure business-critical documents such as order reports are exchanged successfully an exchange acknowledgement has been incorporated. As displayed in the diagram below, the first operation is a service request for existing document IDs. The return value if null delays for 15 minutes and then requests again. If the return is positive it will contain an array of document IDs. At this point we step into the scope of a document exchange. By calling the getDocument service and passing a document ID, the return result will be the physical document represented by the ID, in this case an order report. Now the document provider (Amazon) knows a request has been made by the requestor (merchant partner) and has responded appropriately by sending the document, but does not know if the document has reached the requestor successfully. The requestor now needs to ensure the successful receipt of the document and perform the final operation of acknowledging to the provider that the document has been received successfully. Subsequently the provider can transfer the document ID from the "outbound" queue to the "sent" queue. In all, this design ensures the state of a document is managed by the requestor who is responsible for follow up operations - in this case fulfillment of the order.

By implementing the system designs in Figure 3, Amazon has been able to successfully integrate product offerings from hundreds of merchants - from small domestic companies to large multinational corporations, Amazon's single platform scales to support them all.

More Stories By Allan Wessels

Allan Wessels has over nine years of diverse experience in software development, from software engineering to project management, with stops at Sony Pictures, Microsoft, and most recently, Amazon.com.

Comments (0)

Share your thoughts on this story.

Add your comment
You must be signed in to add a comment. Sign-in | Register

In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.


@MicroservicesExpo Stories
DevOps approaches within “Unicorns” vary significantly from the reality of DevOps in the enterprise. Most enterprises manage portfolios of heterogeneous applications that are increasingly interconnected, delivered by global teams, at various stages of technology maturity, and are often encumbered by additional compliance and governance obligations. In his session at DevOps Summit, Dalibor Siroky, Director and co-founder at Plutora, will discuss the emerging and evolving experiences of Agile, Co...
It's a "given" in software development - release schedules always slip. Requirements shift, developers underestimate timelines, and quality assurance (QA) finds unexpected defects. Another law of software development is that final release dates are often inflexible. If a market or a holiday shopping season defines your release date you understand how important it is to meet a fixed timeline. Once you've promised a release to the business at the end of the quarter, you are under pressure to de...
Here are a few questions to help you assess the scope of your release management challenges. Based on the answers to these questions, you can calculate your Release Management risk factor. This will help you understand what steps you need to take today to mitigate release management risks that accompany software development at scale. These 100 people don't have to be in a single group and the systems maintained don't have to be limited to a single group. Also, note that the systems supported...
With major technology companies and startups seriously embracing IoT strategies, now is the perfect time to attend @ThingsExpo in Silicon Valley. Learn what is going on, contribute to the discussions, and ensure that your enterprise is as "IoT-Ready" as it can be! Internet of @ThingsExpo, taking place Nov 3-5, 2015, at the Santa Clara Convention Center in Santa Clara, CA, is co-located with 17th Cloud Expo and will feature technical sessions from a rock star conference faculty and the leading in...
17th Cloud Expo, taking place Nov 3-5, 2015, at the Santa Clara Convention Center in Santa Clara, CA, will feature technical sessions from a rock star conference faculty and the leading industry players in the world. Cloud computing is now being embraced by a majority of enterprises of all sizes. Yesterday's debate about public vs. private has transformed into the reality of hybrid cloud: a recent survey shows that 74% of enterprises have a hybrid cloud strategy. Meanwhile, 94% of enterprises a...
Lacking the traditional fanfare associated with any technology that can use the word "container" or mention "Docker" in its press release, Ubuntu Core and its new Snappy system management scheme was introduced late last year. Since then, it's been gaining steam with Microsoft and Amazon and Google announcing support for the stripped-down version of the operating system. Ubuntu Core is what's being called a "micro-OS"; a stripped down, lean container-supporting machine that's becoming more pop...
SYS-CON Events announced today Isomorphic Software, the global leader in high-end, web-based business applications, will exhibit at SYS-CON's DevOps Summit 2015 New York, which will take place on June 9-11, 2015, at the Javits Center in New York City, NY. Isomorphic Software is the global leader in high-end, web-based business applications. We develop, market, and support the SmartClient & Smart GWT HTML5/Ajax platform, combining the productivity and performance of traditional desktop software ...
Cloud computing seems destined to be the way enterprises will use information technology. The drastic cost reductions and impressive operational improvements make the transition an unstoppable trend. The “What is cloud computing?” question now, however, seems to be morphing into “Where is cloud computing going?” While software-as-a-service (SaaS) providers see their market rocketing upward as the easiest and quickest path for cloud adoption, infrastructure-as-a-service providers are suffering...
SYS-CON Events announced today that B2Cloud, a provider of enterprise resource planning software, will exhibit at SYS-CON's 16th International Cloud Expo®, which will take place on June 9-11, 2015, at the Javits Center in New York City, NY. B2cloud develops the software you need. They have the ideal tools to help you work with your clients. B2Cloud’s main solutions include AGIS – ERP, CLOHC, AGIS – Invoice, and IZUM
There is no doubt that Big Data is here and getting bigger every day. Building a Big Data infrastructure today is no easy task. There are an enormous number of choices for database engines and technologies. To make things even more challenging, requirements are getting more sophisticated, and the standard paradigm of supporting historical analytics queries is often just one facet of what is needed. As Big Data growth continues, organizations are demanding real-time access to data, allowing immed...
Containers and microservices have become topics of intense interest throughout the cloud developer and enterprise IT communities. Accordingly, attendees at the upcoming 16th Cloud Expo at the Javits Center in New York June 9-11 will find fresh new content in a new track called PaaS | Containers & Microservices Containers are not being considered for the first time by the cloud community, but a current era of re-consideration has pushed them to the top of the cloud agenda. With the launch ...
The world's leading Cloud event, Cloud Expo has launched Microservices Journal on the SYS-CON.com portal, featuring over 19,000 original articles, news stories, features, and blog entries. DevOps Journal is focused on this critical enterprise IT topic in the world of cloud computing. Microservices Journal offers top articles, news stories, and blog posts from the world's well-known experts and guarantees better exposure for its authors than any other publication. Follow new article posts on T...
SYS-CON Events announced today that MangoApps will exhibit at SYS-CON's 16th International Cloud Expo®, which will take place on June 9-11, 2015, at the Javits Center in New York City, NY., and the 17th International Cloud Expo®, which will take place on November 3–5, 2015, at the Santa Clara Convention Center in Santa Clara, CA. MangoApps provides private all-in-one social intranets allowing workers to securely collaborate from anywhere in the world and from any device. Social, mobile, and eas...
There are 182 billion emails sent every day, generating a lot of data about how recipients and ISPs respond. Many marketers take a more-is-better approach to stats, preferring to have the ability to slice and dice their email lists based numerous arbitrary stats. However, fundamentally what really matters is whether or not sending an email to a particular recipient will generate value. Data Scientists can design high-level insights such as engagement prediction models and content clusters that a...
The cloud has transformed how we think about software quality. Instead of preventing failures, we must focus on automatic recovery from failure. In other words, resilience trumps traditional quality measures. Continuous delivery models further squeeze traditional notions of quality. Remember the venerable project management Iron Triangle? Among time, scope, and cost, you can only fix two or quality will suffer. Only in today's DevOps world, continuous testing, integration, and deployment upend...
It was just two years ago, in March of 2013, that the PaaS company dotCloud decided to open-source its container technology under the name "Docker." Docker quickly became a hit, with Docker Meetups spawning weekly worldwide and downloads occurring at record-breaking rates. The dotCloud team quickly changed direction, renamed the company "Docker," and brought in new CEO Ben Golub in July 2013 to lead the new business model. At the time, Docker was about a twenty-person company. Fast-forward to to...
Chuck Piluso will present a study of cloud adoption trends and the power and flexibility of IBM Power and Pureflex cloud solutions. Speaker Bio: Prior to Secure Infrastructure and Services, Mr. Piluso founded North American Telecommunication Corporation, a facilities-based Competitive Local Exchange Carrier licensed by the Public Service Commission in 10 states, serving as the company's chairman and president from 1997 to 2000. Between 1990 and 1997, Mr. Piluso served as chairman & founder...
To manage complex web services with lots of calls to the cloud, many businesses have invested in Application Performance Management (APM) and Network Performance Management (NPM) tools. Together APM and NPM tools are essential aids in improving a business's infrastructure required to support an effective web experience... but they are missing a critical component - Internet visibility. Internet connectivity has always played a role in customer access to web presence, but in the past few years u...
Working with Big Data is challenging, especially when decision makers depend on market insights and intelligence from your data but don't have quick access to it or find it unusable. In their session at 6th Big Data Expo, Ian Khan, Global Strategic Positioning & Brand Manager at Solgenia; Zel Bianco, President, CEO and Co-Founder of Interactive Edge of Solgenia; and Ermanno Bonifazi, CEO & Founder at Solgenia, discussed how a revolutionary cloud-based BI along with mobile analytics is already c...
2015 is the 10th birthday of Ruby on Rails (or simply called Rails), the popular open source web application framework that uses the Ruby programming language. Ever since Rails burst on the scene a decade ago it has continued to scale up as an elegant way to build dynamic websites quickly and efficiently. Rails has garnered a strong following, especially among tech startups. In fact, some of the best known firms out there are using the framework to build their sites. Examples of elegant sites bu...