Microservices Expo Authors: Yeshim Deniz, Elizabeth White, Liz McMillan, Carmen Gonzalez, Pat Romanski

Related Topics: Microservices Expo

Microservices Expo: Article

SOA Feature Story: Real-Time SOA Starts with the Messaging Bus!

The mediator of all component interactions

Service Oriented Architectures are increasingly being used to implement high-performance and real-time systems. Traditional systems operate in "human real-time," where human patience is the limit. Increasingly, however, systems operate in "computer real-time," where the only limits are imposed by the operational speed of the computers and networks.

For example, next-generation Air Traffic Management systems are being developed to accommodate the huge increase in air traffic and link the operational capabilities of agencies such as the Federal Aviation Authority, the Department of Defense (DOD) and the Department of Homeland Security (DHS). These systems require higher information bandwidth (to track more aircraft or more complex "free-flight" trajectories) as well as much lower latencies or delays on the information (to detect flight abnormalities quickly). Similar demands are being made in healthcare, SCADA, network monitoring, energy distribution, transportation, and other critical infrastructure systems.

Best-of-Breed SOA Components
Demanding real-time applications require best-of-breed service-oriented foundational components. There are three kinds of foundational components in a SOA system: A messaging fabric/bus, information transformation/processing engines, and persistence/storage services (see Figure 1). Often these components are integrated into an Enterprise Service Bus (ESB) and hosted in a J2EE Application Server.

Of these foundational components, the Messaging Fabric/Bus is the most critical, since it mediates all interactions between components.

Low-performance SOA systems may use HTTP as the "messaging fabric/bus" to exchange messages between components. This approach is only suitable for non-demanding applications: HTTP isn't reliable, has limited bandwidth, introduces very high latencies, and can't buffer and queue messages and deliver them to systems that are either temporarily unavailable or join at a later time.

The solution is to deploy a high-performance messaging middleware such as RTI Data-Distribution Service, IBM WebSphere MQ, TIBCO, or SonicMQ. These middleware platforms have been developed with scalability and performance in mind. However, they each employ a different architecture optimized for different application scenarios.

Why Does Messaging Performance Matter?
The requirements and expectations of computer-speed real-time far exceed traditional human-speed real-time. Whereas in systems with a human in the loop, real-time meant that the information was available anywhere from fractions of a second to few seconds in the computer-to-computer world, real-time means decisions should be made in milliseconds or even microseconds.

Computer real-time puts more stringent requirements on the messaging infrastructure: Each processing and storage component must get hundreds of thousands of messages/events per second with microsecond or at worst millisecond latencies. This means that the messaging middleware must be able to deliver millions of messages a second system-wide.

And the capacity of the messaging fabric must be able to scale with the capacity of the underlying hardware and not impose any limits beyond those of the underlying hardware infrastructure (CPU speed, cores, speed, and bandwidth of the network) itself. As the CPU and network speeds increase those systems able to take advantage of what the hardware provides will deliver a competitive advantage. In an automated trading system, for instance, the critical metric is not the absolute time it takes to make a decision, but rather whether a decision is taken and the trade executed before competitive trades occur. The same is true in a combat management system.

One final aspect of computer real-time SOA systems is their "inverted performance-load utility curve." This means that the ability to respond in a timely manner becomes more important when the system is experiencing a high load. In a normal utility curve, such as in human real-time systems, degraded performance is acceptable under an increased load. This is because human expectations and patience adjust based on the circumstances (e.g., they understand that on a peak holiday period they may endure longer hold times when calling to make a flight reservation). In contrast, computer-speed real-time systems often have the opposite demands. It is precisely at the moments of high load when the "most critical action" is taking place and it is then when it is most critical to deliver top performance (e.g., it is precisely when market action is heavy that trading decisions must be made quickly).

The differences between human-speed real-time systems and computer-speed real-time systems are summarized in Table 1.

Selecting Messaging Middleware in SOA Systems
Messaging middleware is the key enabler of real-time SOA. However, there are many options. How can you choose the best messaging middleware for a particular real-time SOA system? Five areas distinguish messaging middleware: architecture, quality of service (QoS) control and filters, performance-boosting technologies, real-time determinism, and metrics.

The four basic architectures employed by messaging middleware are: centralized (hub-and-spoke), clustered, federated, and peer-to-peer. (see Figure 2)

A centralized (hub-and-spoke) architecture routes every message though a single server that implements the message "service," contains all the message queues, and brokers every message.
A clustered architecture uses a collection of servers and assigns to each responsibility for some of the messages (like ownership of some of the message queues or topics). Each message is relayed by a server but not all messages use the same server.

A federated architecture also uses a collection of servers, but it uses them as a "resource pool" where queues may appear in multiple servers, and messages may be brokered by one or more servers.

A peer-to-peer architecture doesn't employ any brokers in the critical path. Messages are routed directly from the sender to the receiver.

Each has strengths and weaknesses. Centralized is easiest to administer and can provide stronger transactional semantics but suffers from poor performance, reduced tolerance to faults, and doesn't scale. Clustered is more scalable than centralized but also has reduced fault tolerance and can only offer good performance in a grid environment with all the clients co-located close to the grid. Federated is more scalable, but suffers from higher latency and jitter as each message is brokered by at least two servers. P2P offers the best scalability, performance, lowest jitter, and highest resilience, but is difficult for vendors to implement and offers limited transactional support.

As demands become more real-time, the need for performance, predictability, and balance tips the scale towards P2P architecture. That's why, for example, demanding networks like Voice over IP and Video over IP (like Skype) use peer-to-peer designs.

Quality of Service Control & Filters
QoS control is critical to deliver timely data with low latency and high throughput. CPU, memory, and network bandwidth resources must be shared among all the traffic. However, not all traffic requires the same bandwidth or has the same urgency or level or criticality. Without QoS control, the application has no way to differentiate different traffic classes and their corresponding constraints. As a consequence, the middleware can't make intelligent decisions, prioritize traffic, or ultimately meet the application requirements.

More Stories By Gerardo Pardo-Castellote

Gerardo Pardo-Castellote, PhD, is chief technology officer of Real-Time Innovations Inc.

Comments (3) View Comments

Share your thoughts on this story.

Add your comment
You must be signed in to add a comment. Sign-in | Register

In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.

Most Recent Comments
Gerardo Pardo-Castellote 07/20/08 01:57:08 AM EDT

Regarding the previous comment about "TCP not lining up a message on one connection after a file transfer on another connection." and the "information in the article not being correct."

This is true, but in order for this to occur you would need to open a new TCP connection for every message. This is extremely inefficient, requires a handshake involving a round-trip message, and allocates a lot of system resources. This is certainly something you do not want to do in a real-time system.

So in practice anybody developing a real-time system would have to hold the TCP connection open and send successive messages over it (or course one can keep more than one connection open, and round-robin among them but that does not change fundamental problem if the application is writing quickly). Therefore the information in the article IS correct.

Casual Visitor 06/12/08 03:04:45 PM EDT

TCP does not line up a message on one connection after a file transfer on another connection. Each TCP connection forms its own in-order transfer. If you want to convince people to buy your product, you should avoid putting incorrect information in the article. It is much better to have a good analysis with accurate claims so that people will believe that your product might overcome real problems rather than phantom ones like "messages wait behind file transfers".

Derek Pavatte 01/25/08 02:03:32 AM EST

If everything is automated, I suppose we will have more time to do things more pleasant things than work as much. These technological advancements sound very progressive. Let us all work towards a competent and ethical work environment.

@MicroservicesExpo Stories
DevOps is often described as a combination of technology and culture. Without both, DevOps isn't complete. However, applying the culture to outdated technology is a recipe for disaster; as response times grow and connections between teams are delayed by technology, the culture will die. A Nutanix Enterprise Cloud has many benefits that provide the needed base for a true DevOps paradigm.
As software becomes more and more complex, we, as software developers, have been splitting up our code into smaller and smaller components. This is also true for the environment in which we run our code: going from bare metal, to VMs to the modern-day Cloud Native world of containers, schedulers and micro services. While we have figured out how to run containerized applications in the cloud using schedulers, we've yet to come up with a good solution to bridge the gap between getting your contain...
As organizations realize the scope of the Internet of Things, gaining key insights from Big Data, through the use of advanced analytics, becomes crucial. However, IoT also creates the need for petabyte scale storage of data from millions of devices. A new type of Storage is required which seamlessly integrates robust data analytics with massive scale. These storage systems will act as “smart systems” provide in-place analytics that speed discovery and enable businesses to quickly derive meaningf...
DevOps is often described as a combination of technology and culture. Without both, DevOps isn't complete. However, applying the culture to outdated technology is a recipe for disaster; as response times grow and connections between teams are delayed by technology, the culture will die. A Nutanix Enterprise Cloud has many benefits that provide the needed base for a true DevOps paradigm. In his Day 3 Keynote at 20th Cloud Expo, Chris Brown, a Solutions Marketing Manager at Nutanix, will explore t...
DevOps has often been described in terms of CAMS: Culture, Automation, Measuring, Sharing. While we’ve seen a lot of focus on the “A” and even on the “M”, there are very few examples of why the “C" is equally important in the DevOps equation. In her session at @DevOps Summit, Lori MacVittie, of F5 Networks, explored HTTP/1 and HTTP/2 along with Microservices to illustrate why a collaborative culture between Dev, Ops, and the Network is critical to ensuring success.
With major technology companies and startups seriously embracing Cloud strategies, now is the perfect time to attend @CloudExpo | @ThingsExpo, June 6-8, 2017, at the Javits Center in New York City, NY and October 31 - November 2, 2017, Santa Clara Convention Center, CA. Learn what is going on, contribute to the discussions, and ensure that your enterprise is on the right path to Digital Transformation.
Everyone wants to use containers, but monitoring containers is hard. New ephemeral architecture introduces new challenges in how monitoring tools need to monitor and visualize containers, so your team can make sense of everything. In his session at @DevOpsSummit, David Gildeh, co-founder and CEO of Outlyer, will go through the challenges and show there is light at the end of the tunnel if you use the right tools and understand what you need to be monitoring to successfully use containers in your...
What if you could build a web application that could support true web-scale traffic without having to ever provision or manage a single server? Sounds magical, and it is! In his session at 20th Cloud Expo, Chris Munns, Senior Developer Advocate for Serverless Applications at Amazon Web Services, will show how to build a serverless website that scales automatically using services like AWS Lambda, Amazon API Gateway, and Amazon S3. We will review several frameworks that can help you build serverle...
The IT industry is undergoing a significant evolution to keep up with cloud application demand. We see this happening as a mindset shift, from traditional IT teams to more well-rounded, cloud-focused job roles. The IT industry has become so cloud-minded that Gartner predicts that by 2020, this cloud shift will impact more than $1 trillion of global IT spending. This shift, however, has left some IT professionals feeling a little anxious about what lies ahead. The good news is that cloud computin...
SYS-CON Events announced today that HTBase will exhibit at SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. HTBase (Gartner 2016 Cool Vendor) delivers a Composable IT infrastructure solution architected for agility and increased efficiency. It turns compute, storage, and fabric into fluid pools of resources that are easily composed and re-composed to meet each application’s needs. With HTBase, companies can quickly prov...
An overall theme of Cloud computing and the specific practices within it is fundamentally one of automation. The core value of technology is to continually automate low level procedures to free up people to work on more value add activities, ultimately leading to the utopian goal of full Autonomic Computing. For example a great way to define your plan for DevOps tool chain adoption is through this lens. In this TechTarget article they outline a simple maturity model for planning this.
While DevOps most critically and famously fosters collaboration, communication, and integration through cultural change, culture is more of an output than an input. In order to actively drive cultural evolution, organizations must make substantial organizational and process changes, and adopt new technologies, to encourage a DevOps culture. Moderated by Andi Mann, panelists discussed how to balance these three pillars of DevOps, where to focus attention (and resources), where organizations might...
The rise of containers and microservices has skyrocketed the rate at which new applications are moved into production environments today. While developers have been deploying containers to speed up the development processes for some time, there still remain challenges with running microservices efficiently. Most existing IT monitoring tools don’t actually maintain visibility into the containers that make up microservices. As those container applications move into production, some IT operations t...
For organizations that have amassed large sums of software complexity, taking a microservices approach is the first step toward DevOps and continuous improvement / development. Integrating system-level analysis with microservices makes it easier to change and add functionality to applications at any time without the increase of risk. Before you start big transformation projects or a cloud migration, make sure these changes won’t take down your entire organization.
Software development is a moving target. You have to keep your eye on trends in the tech space that haven’t even happened yet just to stay current. Consider what’s happened with augmented reality (AR) in this year alone. If you said you were working on an AR app in 2015, you might have gotten a lot of blank stares or jokes about Google Glass. Then Pokémon GO happened. Like AR, the trends listed below have been building steam for some time, but they’ll be taking off in surprising new directions b...
@DevOpsSummit has been named the ‘Top DevOps Influencer' by iTrend. iTrend processes millions of conversations, tweets, interactions, news articles, press releases, blog posts - and extract meaning form them and analyzes mobile and desktop software platforms used to communicate, various metadata (such as geo location), and automation tools. In overall placement, @DevOpsSummit ranked as the number one ‘DevOps Influencer' followed by @CloudExpo at third, and @MicroservicesE at 24th.
After more than five years of DevOps, definitions are evolving, boundaries are expanding, ‘unicorns’ are no longer rare, enterprises are on board, and pundits are moving on. Can we now look at an evolution of DevOps? Should we? Is the foundation of DevOps ‘done’, or is there still too much left to do? What is mature, and what is still missing? What does the next 5 years of DevOps look like? In this Power Panel at DevOps Summit, moderated by DevOps Summit Conference Chair Andi Mann, panelists l...
Culture is the most important ingredient of DevOps. The challenge for most organizations is defining and communicating a vision of beneficial DevOps culture for their organizations, and then facilitating the changes needed to achieve that. Often this comes down to an ability to provide true leadership. As a CIO, are your direct reports IT managers or are they IT leaders? The hard truth is that many IT managers have risen through the ranks based on their technical skills, not their leadership abi...
The essence of cloud computing is that all consumable IT resources are delivered as services. In his session at 15th Cloud Expo, Yung Chou, Technology Evangelist at Microsoft, demonstrated the concepts and implementations of two important cloud computing deliveries: Infrastructure as a Service (IaaS) and Platform as a Service (PaaS). He discussed from business and technical viewpoints what exactly they are, why we care, how they are different and in what ways, and the strategies for IT to transi...
Thanks to Docker and the DevOps revolution, microservices have emerged as the new way to build and deploy applications — and there are plenty of great reasons to embrace the microservices trend. If you are going to adopt microservices, you also have to understand that microservice architectures have many moving parts. When it comes to incident management, this presents an important difference between microservices and monolithic architectures. More moving parts mean more complexity to monitor an...