Microservices Expo Authors: Elizabeth White, Liz McMillan, Zakia Bouachraoui, Pat Romanski, Yeshim Deniz

Related Topics: Microservices Expo

Microservices Expo: Blog Feed Post

Optimize Prime: The Self-Optimizing Application Delivery Network

Infrastructure 2.0 enabled application delivery platforms have more than a few things in common with the Transformers

Infrastructure 2.0 Journal on Ulitzer

Infrastructure 2.0 enabled application delivery platforms have more than a few things in common with the Transformers. Like Autobots, there’s more to it than meets the eye.

If you’re familiar with the mythology of the Transformers – and perhaps even if you aren’t – you know that they key attribute of Transformers is their ability to take on “alternate modes” such as cars, trucks, and winged vehicles simply by scanning the object and then adapting their own form to match.

One of the key premises of Infrastructure 2.0 is also the ability of network and application networking solutions to adapt to their environments. While they won’t be transforming their physical manifestations into some other device they can transform their configurations based on the environment in which they are deployed. Like the Transformers ability to take on alternate modes, the ability to react in real-time is a native capability of Infrastructure 2.0 solutions and should not be overlooked by those integrating Infrastructure 2.0 into their cloud-based architectures.

While everyone seems aware of the capability of Infrastructure 2.0 to be managed and integrated with the rest of a cloud-based ecosystem via a standards-based control-plane API, there’s more to infrastructure 2.0 than meets the eye, here. That same dynamic control plane can be used at run-time to transform configuration and policies to better match customer need for balancing of performance and cost across the application infrastructure. That’s the transformative power of infrastructure 2.0, and what will certainly be core to the next generation of network management systems when trying to enforce SLAs across applications, data centers, and cloud computing environments, a.k.a. cloud balancing.

Now I doubt that anytime in the future we’ll hear application delivery controllers describe themselves as autonomous networking organisms from <insert vendor city here> still there are enough similarities between a self-optimizing application delivery network and a Transformer to run with the analogy – and as long as I have the opportunity to legitimately include a picture of Optimus Prime in my blog, well, I’m going to take it.


In the case of application delivery the “transformation” that makes this analogy work may involve many different functionalities: security, acceleration and optimization, core load balancing. Today we’re focusing on the load balancing algorithm, specifically, as the use of load balancing in cloud computing environments in order to achieve “elastic scalability” is a requirement. Unfortunately there is very little time spent on the unique challenges associated with load balancing applications executing in environments with varied compute resource capabilities. One of the mantras of cloud computing is the use of otherwise idle resources to provide the additional compute power necessary to scale an application. What this ignores is that these idle resources may very well be of different capacities in terms of CPU and RAM available. By pooling together these “servers” of varied capacities, it creates a heterogeneous environment which in turn directly impacts the entire application delivery chain. Of particular note should be the load balancing algorithm used to distribute requests across the pool of “servers.”

The problem is that by dynamically adding a server with a different CPU and RAM configuration – whether virtual or physical – to the “pool” of resources across which the Load balancer is distributing requests it changes how effective that algorithm is which in turn impacts application performance and can, unfortunately, actually render smaller instances of servers unavailable in short order. Also a possibility is that it will overwhelm the smaller server before the larger servers, which could – depending on how you have your environment configured – lead to the launching of another small server which, of course, incurs more operating costs.

Consider you have three “super size” servers, all with the same RAM and CPU capacity. A spike in use is anticipated because of some EVENT but not so much that you need a super size server, a “regular sized” server will suffice. You provision it. The spike in use occurs and then the load balancer, which has been distributing traffic based on a round robin algorithm, overwhelms the regular sized server causing timeouts, delays, and other availability and performance related problems for visitors.

What happened? The load balancing algorithm, which was perfectly suited for a homogeneous environment, was not so well-suited to a heterogeneous environment. In fact, it was downright wrong for a heterogeneous environment. What happened is that no one took into consideration that the infrastructure, optimized for a given environment, might not be so optimized if that environment changed and did not appropriately modify the load balancing configuration.


There are a number of solutions that address this particular challenge:

1 Provision homogeneously
If the load balancing algorithm you are using is optimized inherently for a homogeneous environment, then never deviate from that. Ever.

2 Human intervention
Manually change the load balancing algorithm when new servers are added, then change it back when it’s released.

3 Automate
Employ the collaborative nature of a dynamic control plane to automatically recognize the addition of a server that creates a heterogeneous environment and dynamically change the load balancing algorithm to one better suited to a heterogeneous environment, then reverse the change when the environment returns to a homogeneous one.

The load balancing algorithm that might be right for one application might not be right for another, depending on the style of application, its usage patterns, the servers used to serve it, and even the time of year. And changing any of those variables can have an impact on the behavior of the application because it directly impacts the load balancer.

Unfortunately we’re not quite at the point where the load balancer can automatically determine the right load balancing algorithm for you, but there are ways to adjust – dynamically – the algorithm based on the capabilities of the servers (physical and/or virtual) being load balanced so one day it is quite possible that through the magic of Infrastructure 2.0, load balancing algorithms will be modified on-demand based on the type of servers that make up the pool of resources. Today, if you know which algorithm is best given a specific set of resources you can codify the change such that it is automated; it’s only the choice of algorithm that can’t be, today, automatically determined. You probably could develop a system that does automatically determine through trial and error and monitoring of response times and capabilities, but it would not be a trivial task.

In order for application delivery infrastructure to automatically detect and optimize load balancing algorithms itself it’s necessary to first understand the impact of the load balancing algorithm on applications and determine which one is best able to meet the service level agreements in various environments. This will become more important as public and private cloud computing environments are leveraged in new ways and introduce more heterogeneous environments. Seasonal demand might, for example, be met by leveraging different “sizes” of unused capacity across multiple servers in the data center. These “servers” would likely be of different CPU and RAM capabilities and thus would certainly be impacted by the choice of load balancing algorithm. Being able to dynamically modify the load balancing algorithm based on the capacities of application instances is an invaluable tool when attempting to maximize the efficiency of resources while minimizing associated costs. Infrastructure 2.0 enabled load balancing solutions are capable of this level of automation; what they can’t do, yet, is decide which load balancing algorithm to apply. But if you know which one to apply – because you’ve tested and you know, right? – then you can automate the change based on triggers you specify, such as the addition of a server with CPU and RAM that turns a homogeneous environment into a heterogeneous environment. And vice-versa.

Virtualization and cloud computing are definitely game changers. But they not only change the basic rules of the game, they also change the strategy with which you must approach the game. It’s like moving from checkers to chess. There are a great many more moves you can make, and you’ve got to carefully consider how this move right now will impact a move you may need to make later on. One of the most important parts of that new strategy must be to recognize that while the ability to automate provisioning and integrate with the rest of the infrastructure is certainly a key benefit of infrastructure 2.0, just as beneficial is the ability to adjust and optimize the delivery of applications in real time.

Read the original blog entry...

More Stories By Lori MacVittie

Lori MacVittie is responsible for education and evangelism of application services available across F5’s entire product suite. Her role includes authorship of technical materials and participation in a number of community-based forums and industry standards organizations, among other efforts. MacVittie has extensive programming experience as an application architect, as well as network and systems development and administration expertise. Prior to joining F5, MacVittie was an award-winning Senior Technology Editor at Network Computing Magazine, where she conducted product research and evaluation focused on integration with application and network architectures, and authored articles on a variety of topics aimed at IT professionals. Her most recent area of focus included SOA-related products and architectures. She holds a B.S. in Information and Computing Science from the University of Wisconsin at Green Bay, and an M.S. in Computer Science from Nova Southeastern University.

Microservices Articles
In his general session at 19th Cloud Expo, Manish Dixit, VP of Product and Engineering at Dice, discussed how Dice leverages data insights and tools to help both tech professionals and recruiters better understand how skills relate to each other and which skills are in high demand using interactive visualizations and salary indicator tools to maximize earning potential. Manish Dixit is VP of Product and Engineering at Dice. As the leader of the Product, Engineering and Data Sciences team at D...
The now mainstream platform changes stemming from the first Internet boom brought many changes but didn’t really change the basic relationship between servers and the applications running on them. In fact, that was sort of the point. In his session at 18th Cloud Expo, Gordon Haff, senior cloud strategy marketing and evangelism manager at Red Hat, will discuss how today’s workloads require a new model and a new platform for development and execution. The platform must handle a wide range of rec...
When building large, cloud-based applications that operate at a high scale, it’s important to maintain a high availability and resilience to failures. In order to do that, you must be tolerant of failures, even in light of failures in other areas of your application. “Fly two mistakes high” is an old adage in the radio control airplane hobby. It means, fly high enough so that if you make a mistake, you can continue flying with room to still make mistakes. In his session at 18th Cloud Expo, Lee A...
Lori MacVittie is a subject matter expert on emerging technology responsible for outbound evangelism across F5's entire product suite. MacVittie has extensive development and technical architecture experience in both high-tech and enterprise organizations, in addition to network and systems administration expertise. Prior to joining F5, MacVittie was an award-winning technology editor at Network Computing Magazine where she evaluated and tested application-focused technologies including app secu...
Containers and Kubernetes allow for code portability across on-premise VMs, bare metal, or multiple cloud provider environments. Yet, despite this portability promise, developers may include configuration and application definitions that constrain or even eliminate application portability. In this session we'll describe best practices for "configuration as code" in a Kubernetes environment. We will demonstrate how a properly constructed containerized app can be deployed to both Amazon and Azure ...
Modern software design has fundamentally changed how we manage applications, causing many to turn to containers as the new virtual machine for resource management. As container adoption grows beyond stateless applications to stateful workloads, the need for persistent storage is foundational - something customers routinely cite as a top pain point. In his session at @DevOpsSummit at 21st Cloud Expo, Bill Borsari, Head of Systems Engineering at Datera, explored how organizations can reap the bene...
Using new techniques of information modeling, indexing, and processing, new cloud-based systems can support cloud-based workloads previously not possible for high-throughput insurance, banking, and case-based applications. In his session at 18th Cloud Expo, John Newton, CTO, Founder and Chairman of Alfresco, described how to scale cloud-based content management repositories to store, manage, and retrieve billions of documents and related information with fast and linear scalability. He addresse...
SYS-CON Events announced today that DatacenterDynamics has been named “Media Sponsor” of SYS-CON's 18th International Cloud Expo, which will take place on June 7–9, 2016, at the Javits Center in New York City, NY. DatacenterDynamics is a brand of DCD Group, a global B2B media and publishing company that develops products to help senior professionals in the world's most ICT dependent organizations make risk-based infrastructure and capacity decisions.
Discussions of cloud computing have evolved in recent years from a focus on specific types of cloud, to a world of hybrid cloud, and to a world dominated by the APIs that make today's multi-cloud environments and hybrid clouds possible. In this Power Panel at 17th Cloud Expo, moderated by Conference Chair Roger Strukhoff, panelists addressed the importance of customers being able to use the specific technologies they need, through environments and ecosystems that expose their APIs to make true ...
In his keynote at 19th Cloud Expo, Sheng Liang, co-founder and CEO of Rancher Labs, discussed the technological advances and new business opportunities created by the rapid adoption of containers. With the success of Amazon Web Services (AWS) and various open source technologies used to build private clouds, cloud computing has become an essential component of IT strategy. However, users continue to face challenges in implementing clouds, as older technologies evolve and newer ones like Docker c...