Microservices Expo Authors: Liz McMillan, Pat Romanski, Carmen Gonzalez, Elizabeth White, Jason Bloomberg

Related Topics: Microservices Expo

Microservices Expo: Blog Feed Post

Optimize Prime: The Self-Optimizing Application Delivery Network

Infrastructure 2.0 enabled application delivery platforms have more than a few things in common with the Transformers

Infrastructure 2.0 Journal on Ulitzer

Infrastructure 2.0 enabled application delivery platforms have more than a few things in common with the Transformers. Like Autobots, there’s more to it than meets the eye.

If you’re familiar with the mythology of the Transformers – and perhaps even if you aren’t – you know that they key attribute of Transformers is their ability to take on “alternate modes” such as cars, trucks, and winged vehicles simply by scanning the object and then adapting their own form to match.

One of the key premises of Infrastructure 2.0 is also the ability of network and application networking solutions to adapt to their environments. While they won’t be transforming their physical manifestations into some other device they can transform their configurations based on the environment in which they are deployed. Like the Transformers ability to take on alternate modes, the ability to react in real-time is a native capability of Infrastructure 2.0 solutions and should not be overlooked by those integrating Infrastructure 2.0 into their cloud-based architectures.

While everyone seems aware of the capability of Infrastructure 2.0 to be managed and integrated with the rest of a cloud-based ecosystem via a standards-based control-plane API, there’s more to infrastructure 2.0 than meets the eye, here. That same dynamic control plane can be used at run-time to transform configuration and policies to better match customer need for balancing of performance and cost across the application infrastructure. That’s the transformative power of infrastructure 2.0, and what will certainly be core to the next generation of network management systems when trying to enforce SLAs across applications, data centers, and cloud computing environments, a.k.a. cloud balancing.

Now I doubt that anytime in the future we’ll hear application delivery controllers describe themselves as autonomous networking organisms from <insert vendor city here> still there are enough similarities between a self-optimizing application delivery network and a Transformer to run with the analogy – and as long as I have the opportunity to legitimately include a picture of Optimus Prime in my blog, well, I’m going to take it.


In the case of application delivery the “transformation” that makes this analogy work may involve many different functionalities: security, acceleration and optimization, core load balancing. Today we’re focusing on the load balancing algorithm, specifically, as the use of load balancing in cloud computing environments in order to achieve “elastic scalability” is a requirement. Unfortunately there is very little time spent on the unique challenges associated with load balancing applications executing in environments with varied compute resource capabilities. One of the mantras of cloud computing is the use of otherwise idle resources to provide the additional compute power necessary to scale an application. What this ignores is that these idle resources may very well be of different capacities in terms of CPU and RAM available. By pooling together these “servers” of varied capacities, it creates a heterogeneous environment which in turn directly impacts the entire application delivery chain. Of particular note should be the load balancing algorithm used to distribute requests across the pool of “servers.”

The problem is that by dynamically adding a server with a different CPU and RAM configuration – whether virtual or physical – to the “pool” of resources across which the Load balancer is distributing requests it changes how effective that algorithm is which in turn impacts application performance and can, unfortunately, actually render smaller instances of servers unavailable in short order. Also a possibility is that it will overwhelm the smaller server before the larger servers, which could – depending on how you have your environment configured – lead to the launching of another small server which, of course, incurs more operating costs.

Consider you have three “super size” servers, all with the same RAM and CPU capacity. A spike in use is anticipated because of some EVENT but not so much that you need a super size server, a “regular sized” server will suffice. You provision it. The spike in use occurs and then the load balancer, which has been distributing traffic based on a round robin algorithm, overwhelms the regular sized server causing timeouts, delays, and other availability and performance related problems for visitors.

What happened? The load balancing algorithm, which was perfectly suited for a homogeneous environment, was not so well-suited to a heterogeneous environment. In fact, it was downright wrong for a heterogeneous environment. What happened is that no one took into consideration that the infrastructure, optimized for a given environment, might not be so optimized if that environment changed and did not appropriately modify the load balancing configuration.


There are a number of solutions that address this particular challenge:

1 Provision homogeneously
If the load balancing algorithm you are using is optimized inherently for a homogeneous environment, then never deviate from that. Ever.

2 Human intervention
Manually change the load balancing algorithm when new servers are added, then change it back when it’s released.

3 Automate
Employ the collaborative nature of a dynamic control plane to automatically recognize the addition of a server that creates a heterogeneous environment and dynamically change the load balancing algorithm to one better suited to a heterogeneous environment, then reverse the change when the environment returns to a homogeneous one.

The load balancing algorithm that might be right for one application might not be right for another, depending on the style of application, its usage patterns, the servers used to serve it, and even the time of year. And changing any of those variables can have an impact on the behavior of the application because it directly impacts the load balancer.

Unfortunately we’re not quite at the point where the load balancer can automatically determine the right load balancing algorithm for you, but there are ways to adjust – dynamically – the algorithm based on the capabilities of the servers (physical and/or virtual) being load balanced so one day it is quite possible that through the magic of Infrastructure 2.0, load balancing algorithms will be modified on-demand based on the type of servers that make up the pool of resources. Today, if you know which algorithm is best given a specific set of resources you can codify the change such that it is automated; it’s only the choice of algorithm that can’t be, today, automatically determined. You probably could develop a system that does automatically determine through trial and error and monitoring of response times and capabilities, but it would not be a trivial task.

In order for application delivery infrastructure to automatically detect and optimize load balancing algorithms itself it’s necessary to first understand the impact of the load balancing algorithm on applications and determine which one is best able to meet the service level agreements in various environments. This will become more important as public and private cloud computing environments are leveraged in new ways and introduce more heterogeneous environments. Seasonal demand might, for example, be met by leveraging different “sizes” of unused capacity across multiple servers in the data center. These “servers” would likely be of different CPU and RAM capabilities and thus would certainly be impacted by the choice of load balancing algorithm. Being able to dynamically modify the load balancing algorithm based on the capacities of application instances is an invaluable tool when attempting to maximize the efficiency of resources while minimizing associated costs. Infrastructure 2.0 enabled load balancing solutions are capable of this level of automation; what they can’t do, yet, is decide which load balancing algorithm to apply. But if you know which one to apply – because you’ve tested and you know, right? – then you can automate the change based on triggers you specify, such as the addition of a server with CPU and RAM that turns a homogeneous environment into a heterogeneous environment. And vice-versa.

Virtualization and cloud computing are definitely game changers. But they not only change the basic rules of the game, they also change the strategy with which you must approach the game. It’s like moving from checkers to chess. There are a great many more moves you can make, and you’ve got to carefully consider how this move right now will impact a move you may need to make later on. One of the most important parts of that new strategy must be to recognize that while the ability to automate provisioning and integrate with the rest of the infrastructure is certainly a key benefit of infrastructure 2.0, just as beneficial is the ability to adjust and optimize the delivery of applications in real time.

Read the original blog entry...

More Stories By Lori MacVittie

Lori MacVittie is responsible for education and evangelism of application services available across F5’s entire product suite. Her role includes authorship of technical materials and participation in a number of community-based forums and industry standards organizations, among other efforts. MacVittie has extensive programming experience as an application architect, as well as network and systems development and administration expertise. Prior to joining F5, MacVittie was an award-winning Senior Technology Editor at Network Computing Magazine, where she conducted product research and evaluation focused on integration with application and network architectures, and authored articles on a variety of topics aimed at IT professionals. Her most recent area of focus included SOA-related products and architectures. She holds a B.S. in Information and Computing Science from the University of Wisconsin at Green Bay, and an M.S. in Computer Science from Nova Southeastern University.

Microservices Articles
Modern software design has fundamentally changed how we manage applications, causing many to turn to containers as the new virtual machine for resource management. As container adoption grows beyond stateless applications to stateful workloads, the need for persistent storage is foundational - something customers routinely cite as a top pain point. In his session at @DevOpsSummit at 21st Cloud Expo, Bill Borsari, Head of Systems Engineering at Datera, explored how organizations can reap the bene...
"NetApp's vision is how we help organizations manage data - delivering the right data in the right place, in the right time, to the people who need it, and doing it agnostic to what the platform is," explained Josh Atwell, Developer Advocate for NetApp, in this SYS-CON.tv interview at 20th Cloud Expo, held June 6-8, 2017, at the Javits Center in New York City, NY.
The Jevons Paradox suggests that when technological advances increase efficiency of a resource, it results in an overall increase in consumption. Writing on the increased use of coal as a result of technological improvements, 19th-century economist William Stanley Jevons found that these improvements led to the development of new ways to utilize coal. In his session at 19th Cloud Expo, Mark Thiele, Chief Strategy Officer for Apcera, compared the Jevons Paradox to modern-day enterprise IT, examin...
In his session at 20th Cloud Expo, Mike Johnston, an infrastructure engineer at Supergiant.io, discussed how to use Kubernetes to set up a SaaS infrastructure for your business. Mike Johnston is an infrastructure engineer at Supergiant.io with over 12 years of experience designing, deploying, and maintaining server and workstation infrastructure at all scales. He has experience with brick and mortar data centers as well as cloud providers like Digital Ocean, Amazon Web Services, and Rackspace. H...
Skeuomorphism usually means retaining existing design cues in something new that doesn’t actually need them. However, the concept of skeuomorphism can be thought of as relating more broadly to applying existing patterns to new technologies that, in fact, cry out for new approaches. In his session at DevOps Summit, Gordon Haff, Senior Cloud Strategy Marketing and Evangelism Manager at Red Hat, will discuss why containers should be paired with new architectural practices such as microservices ra...
In his session at 20th Cloud Expo, Scott Davis, CTO of Embotics, discussed how automation can provide the dynamic management required to cost-effectively deliver microservices and container solutions at scale. He also discussed how flexible automation is the key to effectively bridging and seamlessly coordinating both IT and developer needs for component orchestration across disparate clouds – an increasingly important requirement at today’s multi-cloud enterprise.
The Software Defined Data Center (SDDC), which enables organizations to seamlessly run in a hybrid cloud model (public + private cloud), is here to stay. IDC estimates that the software-defined networking market will be valued at $3.7 billion by 2016. Security is a key component and benefit of the SDDC, and offers an opportunity to build security 'from the ground up' and weave it into the environment from day one. In his session at 16th Cloud Expo, Reuven Harrison, CTO and Co-Founder of Tufin, ...
DevOps is often described as a combination of technology and culture. Without both, DevOps isn't complete. However, applying the culture to outdated technology is a recipe for disaster; as response times grow and connections between teams are delayed by technology, the culture will die. A Nutanix Enterprise Cloud has many benefits that provide the needed base for a true DevOps paradigm. In their Day 3 Keynote at 20th Cloud Expo, Chris Brown, a Solutions Marketing Manager at Nutanix, and Mark Lav...
Many organizations are now looking to DevOps maturity models to gauge their DevOps adoption and compare their maturity to their peers. However, as enterprise organizations rush to adopt DevOps, moving past experimentation to embrace it at scale, they are in danger of falling into the trap that they have fallen into time and time again. Unfortunately, we've seen this movie before, and we know how it ends: badly.
TCP (Transmission Control Protocol) is a common and reliable transmission protocol on the Internet. TCP was introduced in the 70s by Stanford University for US Defense to establish connectivity between distributed systems to maintain a backup of defense information. At the time, TCP was introduced to communicate amongst a selected set of devices for a smaller dataset over shorter distances. As the Internet evolved, however, the number of applications and users, and the types of data accessed and...