Welcome!

Microservices Expo Authors: Flint Brenton, Liz McMillan, Elizabeth White, Charles Araujo, Ed Witkovic

Related Topics: @DevOpsSummit, Microservices Expo

@DevOpsSummit: Blog Feed Post

Four Ways to Load Test Your API | @DevOpsSummit #API #DevOps #Microservices

How do you make sure your API can handle even the heaviest load?

By Les Worley

You know the feeling. You're surfing along and then BAM! you get the dreaded

503 (Service unavailable)

The server is currently unavailable (because it is overloaded or

down for maintenance).

Service-Unavailable

Do you remember what you were searching for the last time that happened? Or what company you would have bought from? Probably not.

Worse, if you had just clicked on "Submit Order" and waited and waited... and then got such an error... Did you have any idea whether the order went through, or if your credit card was charged??

Chances are that merchant lost you as a customer forever. You Google'd for someone else and never looked back.

Of course, as a developer you want your service to be heavily used. Otherwise, what's the point? But loads depend on many things: time of day, end of month processing, overnight batch runs, press releases, product launches - you name it.

So how do you make sure your API can handle even the heaviest load? How do you keep your customers from walking away?

Don't Lose Authority from an Unreliable API
That's one example - from a consumer point of view - of an API service that couldn't handle the load. It resulted in a lost customer.

When you publish your web service, you want to be the one that everyone looks to as the authority - especially if they pay to use the API.

Perhaps your API is just for "internal use." But there's always someone that relies on the results. Maybe the web team uses your API to search your company's product catalog. They then display the results to your customers. Busy server, no results - no sale. Maybe your API is not so internal after all.

Either way, the last thing you want is for your users to be screaming about an error like

  • Service unavailable
  • Connection rejected
  • Server timed out
  • Or "Unknown error occurred"

If you're the consumer of a service, like a financial transaction API, you want your app to handle any exception that API throws. But as the API publisher, it's up to you to make sure your API can handle heavy loads so it doesn't throw those exceptions. Or if it does, it does so gracefully and in a well-defined and documented manner.

If you can't guarantee your users predictability under heavy load, you'll lose that authority. They'll pay someone else for a more reliable API.

In other words, you need to test the heck out of it.

There are three typical approaches to load testing an API. These, as well as various hybrids, each have drawbacks.

#1 Faking it - API Mocking
The least useful for load testing are API mocks. These are basically temporary placeholders used during development of an API. Developers use these for early unit testing. But they also provide these "dummy" versions to other teams that need to call the unfinished API from within their own code.

Mocks return hard-coded responses, the formats of which often change during development. They have little "meat" behind them, only canned responses that can be served up immediately. And they are disposable - once the actual API is completed, the code is thrown away. In other words, the effort to create the mocks is wasted after their initial use.

And because mocks aren't yet hooked up to actual data sources, their performance isn't at all representative of the real world.

In summary, mocks may be useful for unit testing, but they aren't representative of the real world.

#2 Cloning It - A Full Test Environment
The mock API approach assumes the API isn't actually complete, or that there's no live data source available. If the API is complete and ready for testing, we can use the clone approach. This approach stands up a full-blown test environment using a snapshot of production data.

disadvantages-of-cloned-environments

Often you'll have a copy of all the applications running, too, because your code relies on other APIs, not just database queries.

The good in this approach is that it's very representative of the live system's performance. If your load test brings the system to its knees, no problem! It hurts no one.

The bad parts outweigh the good though. For one thing, production data often contains sensitive customer information, so data privacy regulations can come into play. Also, if your own API makes use of pay-per-use services, this testing can get very expensive. Finally, production data is stateful, even in a cloned environment. If you need to re-run the tests, you'll have to reload the test bed each time.

In summary, cloned environments give you near-production-quality load test results, without harming production if your test causes it to crater. However, they raise privacy concerns, require constant data reloading, and can be very expensive if calling third party APIs.

#3 Hitting production - Load Testing Live APIs
Sadly, the method so many companies use is to test on production itself. Why? Well, it's live, so it's the most representative of all methods. It's also "easy" -  no separate test environment to stand up and maintain. But that's where the usefulness ends.

hitting_production_is_easiest_but

In production, you can't use the "real" data. Instead you have to maintain separate "test accounts." Using test accounts in a live environment reduces privacy concerns to a degree. These data are stateful, of course, so you have to reset them before each test run. And you still have to worry about the cost of pay-per-use APIs.

It's important to schedule your load test "off hours." This can be tricky, since the Internet allows your customers access 24/7. Even then, if your load test brings the server down, or corrupts production data, you're fired (or you'll soon wish you were).

Remember, Service unavailable signals your customers and their customers that your site - or your API service - is unreliable.

#4 API Virtualization - The Risk-free Approach
You want to perform an exhaustive load test of your API - and keep your existing customers and your job. That rules out hitting the production system.

You also need more than a test version of your API in a cloned environment. You need to test the real API in an environment where you can control the load condition for aspect of the test.

That's where API virtualization comes in. A virtual API provides a sandbox environment where you can simulate environmental loads on network bandwidth, server and database connections, simultaneous users and more.

With API virtualization, minimal resources are required for standing up and testing the API. You test the API itself, not the end-to-end application with all its required backend systems.  This eliminates the need for a clone of the production environment. That means you don't have to reset and reload downstream data anymore.

load-testing-with-api-virtualization

The data-in and data-out can be as real as you wish. You can create your own requests and responses to test against, or capture and store actual requests and responses from a known source. The data can be played back during the load test, firing the requests as fast or as slow as you wish.

And those pay-per-use APIs? You can virtualize those, too, simulating a variety of responses and latency that the actual API could introduce.

The result? You can test your API in isolation, gauging the reaction against real data,  under a variety of load conditions. And you can fine tune it until it passes with flying colors. No production system required or harmed.

Don't give in to the strain
Hammering an API in production to simulate a heavy request loads isn't wise. You can end up angering - and losing - real customers, not to mention your job. Not to mention affecting countless downstream systems.

API Virtualization allows you to load test your API in isolation from the rest of the system. By configuring requests and responses - or capturing real ones - your API can respond to a variety of requests with any number of responses - good and bad. With complete control of the test conditions, you can simulate network load, maximum connection limits, latencies and other conditions that happen in the real world.

Knowing how your API will respond under stress allows you to fix it, tune it, optimize it - before you unleash it for your customers to use.

Read the original blog entry...

More Stories By SmartBear Blog

As the leader in software quality tools for the connected world, SmartBear supports more than two million software professionals and over 25,000 organizations in 90 countries that use its products to build and deliver the world’s greatest applications. With today’s applications deploying on mobile, Web, desktop, Internet of Things (IoT) or even embedded computing platforms, the connected nature of these applications through public and private APIs presents a unique set of challenges for developers, testers and operations teams. SmartBear's software quality tools assist with code review, functional and load testing, API readiness as well as performance monitoring of these modern applications.

@MicroservicesExpo Stories
Don’t go chasing waterfall … development, that is. According to a recent post by Madison Moore on Medium featuring insights from several software delivery industry leaders, waterfall is – while still popular – not the best way to win in the marketplace. With methodologies like Agile, DevOps and Continuous Delivery becoming ever more prominent over the past 15 years or so, waterfall is old news. Or, is it? Moore cites a recent study by Gartner: “According to Gartner’s IT Key Metrics Data report, ...
Agile has finally jumped the technology shark, expanding outside the software world. Enterprises are now increasingly adopting Agile practices across their organizations in order to successfully navigate the disruptive waters that threaten to drown them. In our quest for establishing change as a core competency in our organizations, this business-centric notion of Agile is an essential component of Agile Digital Transformation. In the years since the publication of the Agile Manifesto, the conn...
In his keynote at 19th Cloud Expo, Sheng Liang, co-founder and CEO of Rancher Labs, discussed the technological advances and new business opportunities created by the rapid adoption of containers. With the success of Amazon Web Services (AWS) and various open source technologies used to build private clouds, cloud computing has become an essential component of IT strategy. However, users continue to face challenges in implementing clouds, as older technologies evolve and newer ones like Docker c...
The next XaaS is CICDaaS. Why? Because CICD saves developers a huge amount of time. CD is an especially great option for projects that require multiple and frequent contributions to be integrated. But… securing CICD best practices is an emerging, essential, yet little understood practice for DevOps teams and their Cloud Service Providers. The only way to get CICD to work in a highly secure environment takes collaboration, patience and persistence. Building CICD in the cloud requires rigorous ar...
"This all sounds great. But it's just not realistic." This is what a group of five senior IT executives told me during a workshop I held not long ago. We were working through an exercise on the organizational characteristics necessary to successfully execute a digital transformation, and the group was doing their ‘readout.' The executives loved everything we discussed and agreed that if such an environment existed, it would make transformation much easier. They just didn't believe it was reali...
All organizations that did not originate this moment have a pre-existing culture as well as legacy technology and processes that can be more or less amenable to DevOps implementation. That organizational culture is influenced by the personalities and management styles of Executive Management, the wider culture in which the organization is situated, and the personalities of key team members at all levels of the organization. This culture and entrenched interests usually throw a wrench in the work...
"Opsani helps the enterprise adopt containers, help them move their infrastructure into this modern world of DevOps, accelerate the delivery of new features into production, and really get them going on the container path," explained Ross Schibler, CEO of Opsani, and Peter Nickolov, CTO of Opsani, in this SYS-CON.tv interview at DevOps Summit at 21st Cloud Expo, held Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA.
The purpose of this article is draw attention to key SaaS services that are commonly overlooked during contact signing that are essential to ensuring they meet the expectations and requirements of the organization and provide guidance and recommendations for process and controls necessary for achieving quality SaaS contractual agreements.
What's the role of an IT self-service portal when you get to continuous delivery and Infrastructure as Code? This general session showed how to create the continuous delivery culture and eight accelerators for leading the change. Don Demcsak is a DevOps and Cloud Native Modernization Principal for Dell EMC based out of New Jersey. He is a former, long time, Microsoft Most Valuable Professional, specializing in building and architecting Application Delivery Pipelines for hybrid legacy, and cloud ...
The “Digital Era” is forcing us to engage with new methods to build, operate and maintain applications. This transformation also implies an evolution to more and more intelligent applications to better engage with the customers, while creating significant market differentiators. In both cases, the cloud has become a key enabler to embrace this digital revolution. So, moving to the cloud is no longer the question; the new questions are HOW and WHEN. To make this equation even more complex, most ...
CloudEXPO New York 2018, colocated with DXWorldEXPO New York 2018 will be held November 11-13, 2018, in New York City and will bring together Cloud Computing, FinTech and Blockchain, Digital Transformation, Big Data, Internet of Things, DevOps, AI, Machine Learning and WebRTC to one location.
Docker is sweeping across startups and enterprises alike, changing the way we build and ship applications. It's the most prominent and widely known software container platform, and it's particularly useful for eliminating common challenges when collaborating on code (like the "it works on my machine" phenomenon that most devs know all too well). With Docker, you can run and manage apps side-by-side - in isolated containers - resulting in better compute density. It's something that many developer...
In his keynote at 19th Cloud Expo, Sheng Liang, co-founder and CEO of Rancher Labs, discussed the technological advances and new business opportunities created by the rapid adoption of containers. With the success of Amazon Web Services (AWS) and various open source technologies used to build private clouds, cloud computing has become an essential component of IT strategy. However, users continue to face challenges in implementing clouds, as older technologies evolve and newer ones like Docker c...
"We're developing a software that is based on the cloud environment and we are providing those services to corporations and the general public," explained Seungmin Kim, CEO/CTO of SM Systems Inc., in this SYS-CON.tv interview at 21st Cloud Expo, held Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA.
We all know that end users experience the internet primarily with mobile devices. From an app development perspective, we know that successfully responding to the needs of mobile customers depends on rapid DevOps – failing fast, in short, until the right solution evolves in your customers' relationship to your business. Whether you’re decomposing an SOA monolith, or developing a new application cloud natively, it’s not a question of using microservices - not doing so will be a path to eventual ...
Explosive growth in connected devices. Enormous amounts of data for collection and analysis. Critical use of data for split-second decision making and actionable information. All three are factors in making the Internet of Things a reality. Yet, any one factor would have an IT organization pondering its infrastructure strategy. How should your organization enhance its IT framework to enable an Internet of Things implementation? In his session at @ThingsExpo, James Kirkland, Red Hat's Chief Archi...
Containers and Kubernetes allow for code portability across on-premise VMs, bare metal, or multiple cloud provider environments. Yet, despite this portability promise, developers may include configuration and application definitions that constrain or even eliminate application portability. In this session we'll describe best practices for "configuration as code" in a Kubernetes environment. We will demonstrate how a properly constructed containerized app can be deployed to both Amazon and Azure ...
We all know that end users experience the internet primarily with mobile devices. From an app development perspective, we know that successfully responding to the needs of mobile customers depends on rapid DevOps – failing fast, in short, until the right solution evolves in your customers' relationship to your business. Whether you’re decomposing an SOA monolith, or developing a new application cloud natively, it’s not a question of using microservices - not doing so will be a path to eventual ...
We all know that end users experience the Internet primarily with mobile devices. From an app development perspective, we know that successfully responding to the needs of mobile customers depends on rapid DevOps – failing fast, in short, until the right solution evolves in your customers' relationship to your business. Whether you’re decomposing an SOA monolith, or developing a new application cloud natively, it’s not a question of using microservices – not doing so will be a path to eventual b...
The past few years have brought a sea change in the way applications are architected, developed, and consumed—increasing both the complexity of testing and the business impact of software failures. How can software testing professionals keep pace with modern application delivery, given the trends that impact both architectures (cloud, microservices, and APIs) and processes (DevOps, agile, and continuous delivery)? This is where continuous testing comes in. D