Welcome!

Microservices Expo Authors: Elizabeth White, Stackify Blog, Pat Romanski, Liz McMillan, Yeshim Deniz

Related Topics: @BigDataExpo, Java IoT, Microservices Expo, Microsoft Cloud, @CloudExpo

@BigDataExpo: Blog Feed Post

A New Breed of Actionable Analytics

Being smart about your monitoring data collection allows you to isolate and resolve problems much faster

Data is dumb but we can't seem to get enough of it these days. An entire industry has evolved to make sense of the massive amounts of data being generated every day. Massive data collection by itself does not guarantee the context required to solve business and IT problems... If we are smart about data, however, it will lead us in the right direction.

Today's businesses are defined by the software they run on, enabling them to innovate and create new services faster. Apple re-invented the music industry through software; Netflix changed the way we consume movies through software; grocery shopping, taxi-hire, parcel-logistics, car rental, travel booking... these are all industries that have been completely disrupted by software.

As more and more organisations depend upon the software they run on, the consumer demand for superior services, faster innovation, and improved performance continues to accelerate. As a result the software required to compete in this unforgiving market only continues to increase in its complexity.

It was only a few short years ago that when I wanted to book my family vacation, we all made the trip down to the local travel agent and sat around a desk while the agent tapped away on her green-screen searching for available flights and hotels. The process of booking our hotels has obviously changed dramatically, and so have the applications that support the process.

For example, below is a screen shot of the Business Transaction flow through a global travel site when a user makes a simple search for a hotel. All the user does is select a destination, some dates and maybe one or two preferences, but the resultant transaction kicks off more than 200 service requests before the results are returned to the user they can make their selection of preferred hotel.

Hotel Search

This rise in application complexity often means we generate increasingly more structured and unstructured data from our applications such as log files, email, alerts, infrastructure stats, network stats etc. Close to 200 billion emails will be sent in 2014, I wonder how many of those emails are false alerts from monitoring tools?

Keep nothing?

I have been helping companies build monitoring strategies for nearly a decade, and the problem I used to regularly face was dealing with organisations that just didn't have enough data. IT departments didn't have the required information to be able to quickly diagnose and remediate problems when they occurred.

Keep everything?

As organisations continue to have a greater dependency on the software they run on and application complexity continues to increase, there is a danger we swing the other way by collecting and storing too much data to be able to make sense of it in a timely manner. My colleague Jim Hirschauer often refers to this as the "home hoarders effect" drawing parallels with people who simply never throw anything out from their homes. The bigger the piles get the harder it is to find what you need, and at some point that pile is going to topple over.

Keep what you need

There are lots of good reasons to keep data, but "just in-case we need it" is not a good reason. It is important to think about why the data is being captured and stored and how it will help solve problems in the future. By keeping the relevant data and throwing away the clutter, you become more efficient and effective in troubleshooting and resolving problems.

But data alone isn't good enough to solve your business and IT problems. Data in and of itself is one-dimensional and dumb. Data doesn't tell you when there are problems, it doesn't tell you when business is going well, it doesn't tell you anything meaningful without some help and a lot more context. Let's explore an example to illustrate my point.

Let's say we have a couple of data points about a person. The data points are...

Heart Rate = 150 bpm

Blood Pressure = 200 over 100

Now tell me, is this person performing well? With these data points we have no idea. We need more data. The table below shows a list of some possible data points we can collect.

Screen Shot 2014-03-24 at 10.42.29 PM.png

Notice the last attribute in the table. The activity provides us with the context we need to focus on the proper data points to figure out if the person is performing well or not. Here are some more relevant data points...

Distance Run = 100 meters

Time = 9.58s

Now can we determine if the person is performing well or not? I'm not much of a track and field aficionado so I have no idea if this is a good performance or not. I need point of comparison to determine if 9.58 seconds in the 100-meter dash is good. So here is our baseline for comparison sake...

100-Meter World Record Time = 9.69s

Well it looks like the person was performing really well. They set a world record in the 100-meter dash. All of the data points individually didn't tell us anything. We required correlation (context) and analytics (comparison to baseline) in order to turn data into information. I like to refer to this concept as creating Smart Data.

Smart Data Defined

Smart Data is actionable, intelligent, information.

Smart Data is created by performing correlation and analytics on data sets. AppDynamics correlates end user business transaction details with completion status (success, error, exception), response times, and all other data points measured at any given time. It automatically analyzes the entire data set to provide information from which to draw conclusions and take the appropriate action. This information is called Smart Data.

Being smart about your monitoring data collection allows you to isolate and resolve problems much faster, and with a much lower cost of ownership and overhead.

I have recently been working with a company to replace their legacy monitoring tool with AppDynamics. The environment they are monitoring is a good size, consisting of about 1,200 servers and their main application processes approximately 300,000 transactions per minute. They invested in a monitoring tool to help manage the performance of their applications which captured and stored all the data they could "just in-case" it was needed. Unfortunately this approach required an additional 92 servers to be provisioned for the monitoring tool itself, which consumed approximately 80TB of data storage per year. The increasing investments this customer needed to make in hardware, storage, people, and maintenance was too much to manage for them and they decided to look for a different approach.

AppDynamics "smart data" approach to analytics means this particular customer now only requires two reporting servers and the storage requirements were reduced to just 1TB per year. Collecting only the data required to make smart decisions gave them both a 98% reduction in hardware costs and  more effective analytics in the process.

Adding business context

Smart data is not just about resolving problems faster though. In October last year we introduced Real-time Business Metrics and described how AppDynamics customers can use Real-time Business Metrics to extract and present business metrics directly from within their applications. These business metrics provide business context enabling customers to turn smart data into actionable information. Our customers, for example, can see the exact revenue impact of performance problems, the end user experience during an app upgrade, or the real-time impact of marketing campaigns. Smart analytics not only clearly show the business benefits of making immediate improvements, they help direct where limited resources should be invested for further business and application improvement going forward.

AppDynamics is focused on delivering actionable intelligence to sell problems for IT operations and development teams as well as business owners. To learn more about Real-time Business Metrics read here.

The post A New Breed of Actionable Analytics written by appeared first on Application Performance Monitoring Blog from AppDynamics.

Read the original blog entry...

More Stories By Jyoti Bansal

In high-production environments where release cycles are measured in hours or minutes — not days or weeks — there's little room for mistakes and no room for confusion. Everyone has to understand what's happening, in real time, and have the means to do whatever is necessary to keep applications up and running optimally.

DevOps is a high-stakes world, but done well, it delivers the agility and performance to significantly impact business competitiveness.

@MicroservicesExpo Stories
SYS-CON Events announced today that CollabNet, a global leader in enterprise software development, release automation and DevOps solutions, will be a Bronze Sponsor of SYS-CON's 20th International Cloud Expo®, taking place from June 6-8, 2017, at the Javits Center in New York City, NY. CollabNet offers a broad range of solutions with the mission of helping modern organizations deliver quality software at speed. The company’s latest innovation, the DevOps Lifecycle Manager (DLM), supports Value S...
There are two main reasons for infrastructure automation. First, system administrators, IT professionals and DevOps engineers need to automate as many routine tasks as possible. That’s why we build tools at Stackify to help developers automate processes like application performance management, error monitoring, and log management; automation means you have more time for mission-critical tasks. Second, automation makes the management of complex, diverse environments possible and allows rapid scal...
SYS-CON Events announced today that HTBase will exhibit at SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. HTBase (Gartner 2016 Cool Vendor) delivers a Composable IT infrastructure solution architected for agility and increased efficiency. It turns compute, storage, and fabric into fluid pools of resources that are easily composed and re-composed to meet each application’s needs. With HTBase, companies can quickly prov...
This talk centers around how to automate best practices in a multi-/hybrid-cloud world based on our work with customers like GE, Discovery Communications and Fannie Mae. Today’s enterprises are reaping the benefits of cloud computing, but also discovering many risks and challenges. In the age of DevOps and the decentralization of IT, it’s easy to over-provision resources, forget that instances are running, or unintentionally expose vulnerabilities.
SYS-CON Events announced today that Linux Academy, the foremost online Linux and cloud training platform and community, will exhibit at SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. Linux Academy was founded on the belief that providing high-quality, in-depth training should be available at an affordable price. Industry leaders in quality training, provided services, and student certification passes, its goal is to c...
SYS-CON Events announced today that Fusion, a leading provider of cloud services, will exhibit at SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. Fusion, a leading provider of integrated cloud solutions to small, medium and large businesses, is the industry’s single source for the cloud. Fusion’s advanced, proprietary cloud service platform enables the integration of leading edge solutions in the cloud, including cloud...
SYS-CON Events announced today that Peak 10, Inc., a national IT infrastructure and cloud services provider, will exhibit at SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. Peak 10 provides reliable, tailored data center and network services, cloud and managed services. Its solutions are designed to scale and adapt to customers’ changing business needs, enabling them to lower costs, improve performance and focus intern...
One of the biggest challenges with adopting a DevOps mentality is: new applications are easily adapted to cloud-native, microservice-based, or containerized architectures - they can be built for them - but old applications need complex refactoring. On the other hand, these new technologies can require relearning or adapting new, oftentimes more complex, methodologies and tools to be ready for production. In his general session at @DevOpsSummit at 20th Cloud Expo, Chris Brown, Solutions Marketi...
It has never been a better time to be a developer! Thanks to cloud computing, deploying our applications is much easier than it used to be. How we deploy our apps continues to evolve thanks to cloud hosting, Platform-as-a-Service (PaaS), and now Function-as-a-Service. FaaS is the concept of serverless computing via serverless architectures. Software developers can leverage this to deploy an individual "function", action, or piece of business logic. They are expected to start within milliseconds...
We all know that end users experience the internet primarily with mobile devices. From an app development perspective, we know that successfully responding to the needs of mobile customers depends on rapid DevOps – failing fast, in short, until the right solution evolves in your customers' relationship to your business. Whether you’re decomposing an SOA monolith, or developing a new application cloud natively, it’s not a question of using microservices - not doing so will be a path to eventual ...
@DevOpsSummit at Cloud taking place June 6-8, 2017, at Javits Center, New York City, is co-located with the 20th International Cloud Expo and will feature technical sessions from a rock star conference faculty and the leading industry players in the world. The widespread success of cloud computing is driving the DevOps revolution in enterprise IT. Now as never before, development teams must communicate and collaborate in a dynamic, 24/7/365 environment. There is no time to wait for long developm...
With 10 simultaneous tracks, keynotes, general sessions and targeted breakout classes, Cloud Expo and @ThingsExpo are two of the most important technology events of the year. Since its launch over eight years ago, Cloud Expo and @ThingsExpo have presented a rock star faculty as well as showcased hundreds of sponsors and exhibitors! In this blog post, I provide 7 tips on how, as part of our world-class faculty, you can deliver one of the most popular sessions at our events. But before reading the...
The purpose of this article is draw attention to key SaaS services that are commonly overlooked during contact signing that are essential to ensuring they meet the expectations and requirements of the organization and provide guidance and recommendations for process and controls necessary for achieving quality SaaS contractual agreements.
SYS-CON Events announced today that OpsGenie will exhibit at SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. Founded in 2012, OpsGenie is an alerting and on-call management solution for dev and ops teams. OpsGenie provides the tools needed to design actionable alerts, manage on-call schedules and escalations, and ensure that the right people are notified at the right time, using multiple notification methods.
The first step to solving a problem is recognizing that it actually exists. And whether you've realized it or not, cloud services are a problem for your IT department. Even if you feel like you have a solid grasp of cloud technology and the nuances of making a cloud purchase, business leaders don't share the same confidence. Nearly 80% feel that IT lacks the skills necessary to help with cloud purchases-and they're looking to cloud brokers for help instead. It's time to admit we have a cloud s...
According to a recent Gartner study, by 2020, it will be unlikelythat any enterprise will have a “no cloud” policy, and hybrid will be the most common use of the cloud. While the benefits of leveraging public cloud infrastructures are well understood, the desire to keep critical workloads and data on-premise in the private data center still remains. For enterprises, the hybrid cloud provides a best of both worlds solution. However, the leading factor that determines the preference to the hybrid ...
In this modern world of IT, you've probably got some new colleagues in your life-namely, the cloud and SaaS providers who now hold your infrastructure in their hands. These business relationships-yes, they're technology-based, but cloud and SaaS are business models-will become as important to your IT team and your company as the hardware and software you used to install. Once you've adopted SaaS, or inherited SaaS, it's on you to avoid price hikes, licensing issues and app or provider sprawl....
A completely new computing platform is on the horizon. They’re called Microservers by some, ARM Servers by others, and sometimes even ARM-based Servers. No matter what you call them, Microservers will have a huge impact on the data center and on server computing in general. Although few people are familiar with Microservers today, their impact will be felt very soon. This is a new category of computing platform that is available today and is predicted to have triple-digit growth rates for some ...
In IT, we sometimes coin terms for things before we know exactly what they are and how they’ll be used. The resulting terms may capture a common set of aspirations and goals – as “cloud” did broadly for on-demand, self-service, and flexible computing. But such a term can also lump together diverse and even competing practices, technologies, and priorities to the point where important distinctions are glossed over and lost.
DevOps is speeding towards the IT world like a freight train and the hype around it is deafening. There is no reason to be afraid of this change as it is the natural reaction to the agile movement that revolutionized development just a few years ago. By definition, DevOps is the natural alignment of IT performance to business profitability. The relevance of this has yet to be quantified but it has been suggested that the route to the CEO’s chair will come from the IT leaders that successfully ma...