Cloud computing has been a force in the industry for the past few years. Regardless of your feelings for it, one fact remains true ─ you have to monitor it just like any other system in your infrastructure, sometimes more so. The cloud presents a few challenges for traditional monitoring: the base hardware isn't yours; the instance isn't on your network and sometimes the machine can just vanish never to be heard from again.

New Instances Create Old Problems
Let's start with the latter problem. There are many tools out there to help you manage your cloud instances, most of which understand that things go away, recognize that, and boot up new machines on the fly with a command or two. How do we tie that into your monitoring though? Traditional systems require sometimes complex configuration changes and service restarts to recognize those changes. Some companies I talk to boot up and shut down in excess of 100 instances a day. That is a new instance every 4.8 minutes given an eight-hour workday. To have to continually restart my monitoring system to handle that volume is crazy, especially given that some systems, if rebooted every five minutes, might not have time to run all their checks, and you end up not monitoring what you think you are. This is fundamentally flawed and eliminates much of the value that going cloud delivers to an organization that relies on IT performance to ensure business performance.

The system you use to monitor the cloud needs to be just as flexible and configurable as the cloud. You need a system that can handle on-the-fly changes without restarts. On top of that, you don't want to be doing these changes manually; even for a good admin, having to add checks to the system every five or so minutes is going to be stressful. We want a system with an API that can handle all the heavy lifting for us, something where check configuration can be tied into the management system.

While we are talking about flexibility, let's figure out how we are going to get the data from our cloud instance into the system. Cloud instances are tailored to give you just the power you need so you don't pay for extra computing. Adding a heavy agent to that to get data is either going to bring your actual application to a halt or end up costing you extra per instance just to collect data. We want a lightweight system, something that is agentless or has the flexibility to run whatever agent you want, or something of your own design.

Exposing Data, Driving Cross-Functional Knowledge
Most cloud providers and even most applications these days expose their data in some fashion that can be pulled remotely, sans agent. System-level data is often available over SNMP, Java apps have JMX, and a plethora of other applications, open source and otherwise, provide easy-to-use REST APIs to pull data via XML or JSON. With all of this data exposed for you, being able to run agentless and pull your data can have little to no impact on your running system. But sometimes we need an agent, maybe it is to look at something that isn't exposed or exposed in a way that isn't viewable remotely. If that's the case, we want a choice─ can I run agent X or Y, or can I write a quick script to expose that data for me? Either way, you want the flexibility to follow your own path, doing what is right for you (and your organization).

An often overlooked item is that the data you collect from the cloud should be treated just like any other data point you collect. Often it's the case that because many cloud providers give you graphs and data in their own systems dashboard, it's looked at as "good enough." The problem with "good enough" is that while these tools provide an okay insight into that provider's systems, how are you going to correlate that data with your physical infrastructure or other cloud providers? What data does your provider's tool even look at, is it going to be useful for your developers, your analysts or just your system administrators?

Whether you are in the cloud or not, if your monitoring / trending tool doesn't allow you to look at data for any business unit in your organization, it's time to move on from it and use something that does. Data is powerful; the more data you have the more informed decisions are, and the more data you can share among your co-workers, the stronger you become as a unit or business. It provides departments and groups context on how the things they do impact others and the company. Shared knowledge ensures all are pulling in the same direction toward the same goals. It enables IT operations to align with and drive business performance.

The cloud has been a disruptive force for many businesses, and tools that embrace the cloud are leading the way forward. Cloud computing has forced us to think differently about how we do administration. While we open our minds to those new ideas, it's a good time to do a little inventory on your tools, throw away the ones that have fallen behind and take a look at what is out there that can perform the old duties plus more.

We've seen it all over the place. The cloud has evolved, and in doing so, has evolved business as we know it. Whether it's a monitoring tool or enterprise software, simply retrofitting old code with a SaaS front end and calling it cloud doesn't mean it's the right choice to achieve the goals you're working towards. The architecture is integral, and those tools and systems that are purpose-built for today's cloud environment enable your organization to become adaptable - not only to internal shifts, but to external factors. Those organizations that don't adapt, die - fundamental evolutionary theory. Being able to dynamically monitor your systems and operations enables the real-time adaptability that ensures your future as a member of the dominant species in an environment that is changing at the speed of cloud.