Welcome!

Microservices Expo Authors: Gerardo A Dada, Roger Strukhoff, XebiaLabs Blog, Elizabeth White, Liz McMillan

Related Topics: Microservices Expo, Containers Expo Blog

Microservices Expo: Article

Monitoring Web Applications Using Hyperic

Monitoring enables us to anticipate the issues

Monitoring applications is an important aspect of IT industry. Huge investments are made in setting and maintaining the IT infrastructure be it cloud or physical. To ensure maximum performance across your business, problems have to be identified and resolved before they affect the users. Monitoring enables us to anticipate the issues and hence resolve them before they start appearing as problems to users. We need to have visibility into IT infrastructure to identify the issue origin. For example, by monitoring the applications proactively, we can figure out the root cause whether its poor performance is due to the problem at resource level or at application level or at any other place and take action before it starts impacting the user. It gives us more control and confidence on the application and hence helps us in meeting the service quality requirements.

Performance issues can arise due to problems at any level - machine/ application/ transaction. We should be able to monitor through all levels so that exact point of origin can be located. Most of the current solutions provide monitoring at any one particular level. For example, some cloud based infrastructure solutions provide monitoring at machine level only. Problem need not be necessarily to be at machine level. There could be problem at the application level like increased thread count, more number of active connections to valuable resources, etc or at transaction level like response time, request rate, etc. Thus, monitoring across different levels gives us visibility to know where the problem lies. Once the problem origin is known, we can take corrective actions to resolve it before it impacts the user experience.

There should be a solution to achieve the same to dig out the actual problem. We are going to deal with it in this article. We are going to see how to monitor different metrics (CPU utilization, thread count per minute, response time) at different levels. We will use Hyperic as our monitoring tool to measure the metrics. It is a systems monitoring, server monitoring and IT management software. It is based on server-agent model. We will have a look at the basic architecture of Hyperic as well.

Use-case scenario - Problem statement

Consider a case where a web application becomes famous and workload increases on it. This causes the application's performance to go down which results in slow loading of web pages. There could be many reasons for the degradation of performance like CPU utilization exceeds certain level, application logic is poor, less number of resources in infrastructure which are not able to sustain the load and many more. Issue could be at any level as discussed. Availability and performance marks end users' experience. This necessitates the monitoring of applications at different levels so that any unexpected behaviour could be avoided by taking corrective measures on time.

Deep dive into the solution
One of the most common complaints heard from end user is ‘website responds very slowly'. Poor performance of application can drift the user away from web application and leaves a bad impact on them. To find out the reason for poor performance of application, we have to measure the metrics at different levels to dig out the actual issue. We are programmatically going to monitor CPU utilization at machine level, used thread count per minute at application level and response time at transaction level.

Monitoring requires a reliable tool which can track the metrics periodically and update the application administrators.  There are lots of monitoring tools present in the market with varying features. Some of the most commonly used are Hyperic, Nagios, Cacti, Ganglia, etc.  Most of the tools in the market represent the metric values in visual form which helps the administrators to monitor and analyze any unexpected behaviour of application.

Hyperic is one of the leading monitoring tools in this arena. Hyperic comes in two flavors - open source (under GPL licensing) and enterprise edition (commercial license). It is based on server-agent model.  Following is a short description of Hyperic key components:

  1. HQ Agent: HQ agent is responsible for collecting data of the machine in which it is installed and send it to server. Each machine which is to be monitored should have an HQ agent running.
  2. HQ Server: HQ server consolidates the data sent by agents installed on different machines and persist it in the Hyperic database. HQ server maintains the inventory which keeps the information of all the resources and their monitored values.
  3. HQ Portal: HQ portal is a graphical user interface which gives complete information about the resources being monitored. User can add/ remove/perform control actions/set alarms from this portal. It is highly customizable with provision of choosing the metric to be collected and changing the collection intervals.
  4. HQ Web Service API: It has java based API called HQAPI. It helps in accessing the resources and their metrics programmatically.
  5. HQ Plugins: Hyperic has its own set of resource plugins for collecting metrics and other operations. It is also possible to build a new plugin to support additional functionality.

A large number of metrics is managed by Hyperic. In case of virtual environment, Hyperic agents are installed on virtual machines. We will use HQAPI to access the resources.

Consider a web application deployed on Apache Tomcat as application server. We will deploy famous - Jpetstore web portal on application server and then monitor the same.

Deploy web application on application server
Copy the jpetstore application under ${Apache_Tomcat_Installation_dir}/webapps and start tomcat. Try accessing - http://localhost:8080/jpetstore link and if it shows jpetstore homepage that means application is deployed successfully.

The way resources are modelled in hierarchical fashion in Hyperic UI console, same way resources could be accessed programmatically using HQ Java API. Top most resource is Platform and beneath it are present services running which are allowed to be monitored in that machine. Let's see how to monitor different metrics at different levels using HQAPI.

More Stories By Akansha Jain

Akansha works as a Technology Analyst at SETLabs, R&D division, at Infosys Technologies Ltd. She has close to 4 years of experience in development of Cloud computing, Java and Java EE applications, Eclipse Plugin Architecture, Software Factory, Web 2.0,etc. Earlier publication works are available on devx.com.

@MicroservicesExpo Stories
Today’s IT environments are increasingly heterogeneous, with Linux, Java, Oracle and MySQL considered nearly as common as traditional Windows environments. In many cases, these platforms have been integrated into an organization’s Windows-based IT department by way of an acquisition of a company that leverages one of those platforms. In other cases, the applications may have been part of the IT department for years, but managed by a separate department or singular administrator. Still, whether...
As we enter the final week before the 19th International Cloud Expo | @ThingsExpo in Santa Clara, CA, it's time for me to reflect on six big topics that will be important during the show. Hybrid Cloud: This general-purpose term seems to provide a comfort zone for many enterprise IT managers. It sounds reassuring to be able to work with one of the major public-cloud providers like AWS or Microsoft Azure while still maintaining an on-site presence.
I’m a huge fan of open source DevOps tools. I’m also a huge fan of scaling open source tools for the enterprise. But having talked with my fair share of companies over the years, one important thing I’ve learned is that you can’t scale your release process using open source tools alone. They simply require too much scripting and maintenance when used that way. Scripting may be fine for smaller organizations, but it’s not ok in an enterprise environment that includes many independent teams and to...
Get deep visibility into the performance of your databases and expert advice for performance optimization and tuning. You can't get application performance without database performance. Give everyone on the team a comprehensive view of how every aspect of the system affects performance across SQL database operations, host server and OS, virtualization resources and storage I/O. Quickly find bottlenecks and troubleshoot complex problems.
Between 2005 and 2020, data volumes will grow by a factor of 300 – enough data to stack CDs from the earth to the moon 162 times. This has come to be known as the ‘big data’ phenomenon. Unfortunately, traditional approaches to handling, storing and analyzing data aren’t adequate at this scale: they’re too costly, slow and physically cumbersome to keep up. Fortunately, in response a new breed of technology has emerged that is cheaper, faster and more scalable. Yet, in meeting these new needs they...
In his general session at 19th Cloud Expo, Manish Dixit, VP of Product and Engineering at Dice, discussed how Dice leverages data insights and tools to help both tech professionals and recruiters better understand how skills relate to each other and which skills are in high demand using interactive visualizations and salary indicator tools to maximize earning potential. Manish Dixit is VP of Product and Engineering at Dice. As the leader of the Product, Engineering and Data Sciences team at D...
In IT, we sometimes coin terms for things before we know exactly what they are and how they’ll be used. The resulting terms may capture a common set of aspirations and goals – as “cloud” did broadly for on-demand, self-service, and flexible computing. But such a term can also lump together diverse and even competing practices, technologies, and priorities to the point where important distinctions are glossed over and lost.
Monitoring of Docker environments is challenging. Why? Because each container typically runs a single process, has its own environment, utilizes virtual networks, or has various methods of managing storage. Traditional monitoring solutions take metrics from each server and applications they run. These servers and applications running on them are typically very static, with very long uptimes. Docker deployments are different: a set of containers may run many applications, all sharing the resource...
The 20th International Cloud Expo has announced that its Call for Papers is open. Cloud Expo, to be held June 6-8, 2017, at the Javits Center in New York City, brings together Cloud Computing, Big Data, Internet of Things, DevOps, Containers, Microservices and WebRTC to one location. With cloud computing driving a higher percentage of enterprise IT budgets every year, it becomes increasingly important to plant your flag in this fast-expanding business opportunity. Submit your speaking proposal ...
You have great SaaS business app ideas. You want to turn your idea quickly into a functional and engaging proof of concept. You need to be able to modify it to meet customers' needs, and you need to deliver a complete and secure SaaS application. How could you achieve all the above and yet avoid unforeseen IT requirements that add unnecessary cost and complexity? You also want your app to be responsive in any device at any time. In his session at 19th Cloud Expo, Mark Allen, General Manager of...
In his session at 19th Cloud Expo, Claude Remillard, Principal Program Manager in Developer Division at Microsoft, contrasted how his team used config as code and immutable patterns for continuous delivery of microservices and apps to the cloud. He showed how the immutable patterns helps developers do away with most of the complexity of config as code-enabling scenarios such as rollback, zero downtime upgrades with far greater simplicity. He also demoed building immutable pipelines in the cloud ...
"Dice has been around for the last 20 years. We have been helping tech professionals find new jobs and career opportunities," explained Manish Dixit, VP of Product and Engineering at Dice, in this SYS-CON.tv interview at 19th Cloud Expo, held November 1-3, 2016, at the Santa Clara Convention Center in Santa Clara, CA.
@DevOpsSummit taking place June 6-8, 2017 at Javits Center, New York City, is co-located with the 20th International Cloud Expo and will feature technical sessions from a rock star conference faculty and the leading industry players in the world. @DevOpsSummit at Cloud Expo New York Call for Papers is now open.
Rapid innovation, changing business landscapes, and new IT demands force businesses to make changes quickly. In the eyes of many, containers are at the brink of becoming a pervasive technology in enterprise IT to accelerate application delivery. In this presentation, attendees learned about the: The transformation of IT to a DevOps, microservices, and container-based architecture What are containers and how DevOps practices can operate in a container-based environment A demonstration of how ...
Cloud Expo, Inc. has announced today that Andi Mann returns to 'DevOps at Cloud Expo 2017' as Conference Chair The @DevOpsSummit at Cloud Expo will take place on June 6-8, 2017, at the Javits Center in New York City, NY. "DevOps is set to be one of the most profound disruptions to hit IT in decades," said Andi Mann. "It is a natural extension of cloud computing, and I have seen both firsthand and in independent research the fantastic results DevOps delivers. So I am excited to help the great t...
Without lifecycle traceability and visibility across the tool chain, stakeholders from Planning-to-Ops have limited insight and answers to who, what, when, why and how across the DevOps lifecycle. This impacts the ability to deliver high quality software at the needed velocity to drive positive business outcomes. In his general session at @DevOpsSummit at 19th Cloud Expo, Phil Hombledal, Solution Architect at CollabNet, discussed how customers are able to achieve a level of transparency that e...
Application transformation and DevOps practices are two sides of the same coin. Enterprises that want to capture value faster, need to deliver value faster – time value of money principle. To do that enterprises need to build cloud-native apps as microservices by empowering teams to build, ship, and run in production. In his session at @DevOpsSummit at 19th Cloud Expo, Neil Gehani, senior product manager at HPE, discussed what every business should plan for how to structure their teams to delive...
DevOps is being widely accepted (if not fully adopted) as essential in enterprise IT. But as Enterprise DevOps gains maturity, expands scope, and increases velocity, the need for data-driven decisions across teams becomes more acute. DevOps teams in any modern business must wrangle the ‘digital exhaust’ from the delivery toolchain, "pervasive" and "cognitive" computing, APIs and services, mobile devices and applications, the Internet of Things, and now even blockchain. In this power panel at @...
SYS-CON Events announced today that Dataloop.IO, an innovator in cloud IT-monitoring whose products help organizations save time and money, has been named “Bronze Sponsor” of SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. Dataloop.IO is an emerging software company on the cutting edge of major IT-infrastructure trends including cloud computing and microservices. The company, founded in the UK but now based in San Fran...
@DevOpsSummit at Cloud taking place June 6-8, 2017, at Javits Center, New York City, is co-located with the 20th International Cloud Expo and will feature technical sessions from a rock star conference faculty and the leading industry players in the world. The widespread success of cloud computing is driving the DevOps revolution in enterprise IT. Now as never before, development teams must communicate and collaborate in a dynamic, 24/7/365 environment. There is no time to wait for long developm...