Welcome!

Microservices Expo Authors: Pat Romanski, Liz McMillan, Elizabeth White, Mamoon Yunus, Jason Bloomberg

Related Topics: @DevOpsSummit, Java IoT, Microservices Expo, Linux Containers, @CloudExpo, SDN Journal

@DevOpsSummit: Blog Feed Post

Measuring Your DevOps Success

When you start delivering your DevOps initiatives, the measurement starts immediately and is constant

A survey from the Vanson Bourne market research agency (with CA) late in 2013 indicated that 39% of those surveyed had adopted some form of DevOps and 27% were planning to do so in the near future. Despite this being such a hot topic in the IT sector, and with a high level of take-up, the question we are still most commonly asked is: “Where do we start?”

Our answer is always that an organization’s current position must be baselined first. Having a baseline means you can build a business case, apply targets and goals to your projects and measure your success as you progress through your project with the ultimate goal being to report back to the board on how you used the money to save or make more money – and improved your teams’ satisfaction.

DevOps Metrics for Baselining and Measuring Success
There are hard, quantifiable technical and financial metrics we can track, such as:

  • Number and frequency of software releases
  • Volume of defects
  • Time/cost per release
  • MTTR*
  • Number and frequency of outages / performance issues
  • Revenue/profit impact of outages / performance issues
  • Number and cost of resources

It’s worth noting that one of the biggest inhibitors to success of DevOps and related tooling projects is people’s perceptions that they are at risk of losing their jobs as their work becomes automated. Often, particularly in areas like release and deployment management, we find that there are specific individuals who hold all the knowledge around a current process (they wrote all the scripts for example) and who are viewed as heroes when they are the only person who can fix an issue and often do it out of office hours and at short notice – but are, in fact, bottlenecks. These individuals are often highly talented, but feel secure in the indispensable role they have created for themselves. Though they will often be happier when freed up to do more creative and rewarding work, they are often fearful and this needs to be addressed.

Cultural Metrics
Although cultural metrics are difficult to apply hard dollar value to, DevOps is about resolving conflict in the workplace, eliminating stress and avoiding burnout – and they are measurable. Happy people are more productive – their health is better, they have more ideas, work more effectively and will put in the extra mile. You can measure across a number of key cultural indicators around feelings about change, failure, going to work, what a typical day’s work entails, in addition to a number of cultural attributes such as:

  • Cross-skilling, knowledge sharing and pairing between teams
  • Working in a fluid but focussed manner
  • Working in multidisciplinary teams
  • Organizing teams around projects rather than skill-sets
  • Constantly dancing on the edge of failure (in a good way)
  • Position around business demand
  • Extraneous lines of code
  • Attitude to continuous improvement
  • Obsession with metrics
  • Technological experimentation
  • Team autonomy

You can also look at a number of team features such as:

  • Rewards and feelings of success
  • Hierarchical and political obstacles and annoyances
  • Inspiring and fostering creativity

Process Metrics
DevOps is not a process or a tool – but there are a number of processes in the software development lifecycle (SDLC) that affect both traditional development and operations staff to greater or lesser degrees that need to be taken into consideration. All of these process components can be optimized, and all of them can then be improved upon further using appropriate software tooling. An ultimate goal of a typical DevOps project is often to attain true continuous delivery (CD) by linking these processes and tools together to allow fully tested, production-ready, committed code to proceed to live without impediment – we often refer to the software infrastructure piece of this as the DevOps toolchain. When baselining current state, it’s useful to measure these component processes and their relative maturity (taking into account use of existing tools and success of implementation). Typically, we look at:

  • Requirements elicitation and management
  • Agile development
  • Build
  • Release and deployment
  • Unit testing
  • User Acceptance Testing
  • Quality Assurance
  • Application Performance Monitoring
  • Cloud

How to Influence Metrics
Once you’ve baselined your current position, it’s time to think about your desired future state. Your baselining exercise will probably have highlighted where the key bottlenecks are and areas on which to concentrate. Although we preach, “People, then Process, then Tools”, there are tools, in particular Application Performance Management that can help discover bottlenecks and issues upfront – although it’s imperative you have the right people then using and acting on this data and put the right processes in behind in terms of defect tracking, corrective development, versioning build and deployment.

While you might be stumped on ideas for influencing cultural, especially in well-established enterprises, it is by no means impossible. The key is understanding the current culture. For instance, does a culture of “blame” exist? What happens when there is a production outage? How motivated and rewarded do staff feel? Once you’ve identified the prevailing culture you can establish a program of cultural initiatives to move the culture to one that is more productive and collaborative.

DevOps Tools for Change
There are a number of tools that can influence the harder and software metrics – for example:

Application Performance Monitoring (APM)

  • Reduces MTTR
  • Makes it easier to create a collaborative approach in teams dealing with issues
  • Identifies root cause fast; eliminating blame games

Application Release Automation (ARA)

  • Enables development to seamlessly transition code to operations who can quickly and consistently deploy into production
  • Enables instant rollback or redeploy when an error is identified in production
  • Reduces fear of failure as rollback/redeploy is so easy

Integration Testing & Virtualization

  • Mimics the production environment so successful test are guaranteed to run
  • Allows testers to ‘shift left’ in test process and collaborate with developers early
  • Fast testing enables fast, confident throughput of change

When to Measure, How to Tweak
When you start delivering your DevOps initiatives, the measurement starts immediately and is constant. You will be looking for upward trends, as well as downward trends. Make sure you share reports regularly with the team – try weekly with the core team and monthly with the extended team. Highlight success and elicit ideas for improvement where areas have proved more challenging. Try things – tweak, monitor, tweak again. But remember: “Any improvement not made at the constraint is an illusion.”

What to Do with Success
Celebrate success! Create rewards and incentive programmes for teams when metrics targets are achieved. Part of the DevOp’s agenda is about improving working conditions – depressurising and destressing environments and having everyone working together in harmony and eliminating disasters, catastrophes, blame and brinkmanship.

Why ask for an external DevOps Maturity Assessment
While no one’s going to understand your business as well as yourselves, we often meet organizations who are struggling to find the time – they know there are improvements to be made but they are so busy with firefighting they can’t conceive of stopping and taking stock of their current position. Also, as human beings are emotional creatures — it’s natural for all working environments to have some level of politics or hierarchies going on — it’s often helpful to have an outsider take a pragmatic, neutral view of a situation. So if you are ready to baseline your current DevOps state and identify the DevOps initiatives that will have real, positive impact on your business, but feel you do not have the time to figure it all out yourselves – please do get in touch with us @ranger4ltd.

* The MTTR is the Mean Time to Repair, Resolve or Resolution – each of the definitions mean the same thing and can be used interchangeably. This term is more commonly used when talking about Application Performance Management and the speed at which an outage or performance issue can be fixed, but equally can be used when talking about testing and eliminating defects

 

Take five minutes to get complete visibility into the performance of your production applications with AppDynamics today.

This is guest post by our partners at Ranger 4. Find out more about them on by following @ranger4ltd on twitter.

The post Measuring Your DevOps Success written by Helen Beal appeared first on Application Performance Monitoring Blog from AppDynamics.

Read the original blog entry...

More Stories By Jyoti Bansal

In high-production environments where release cycles are measured in hours or minutes — not days or weeks — there's little room for mistakes and no room for confusion. Everyone has to understand what's happening, in real time, and have the means to do whatever is necessary to keep applications up and running optimally.

DevOps is a high-stakes world, but done well, it delivers the agility and performance to significantly impact business competitiveness.

@MicroservicesExpo Stories
Did you know that you can develop for mainframes in Java? Or that the testing and deployment can be automated across mobile to mainframe? In his session and demo at @DevOpsSummit at 21st Cloud Expo, Dana Boudreau, a Senior Director at CA Technologies, will discuss how increasingly teams are developing with agile methodologies, using modern development environments, and automating testing and deployments, mobile to mainframe.
As DevOps methodologies expand their reach across the enterprise, organizations face the daunting challenge of adapting related cloud strategies to ensure optimal alignment, from managing complexity to ensuring proper governance. How can culture, automation, legacy apps and even budget be reexamined to enable this ongoing shift within the modern software factory?
While some vendors scramble to create and sell you a fancy solution for monitoring your spanking new Amazon Lambdas, hear how you can do it on the cheap using just built-in Java APIs yourself. By exploiting a little-known fact that Lambdas aren’t exactly single-threaded, you can effectively identify hot spots in your serverless code. In his session at @DevOpsSummit at 21st Cloud Expo, Dave Martin, Product owner at CA Technologies, will give a live demonstration and code walkthrough, showing how ...
@DevOpsSummit at Cloud Expo taking place Oct 31 - Nov 2, 2017, at the Santa Clara Convention Center, Santa Clara, CA, is co-located with the 21st International Cloud Expo and will feature technical sessions from a rock star conference faculty and the leading industry players in the world. The widespread success of cloud computing is driving the DevOps revolution in enterprise IT. Now as never before, development teams must communicate and collaborate in a dynamic, 24/7/365 environment. There is ...
API Security is complex! Vendors like Forum Systems, IBM, CA and Axway have invested almost 2 decades of engineering effort and significant capital in building API Security stacks to lockdown APIs. The API Security stack diagram shown below is a building block for rapidly locking down APIs. The four fundamental pillars of API Security - SSL, Identity, Content Validation and deployment architecture - are discussed in detail below.
We define Hybrid IT as a management approach in which organizations create a workload-centric and value-driven integrated technology stack that may include legacy infrastructure, web-scale architectures, private cloud implementations along with public cloud platforms ranging from Infrastructure-as-a-Service to Software-as-a-Service.
There are several reasons why businesses migrate their operations to the cloud. Scalability and price are among the most important factors determining this transition. Unlike legacy systems, cloud based businesses can scale on demand. The database and applications in the cloud are not rendered simply from one server located in your headquarters, but is instead distributed across several servers across the world. Such CDNs also bring about greater control in times of uncertainty. A database hack ...
In his session at 20th Cloud Expo, Scott Davis, CTO of Embotics, discussed how automation can provide the dynamic management required to cost-effectively deliver microservices and container solutions at scale. He also discussed how flexible automation is the key to effectively bridging and seamlessly coordinating both IT and developer needs for component orchestration across disparate clouds – an increasingly important requirement at today’s multi-cloud enterprise.
Docker is on a roll. In the last few years, this container management service has become immensely popular in development, especially given the great fit with agile-based projects and continuous delivery. In this article, I want to take a brief look at how you can use Docker to accelerate and streamline the software development lifecycle (SDLC) process.
In his session at 20th Cloud Expo, Chris Carter, CEO of Approyo, discussed the basic set up and solution for an SAP solution in the cloud and what it means to the viability of your company. Chris Carter is CEO of Approyo. He works with business around the globe, to assist them in their journey to the usage of Big Data in the forms of Hadoop (Cloudera and Hortonwork's) and SAP HANA. At Approyo, we support firms who are looking for knowledge to grow through current business process, where even 1%...
With Cloud Foundry you can easily deploy and use apps utilizing websocket technology, but not everybody realizes that scaling them out is not that trivial. In his session at 21st Cloud Expo, Roman Swoszowski, CTO and VP, Cloud Foundry Services, at Grape Up, will show you an example of how to deal with this issue. He will demonstrate a cloud-native Spring Boot app running in Cloud Foundry and communicating with clients over websocket protocol that can be easily scaled horizontally and coordinate...
IT organizations are moving to the cloud in hopes to approve efficiency, increase agility and save money. Migrating workloads might seem like a simple task, but what many businesses don’t realize is that application migration criteria differs across organizations, making it difficult for architects to arrive at an accurate TCO number. In his session at 21st Cloud Expo, Joe Kinsella, CTO of CloudHealth Technologies, will offer a systematic approach to understanding the TCO of a cloud application...
DevOps at Cloud Expo, taking place October 31 - November 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA, is co-located with 21st Cloud Expo and will feature technical sessions from a rock star conference faculty and the leading industry players in the world. The widespread success of cloud computing is driving the DevOps revolution in enterprise IT. Now as never before, development teams must communicate and collaborate in a dynamic, 24/7/365 environment. There is no time to w...
API Security has finally entered our security zeitgeist. OWASP Top 10 2017 - RC1 recognized API Security as a first class citizen by adding it as number 10, or A-10 on its list of web application vulnerabilities. We believe this is just the start. The attack surface area offered by API is orders or magnitude larger than any other attack surface area. Consider the fact the APIs expose cloud services, internal databases, application and even legacy mainframes over the internet. What could go wrong...
The goal of Continuous Testing is to shift testing left to find defects earlier and release software faster. This can be achieved by integrating a set of open source functional and performance testing tools in the early stages of your software delivery lifecycle. There is one process that binds all application delivery stages together into one well-orchestrated machine: Continuous Testing. Continuous Testing is the conveyer belt between the Software Factory and production stages. Artifacts are m...
Cloud adoption is often driven by a desire to increase efficiency, boost agility and save money. All too often, however, the reality involves unpredictable cost spikes and lack of oversight due to resource limitations. In his session at 20th Cloud Expo, Joe Kinsella, CTO and Founder of CloudHealth Technologies, tackled the question: “How do you build a fully optimized cloud?” He will examine: Why TCO is critical to achieving cloud success – and why attendees should be thinking holistically ab...
Web services have taken the development world by storm, especially in recent years as they've become more and more widely adopted. There are naturally many reasons for this, but first, let's understand what exactly a web service is. The World Wide Web Consortium (W3C) defines "web of services" as "message-based design frequently found on the Web and in enterprise software". Basically, a web service is a method of sending a message between two devices through a network. In practical terms, this ...
In his session at @DevOpsSummit at 20th Cloud Expo, Kelly Looney, director of DevOps consulting for Skytap, showed how an incremental approach to introducing containers into complex, distributed applications results in modernization with less risk and more reward. He also shared the story of how Skytap used Docker to get out of the business of managing infrastructure, and into the business of delivering innovation and business value. Attendees learned how up-front planning allows for a clean sep...
In IT, we sometimes coin terms for things before we know exactly what they are and how they’ll be used. The resulting terms may capture a common set of aspirations and goals – as “cloud” did broadly for on-demand, self-service, and flexible computing. But such a term can also lump together diverse and even competing practices, technologies, and priorities to the point where important distinctions are glossed over and lost.
"At the keynote this morning we spoke about the value proposition of Nutanix, of having a DevOps culture and a mindset, and the business outcomes of achieving agility and scale, which everybody here is trying to accomplish," noted Mark Lavi, DevOps Solution Architect at Nutanix, in this SYS-CON.tv interview at @DevOpsSummit at 20th Cloud Expo, held June 6-8, 2017, at the Javits Center in New York City, NY.