Microservices Expo Authors: Hollis Tibbetts, Liz McMillan, Elizabeth White, Yeshim Deniz, Ian Khan

Related Topics: @DevOpsSummit, Java IoT, Microservices Expo, Linux Containers, @CloudExpo, SDN Journal

@DevOpsSummit: Blog Feed Post

Measuring Your DevOps Success

When you start delivering your DevOps initiatives, the measurement starts immediately and is constant

A survey from the Vanson Bourne market research agency (with CA) late in 2013 indicated that 39% of those surveyed had adopted some form of DevOps and 27% were planning to do so in the near future. Despite this being such a hot topic in the IT sector, and with a high level of take-up, the question we are still most commonly asked is: “Where do we start?”

Our answer is always that an organization’s current position must be baselined first. Having a baseline means you can build a business case, apply targets and goals to your projects and measure your success as you progress through your project with the ultimate goal being to report back to the board on how you used the money to save or make more money – and improved your teams’ satisfaction.

DevOps Metrics for Baselining and Measuring Success
There are hard, quantifiable technical and financial metrics we can track, such as:

  • Number and frequency of software releases
  • Volume of defects
  • Time/cost per release
  • MTTR*
  • Number and frequency of outages / performance issues
  • Revenue/profit impact of outages / performance issues
  • Number and cost of resources

It’s worth noting that one of the biggest inhibitors to success of DevOps and related tooling projects is people’s perceptions that they are at risk of losing their jobs as their work becomes automated. Often, particularly in areas like release and deployment management, we find that there are specific individuals who hold all the knowledge around a current process (they wrote all the scripts for example) and who are viewed as heroes when they are the only person who can fix an issue and often do it out of office hours and at short notice – but are, in fact, bottlenecks. These individuals are often highly talented, but feel secure in the indispensable role they have created for themselves. Though they will often be happier when freed up to do more creative and rewarding work, they are often fearful and this needs to be addressed.

Cultural Metrics
Although cultural metrics are difficult to apply hard dollar value to, DevOps is about resolving conflict in the workplace, eliminating stress and avoiding burnout – and they are measurable. Happy people are more productive – their health is better, they have more ideas, work more effectively and will put in the extra mile. You can measure across a number of key cultural indicators around feelings about change, failure, going to work, what a typical day’s work entails, in addition to a number of cultural attributes such as:

  • Cross-skilling, knowledge sharing and pairing between teams
  • Working in a fluid but focussed manner
  • Working in multidisciplinary teams
  • Organizing teams around projects rather than skill-sets
  • Constantly dancing on the edge of failure (in a good way)
  • Position around business demand
  • Extraneous lines of code
  • Attitude to continuous improvement
  • Obsession with metrics
  • Technological experimentation
  • Team autonomy

You can also look at a number of team features such as:

  • Rewards and feelings of success
  • Hierarchical and political obstacles and annoyances
  • Inspiring and fostering creativity

Process Metrics
DevOps is not a process or a tool – but there are a number of processes in the software development lifecycle (SDLC) that affect both traditional development and operations staff to greater or lesser degrees that need to be taken into consideration. All of these process components can be optimized, and all of them can then be improved upon further using appropriate software tooling. An ultimate goal of a typical DevOps project is often to attain true continuous delivery (CD) by linking these processes and tools together to allow fully tested, production-ready, committed code to proceed to live without impediment – we often refer to the software infrastructure piece of this as the DevOps toolchain. When baselining current state, it’s useful to measure these component processes and their relative maturity (taking into account use of existing tools and success of implementation). Typically, we look at:

  • Requirements elicitation and management
  • Agile development
  • Build
  • Release and deployment
  • Unit testing
  • User Acceptance Testing
  • Quality Assurance
  • Application Performance Monitoring
  • Cloud

How to Influence Metrics
Once you’ve baselined your current position, it’s time to think about your desired future state. Your baselining exercise will probably have highlighted where the key bottlenecks are and areas on which to concentrate. Although we preach, “People, then Process, then Tools”, there are tools, in particular Application Performance Management that can help discover bottlenecks and issues upfront – although it’s imperative you have the right people then using and acting on this data and put the right processes in behind in terms of defect tracking, corrective development, versioning build and deployment.

While you might be stumped on ideas for influencing cultural, especially in well-established enterprises, it is by no means impossible. The key is understanding the current culture. For instance, does a culture of “blame” exist? What happens when there is a production outage? How motivated and rewarded do staff feel? Once you’ve identified the prevailing culture you can establish a program of cultural initiatives to move the culture to one that is more productive and collaborative.

DevOps Tools for Change
There are a number of tools that can influence the harder and software metrics – for example:

Application Performance Monitoring (APM)

  • Reduces MTTR
  • Makes it easier to create a collaborative approach in teams dealing with issues
  • Identifies root cause fast; eliminating blame games

Application Release Automation (ARA)

  • Enables development to seamlessly transition code to operations who can quickly and consistently deploy into production
  • Enables instant rollback or redeploy when an error is identified in production
  • Reduces fear of failure as rollback/redeploy is so easy

Integration Testing & Virtualization

  • Mimics the production environment so successful test are guaranteed to run
  • Allows testers to ‘shift left’ in test process and collaborate with developers early
  • Fast testing enables fast, confident throughput of change

When to Measure, How to Tweak
When you start delivering your DevOps initiatives, the measurement starts immediately and is constant. You will be looking for upward trends, as well as downward trends. Make sure you share reports regularly with the team – try weekly with the core team and monthly with the extended team. Highlight success and elicit ideas for improvement where areas have proved more challenging. Try things – tweak, monitor, tweak again. But remember: “Any improvement not made at the constraint is an illusion.”

What to Do with Success
Celebrate success! Create rewards and incentive programmes for teams when metrics targets are achieved. Part of the DevOp’s agenda is about improving working conditions – depressurising and destressing environments and having everyone working together in harmony and eliminating disasters, catastrophes, blame and brinkmanship.

Why ask for an external DevOps Maturity Assessment
While no one’s going to understand your business as well as yourselves, we often meet organizations who are struggling to find the time – they know there are improvements to be made but they are so busy with firefighting they can’t conceive of stopping and taking stock of their current position. Also, as human beings are emotional creatures — it’s natural for all working environments to have some level of politics or hierarchies going on — it’s often helpful to have an outsider take a pragmatic, neutral view of a situation. So if you are ready to baseline your current DevOps state and identify the DevOps initiatives that will have real, positive impact on your business, but feel you do not have the time to figure it all out yourselves – please do get in touch with us @ranger4ltd.

* The MTTR is the Mean Time to Repair, Resolve or Resolution – each of the definitions mean the same thing and can be used interchangeably. This term is more commonly used when talking about Application Performance Management and the speed at which an outage or performance issue can be fixed, but equally can be used when talking about testing and eliminating defects


Take five minutes to get complete visibility into the performance of your production applications with AppDynamics today.

This is guest post by our partners at Ranger 4. Find out more about them on by following @ranger4ltd on twitter.

The post Measuring Your DevOps Success written by Helen Beal appeared first on Application Performance Monitoring Blog from AppDynamics.

Read the original blog entry...

More Stories By AppDynamics Blog

In high-production environments where release cycles are measured in hours or minutes — not days or weeks — there's little room for mistakes and no room for confusion. Everyone has to understand what's happening, in real time, and have the means to do whatever is necessary to keep applications up and running optimally.

DevOps is a high-stakes world, but done well, it delivers the agility and performance to significantly impact business competitiveness.

@MicroservicesExpo Stories
A completely new computing platform is on the horizon. They’re called Microservers by some, ARM Servers by others, and sometimes even ARM-based Servers. No matter what you call them, Microservers will have a huge impact on the data center and on server computing in general. Although few people are familiar with Microservers today, their impact will be felt very soon. This is a new category of computing platform that is available today and is predicted to have triple-digit growth rates for some ...
Without lifecycle traceability and visibility across the tool chain, stakeholders from Planning-to-Ops have limited insight and answers to who, what, when, why and how across the DevOps lifecycle. This impacts the ability to deliver high quality software at the needed velocity to drive positive business outcomes. In his general session at @DevOpsSummit at 19th Cloud Expo, Eric Robertson, General Manager at CollabNet, will discuss how customers are able to achieve a level of transparency that e...
SYS-CON Events announced today that Enzu will exhibit at the 19th International Cloud Expo, which will take place on November 1–3, 2016, at the Santa Clara Convention Center in Santa Clara, CA. Enzu’s mission is to be the leading provider of enterprise cloud solutions worldwide. Enzu enables online businesses to use its IT infrastructure to their competitive advantage. By offering a suite of proven hosting and management services, Enzu wants companies to focus on the core of their online busine...
@DevOpsSummit has been named the ‘Top DevOps Influencer' by iTrend. iTrend processes millions of conversations, tweets, interactions, news articles, press releases, blog posts - and extract meaning form them and analyzes mobile and desktop software platforms used to communicate, various metadata (such as geo location), and automation tools. In overall placement, @DevOpsSummit ranked as the number one ‘DevOps Influencer' followed by @CloudExpo at third, and @MicroservicesE at 24th.
24Notion is full-service global creative digital marketing, technology and lifestyle agency that combines strategic ideas with customized tactical execution. With a broad understand of the art of traditional marketing, new media, communications and social influence, 24Notion uniquely understands how to connect your brand strategy with the right consumer. 24Notion ranked #12 on Corporate Social Responsibility - Book of List.
In his keynote at 19th Cloud Expo, Sheng Liang, co-founder and CEO of Rancher Labs, will discuss the technological advances and new business opportunities created by the rapid adoption of containers. With the success of Amazon Web Services (AWS) and various open source technologies used to build private clouds, cloud computing has become an essential component of IT strategy. However, users continue to face challenges in implementing clouds, as older technologies evolve and newer ones like Docke...
The reason I believe digital transformation is not only more than a fad, but is actually a life-or-death imperative for every business and IT executive on the planet is simple: there will be no place for an “industrial enterprise” in a digital world. Transformation, by definition, is a metamorphosis from one state to another, wholly new state. As such, a true digital transformation must be the act of transforming an industrial-era organization into something wholly different – the Digital Enter...
Just over a week ago I received a long and loud sustained applause for a presentation I delivered at this year’s Cloud Expo in Santa Clara. I was extremely pleased with the turnout and had some very good conversations with many of the attendees. Over the next few days I had many more meaningful conversations and was not only happy with the results but also learned a few new things. Here is everything I learned in those three days distilled into three short points.
In his session at 19th Cloud Expo, Claude Remillard, Principal Program Manager in Developer Division at Microsoft, will contrast how his team used config as code and immutable patterns for continuous delivery of microservices and apps to the cloud. He will show the immutable patterns helps developers do away with most of the complexity of config as code-enabling scenarios such as rollback, zero downtime upgrades with far greater simplicity. He will also have live demos of building immutable pipe...
Application transformation and DevOps practices are two sides of the same coin. Enterprises that want to capture value faster, need to deliver value faster – time value of money principle. To do that enterprises need to build cloud-native apps as microservices by empowering teams to build, ship, and run in production. In his session at @DevOpsSummit at 19th Cloud Expo, Neil Gehani, senior product manager at HPE, will discuss what every business should plan for how to structure their teams to d...
When we talk about the impact of BYOD and BYOA and the Internet of Things, we often focus on the impact on data center architectures. That's because there will be an increasing need for authentication, for access control, for security, for application delivery as the number of potential endpoints (clients, devices, things) increases. That means scale in the data center. What we gloss over, what we skip, is that before any of these "things" ever makes a request to access an application it had to...
SYS-CON Events announced today that Transparent Cloud Computing (T-Cloud) Consortium will exhibit at the 19th International Cloud Expo®, which will take place on November 1–3, 2016, at the Santa Clara Convention Center in Santa Clara, CA. The Transparent Cloud Computing Consortium (T-Cloud Consortium) will conduct research activities into changes in the computing model as a result of collaboration between "device" and "cloud" and the creation of new value and markets through organic data proces...
The evolution of JavaScript and HTML 5 to support a genuine component based framework (Web Components) with the necessary tools to deliver something close to a native experience including genuine realtime networking (UDP using WebRTC). HTML5 is evolving to offer built in templating support, the ability to watch objects (which will speed up Angular) and Web Components (which offer Angular Directives). The native level support will offer a massive performance boost to frameworks having to fake all...
In many organizations governance is still practiced by phase or stage gate peer review, and Agile projects are forced to accommodate, which leads to WaterScrumFall or worse. But governance criteria and policies are often very weak anyway, out of date or non-existent. Consequently governance is frequently a matter of opinion and experience, highly dependent upon the experience of individual reviewers. As we all know, a basic principle of Agile methods is delegation of responsibility, and ideally ...
Today every business relies on software to drive the innovation necessary for a competitive edge in the Application Economy. This is why collaboration between development and operations, or DevOps, has become IT’s number one priority. Whether you are in Dev or Ops, understanding how to implement a DevOps strategy can deliver faster development cycles, improved software quality, reduced deployment times and overall better experiences for your customers.
Apache Hadoop is a key technology for gaining business insights from your Big Data, but the penetration into enterprises is shockingly low. In fact, Apache Hadoop and Big Data proponents recognize that this technology has not yet achieved its game-changing business potential. In his session at 19th Cloud Expo, John Mertic, director of program management for ODPi at The Linux Foundation, will explain why this is, how we can work together as an open data community to increase adoption, and the i...
JetBlue Airways uses virtual environments to reduce software development costs, centralize performance testing, and create a climate for continuous integration and real-time monitoring of mobile applications. The next BriefingsDirect Voice of the Customer performance engineering case study discussion examines how JetBlue Airways in New York uses virtual environments to reduce software development costs, centralize performance testing, and create a climate for continuous integration and real-tim...
All clouds are not equal. To succeed in a DevOps context, organizations should plan to develop/deploy apps across a choice of on-premise and public clouds simultaneously depending on the business needs. This is where the concept of the Lean Cloud comes in - resting on the idea that you often need to relocate your app modules over their life cycles for both innovation and operational efficiency in the cloud. In his session at @DevOpsSummit at19th Cloud Expo, Valentin (Val) Bercovici, CTO of So...
Virgil consists of an open-source encryption library, which implements Cryptographic Message Syntax (CMS) and Elliptic Curve Integrated Encryption Scheme (ECIES) (including RSA schema), a Key Management API, and a cloud-based Key Management Service (Virgil Keys). The Virgil Keys Service consists of a public key service and a private key escrow service. 

SYS-CON Events announced today that eCube Systems, the leading provider of modern development tools and best practices for Continuous Integration on OpenVMS, will exhibit at SYS-CON's @DevOpsSummit at Cloud Expo New York, which will take place on June 7-9, 2016, at the Javits Center in New York City, NY. eCube Systems offers a family of middleware products and development tools that maximize return on technology investment by leveraging existing technical equity to meet evolving business needs. ...