Microservices Expo Authors: John Rauser, Liz McMillan, Madhavan Krishnan, VP, Cloud Solutions, Virtusa, Jason Bloomberg, Pat Romanski

Related Topics: Microservices Expo

Microservices Expo: Blog Feed Post

Opscode, Turning Sysadmins into Superheroes

Mass Server Configuration the Most Under-Served Disciplines in Systems Management

Opscode Configuration Management

For the last few years I have had an interest in configuration management of IT infrastructure. While by no means an expert I have a considerable amount of experience with the problems associated with mass server configuration and have come to believe it also one of the most under-served disciplines in systems management.

In a previous life I had an operations role maintaining primarily Linux servers and other open source infrastructure. In 2006 I worked on launching the open source NetDirector project, a graphical tool for configuring open source infrastructure like Apache, Samba, LDAP and NFS servers. During that time the challenges of maintaining server configurations started to really come to light for me.

The Challenges of Server Configuration

A large part of configuring infrastructure is repetitive and time consuming. Many sysadmins rely on their own scripts to help manage the process but it’s still a one-off for each administrator and their individual knowledge seldom if ever gets institutionalized throughout the entire organization.

Having a framework for maintaining configuration data is important. Configuration scripts are often authored as shell scripts or Perl or even Python but are seldom maintained or used beyond the original author. Some technologies use plugins that could be used to abstract configuration variables, users and systems then shared among users of the technology but plugins are often specific to a given technology and non-transferrable.

Despite the breadth of this problem there are relatively few solutions that can easily be consumed by medium-sized enterprises outside of large management suites available from the Big Four (HP, IBM, CA and BMC). The independent software vendors have all been consumed by bigger less focused organizations including Opsware (acquired by HP), Bladelogic (acquired by BMC) and Configuresoft (acquired by by VMware). In my opinion there is no real leader in this space.

In a conversation with Opscode CEO, Jesse Robbins, he shared his experience maintaining availability for web properties at Amazon.com. As a top ops guy and “master of disaster” at Amazon.com he had no access to these tools, they were simply sold in a way that was inconsistent with the way he evaluated and consumed products and services.  Opscode, a relatively new company develops the open source Chef project, which automates IT management via a client-server platform.

Opscode’s approach to server configuration challenges is to use recipes written in Ruby, the chef domain specific language(DSL). Then these cookbooks can be executed securely by the Chef client-server architecture and finally Chef is available as open source software to download, use and redistribute. In a nutshell Opscode met the following criteria is what interested me about their technology which is relatively easy to use, share configuration recipes and consume.

Chef, The Open Source Project

Chef is a systems integration framework released under the Apache License Version 2.0.  Chef, can manage servers by writing code in Ruby stored in configuration recipes called  cookbooks. Chef can integrate with existing infrastructure like LDAP via libraries using arbitrary Ruby code, either to extend Chef’s or to implement custom classes. Users can also configure applications that have dependencies on other parts of the infrastructure like databases and discern that information via the Chef server. However, I like Robbins’ description of Chef — sysadmin robot performing configuration tasks automatically and much more quickly than a single admin could ever hope to.

Though Chef was only released on January 15th , 2009 it has gotten rapid adoption and gained a large number of contributors. According to the Opscode wiki there are 157 approved contributors to Opscode projects and well over 20 companies. Beyond that the #chef IRC channel is typically attended by over 100 users and Opscode staff, signs of a healthy, growing open source community.

Opscode, The Platform

The Opscode Platform is the commercial offering from Opscode Inc. It is a centrally managed data store hosted by Opscode into which servers publish data such as IP addresses, loaded kernel modules, OS versions and more delivered as infrastructrue-as-a-service (IaaS). This data on the Opscode Platform can be accessed and becomes useful in the following ways:

  • Search-based Automation: All the data collected by the Opscode platform is indexed and searchable. Users can dynamically query this data from within Chef recipes to configure services that require complex configuration.
  • Role-based Access Control: The data index is has an access control system enabling administrators to centrally manage the level of infrastructure access.
  • Portability: The data stored on the Opscode Platform serves as a virtual blueprint of a given infrastructure, making it much easier to create perfect clones of a production environment.

The Opscode Platform is in a free beta release for the next 60 days. After the trial period, participants can manage up to 20 nodes on the Platform for $50 per month and $5 per month for each additional node. Pricing and availability information is available on their website.

Opscode, The Company

Opscode was founded in 2008 by Jesse Robbins and Adam Jacob, both experienced web operations leaders. Since then they have recruited a top notch team including cloud and systems management expert John Willis, Adrian Cole leader of the jclouds project and Christopher Brown a Founding Member, Architect, and Lead Developer for Amazon.com’s Elastic Compute Cloud (EC2).

Opscode today announced that they closed an $11 million Series B round of funding. The round was led by Battery Ventures (who also was an investor in BladLogic before it was acquired by BMC)  and includes a follow-on investment from Draper Fisher Jurvetson(DFJ) whose other open source investments include SugarCRM and Fonality. DFJ led Opscode’s Series A round of funding of $2.5 million, bringing the total amount raised for the company to $13.5 million, a sizable amount of capital to bring this technology to market.

Opscode is also seeing good adoption of Chef, not only do they have a few thousand active users on their wiki plus chef is currently in production at numerous top websites, including 37Signals, Etsy, IGN Entertainment, Scribd, and Wikia. Not only are web jockeys using chef but other large infrastructure providers are contributing to the project. Engine Yard, Rackspace, RightScale and the Springsource division of VMware have signed on to contribute to the project. They are even being very public about it as seen in this endorsement:

“We are excited about the open source contributions the Springsource Division of VMware has made to Opscode Chef.” said Javier Soltero, CTO of Springsource Management Products at VMware. “Chef is an important tool for automating infrastructure management and we look forward to its continued growth and success.”

Making Sysadmins into Superheroes

Opscode Chef is a hugely powerful tool that can greatly amplify the knowledge an effectiveness of systems administrators by automating a significant number of their maintenance tasks, improving their productivity and allowing them to focus on higher value tasks. Not only does Chef provide a framework for building systems but repairing them, keeping availability high and time to resolution low. This gives IT professionals a lot of leverage in getting their tasks done, allowing them to solve a problem once and then automate the process going forward. In other words Chef can turn systems administrators into super heroes by vastly improving their productivity and overall quality fo service.

The need that Opscode addresses can be filled to some degree through other software. Cfengine, Puppet and bcfg2 are all open source software solutions that address server configuration needs and have been around for some time. As mentioned above there is also large management suites that handle the same problems though they are expensive and have their own limitations. What is unique about Opscode approach is that they offer a robust, fully featured software platform as open source and a commercial offering that has full compatibility with the open source project.

This is somewhat unique as many commercial open source projects have a specific feature set that are only available to their enterprise customers. In a conversation with Opscode VP of Service, John M. Willis we discussed those users who are not interested in the Opscode platform but still want commercial support. He said that Opscode is building a select high-quality partner network that can handle these requests. Most recently Opscode has announced a partnership with DTO Solutions who employ members of the Control Tier project and is a big proponent of the DevOps approach to infrastructure management. Other partners will soon be onboard as well.

I am very much a fan of Opscode and there approach though my description probably doesn’t do it justice. Theoretically, a systems administrator who successfully implements automation tools such as Opscode can improve not only their productivity but the uptime of servers by improving speed to resolution for outages.  Opscode has a great opportunity thanks to a talented team, a novel go-to-market plan and a real need for these types of tools among IT professionals.

Related Articles

Technorati Tags: , , , , , , , ,

Read the original blog entry...

More Stories By Mark R. Hinkle

Mark Hinkle is the Senior Director, Open Soure Solutions at Citrix. He also is along-time open source expert and advocate. He is a co-founder of both the Open Source Management Consortium and the Desktop Linux Consortium. He has served as Editor-in-Chief for both LinuxWorld Magazine and Enterprise Open Source Magazine. Hinkle is also the author of the book, "Windows to Linux Business Desktop Migration" (Thomson, 2006). His blog on open source, technology, and new media can be found at http://www.socializedsoftware.com.

@MicroservicesExpo Stories
While we understand Agile as a means to accelerate innovation, manage uncertainty and cope with ambiguity, many are inclined to think that it conflicts with the objectives of traditional engineering projects, such as building a highway, skyscraper or power plant. These are plan-driven and predictive projects that seek to avoid any uncertainty. This type of thinking, however, is short-sighted. Agile approaches are valuable in controlling uncertainty because they constrain the complexity that ste...
Agile has finally jumped the technology shark, expanding outside the software world. Enterprises are now increasingly adopting Agile practices across their organizations in order to successfully navigate the disruptive waters that threaten to drown them. In our quest for establishing change as a core competency in our organizations, this business-centric notion of Agile is an essential component of Agile Digital Transformation. In the years since the publication of the Agile Manifesto, the conn...
The cloud revolution in enterprises has very clearly crossed the phase of proof-of-concepts into a truly mainstream adoption. One of most popular enterprise-wide initiatives currently going on are “cloud migration” programs of some kind or another. Finding business value for these programs is not hard to fathom – they include hyperelasticity in infrastructure consumption, subscription based models, and agility derived from rapid speed of deployment of applications. These factors will continue to...
"This all sounds great. But it's just not realistic." This is what a group of five senior IT executives told me during a workshop I held not long ago. We were working through an exercise on the organizational characteristics necessary to successfully execute a digital transformation, and the group was doing their ‘readout.' The executives loved everything we discussed and agreed that if such an environment existed, it would make transformation much easier. They just didn't believe it was reali...
"Opsani helps the enterprise adopt containers, help them move their infrastructure into this modern world of DevOps, accelerate the delivery of new features into production, and really get them going on the container path," explained Ross Schibler, CEO of Opsani, and Peter Nickolov, CTO of Opsani, in this SYS-CON.tv interview at DevOps Summit at 21st Cloud Expo, held Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA.
"We're developing a software that is based on the cloud environment and we are providing those services to corporations and the general public," explained Seungmin Kim, CEO/CTO of SM Systems Inc., in this SYS-CON.tv interview at 21st Cloud Expo, held Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA.
Enterprises are adopting Kubernetes to accelerate the development and the delivery of cloud-native applications. However, sharing a Kubernetes cluster between members of the same team can be challenging. And, sharing clusters across multiple teams is even harder. Kubernetes offers several constructs to help implement segmentation and isolation. However, these primitives can be complex to understand and apply. As a result, it’s becoming common for enterprises to end up with several clusters. Thi...
"Codigm is based on the cloud and we are here to explore marketing opportunities in America. Our mission is to make an ecosystem of the SW environment that anyone can understand, learn, teach, and develop the SW on the cloud," explained Sung Tae Ryu, CEO of Codigm, in this SYS-CON.tv interview at 21st Cloud Expo, held Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA.
"CA has been doing a lot of things in the area of DevOps. Now we have a complete set of tool sets in order to enable customers to go all the way from planning to development to testing down to release into the operations," explained Aruna Ravichandran, Vice President of Global Marketing and Strategy at CA Technologies, in this SYS-CON.tv interview at DevOps Summit at 21st Cloud Expo, held Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA.
The nature of test environments is inherently temporary—you set up an environment, run through an automated test suite, and then tear down the environment. If you can reduce the cycle time for this process down to hours or minutes, then you may be able to cut your test environment budgets considerably. The impact of cloud adoption on test environments is a valuable advancement in both cost savings and agility. The on-demand model takes advantage of public cloud APIs requiring only payment for t...
Cavirin Systems has just announced C2, a SaaS offering designed to bring continuous security assessment and remediation to hybrid environments, containers, and data centers. Cavirin C2 is deployed within Amazon Web Services (AWS) and features a flexible licensing model for easy scalability and clear pay-as-you-go pricing. Although native to AWS, it also supports assessment and remediation of virtual or container instances within Microsoft Azure, Google Cloud Platform (GCP), or on-premise. By dr...
Let's do a visualization exercise. Imagine it's December 31, 2018, and you're ringing in the New Year with your friends and family. You think back on everything that you accomplished in the last year: your company's revenue is through the roof thanks to the success of your product, and you were promoted to Lead Developer. 2019 is poised to be an even bigger year for your company because you have the tools and insight to scale as quickly as demand requires. You're a happy human, and it's not just...
Many enterprise and government IT organizations are realizing the benefits of cloud computing by extending IT delivery and management processes across private and public cloud services. But they are often challenged with balancing the need for centralized cloud governance without stifling user-driven innovation. This strategy requires an approach that fundamentally reshapes how IT is delivered today, shifting the focus from infrastructure to services aggregation, and mixing and matching the bes...
identify the sources of event storms and performance anomalies will require automated, real-time root-cause analysis. I think Enterprise Management Associates said it well: “The data and metrics collected at instrumentation points across the application ecosystem are essential to performance monitoring and root cause analysis. However, analytics capable of transforming data and metrics into an application-focused report or dashboards are what separates actual application monitoring from relat...
The benefits of automation are well documented; it increases productivity, cuts cost and minimizes errors. It eliminates repetitive manual tasks, freeing us up to be more innovative. By that logic, surely, we should automate everything possible, right? So, is attempting to automate everything a sensible - even feasible - goal? In a word: no. Consider this your short guide as to what to automate and what not to automate.
DevOps teams have more on their plate than ever. As infrastructure needs grow, so does the time required to ensure that everything's running smoothly. This makes automation crucial - especially in the server and network monitoring world. Server monitoring tools can save teams time by automating server management and providing real-time performance updates. As budgets reset for the New Year, there is no better time to implement a new server monitoring tool (or re-evaluate your current solution)....
While some developers care passionately about how data centers and clouds are architected, for most, it is only the end result that matters. To the majority of companies, technology exists to solve a business problem, and only delivers value when it is solving that problem. 2017 brings the mainstream adoption of containers for production workloads. In his session at 21st Cloud Expo, Ben McCormack, VP of Operations at Evernote, discussed how data centers of the future will be managed, how the p...
We just came off of a review of a product that handles both containers and virtual machines in the same interface. Under the covers, implementation of containers defaults to LXC, though recently Docker support was added. When reading online, or searching for information, increasingly we see “Container Management” products listed as competitors to Docker, when in reality things like Rocket, LXC/LXD, and Virtualization are Dockers competitors. After doing some looking around, we have decided tha...
High-velocity engineering teams are applying not only continuous delivery processes, but also lessons in experimentation from established leaders like Amazon, Netflix, and Facebook. These companies have made experimentation a foundation for their release processes, allowing them to try out major feature releases and redesigns within smaller groups before making them broadly available. In his session at 21st Cloud Expo, Brian Lucas, Senior Staff Engineer at Optimizely, discussed how by using ne...
Digital transformation has changed the way users interact with the world, and the traditional healthcare experience no longer meets rising consumer expectations. Enterprise Health Clouds (EHCs) are designed to easily and securely deliver the smart and engaging digital health experience that patients expect today, while ensuring the compliance and data integration that care providers require. Jikku Venkat