|By XebiaLabs Blog||
|February 22, 2017 10:00 AM EST||
Guest Post: Building a Winning DevOps Team
By Adi Glasman
DevOps, and Ops in general, is one of the most important components of a company’s production machine. Investing in building a winning DevOps team should be a priority for every company and will pay off quickly.
As organizations and operations grow (products, features, number of customers, etc.), the scale of traffic, data, OS types and so on, increase rapidly. More products to support, new features to release and a lot more in-house cycles (e.g., QA, CI\CD, SME, SRE) need to be managed.
A DevOps team’s first mission is to build, maintain and support the company’s technology stack. Whether it’s a PaaS, SaaS or IaaS environment, microservices or monolithic, Docker or K8S, it does not matter – DevOps engineers need to be able to support any environment the company chooses at any time and anywhere (On Premises, Data Centers, Cloud).
Here are the subject areas any organization and it’s DevOps engineers have to deal with:
The list goes on, and every item is its own area of knowledge. It’s “a small world” of expertise, and any DevOps engineer is expected to master it all (or at least a large subset of it ¯\_(ツ)_/¯).
For example, let’s say your company is building the next edge, top of the line API serving tool. The tool is web based and expected to serve TB of daily transactions. TB of events pounding your chosen big data tools, logging everything for sanity, monitoring everything so you’ll be able to wake anyone up when necessary, profiling, ingesting, segmenting…It’s growing and doing so exponentially.
Recruiting and finding DevOps engineers to support this environment is challenging. Finding someone who understands all the aforementioned subjects is rare, and if you do find that person, they’ll cost you accordingly. (Let’s assume money is not an obstacle right now and focus on finding the right engineer to fulfill, create and maintain your environment.)
A Six Step Process
My strategy is divide and conquer. Let’s break down and define what we’re looking for.
Defining the interview process is crucial for successful hiring. A good process will result in your finding a quality candidate (technical, personal and cultural) and will help maximize the candidate’s compatibility with the company and role.
Throughout my years of experience, I’ve defined a 6 step process that helps me form and create winning engineering teams:
- Step 1 – Define the role, and sync with the company recruiter.
- Step 2 – Phone screen.
- Step 3 – Break down the skills to interview modules.
- Step 4 – Prepare questionnaire for each module.
- Step 5 – Form your interviewer squad and begin interviewing.
- Step 6 – Closer. All interviewers meet and decide.
Step 1. Define the role, and sync with the company recruiter.
The company recruiter will be your liaison to the candidate and will have the first interaction with them. It’s important to follow these steps:
- The recruiting manager must have a conversation with the recruiter and make sure they understand the job description and the role’s technical requirements. Talk with them, explain yourself, explain the role and expectations. Don’t skip this stage. If you do, you’ll end up wasting everyone’s time in the long run.
- The recruiter should have a phone conversation with the candidate. The purpose of this conversation is to understand if the candidate is a good fit for the company and how the position fits within the company culture, the candidate’s career development, and so on. This call also involves verifying the content of the candidate’s CV.
The recruiter should pass their feedback to the hiring manager and if both gives thumbs up then proceed to Step 2.
Step 2. Conduct a phone screen interview.
Phone screen interviews are very helpful and significant time savers. Since the process is time consuming and involves numerous engineers interviewing the candidate, it’s important your time is well spent. You should respect the candidate’s time as well.
After posting\advertising the job, people will start applying. Your time is precious so treat it accordingly! Even if the CV is appealing and it seems like the candidate is a perfect match, you still have to set up two phone calls, each being no more than 15-20 minutes:
- Recruiting Manager Interview – The recruiting manager needs to talk with the candidate to understand if the person is qualified and is a good fit for the team and company culture. They should also use the time to find out whether the candidate thinks the role meets their career aspirations, as well as validate the candidate’s CV.
- Technical Interviewer – This can be any existing DevOps engineer or software engineer who can ask a series of easy, medium or hard questions related to the posted position and candidate’s background.
The recruiting manager and the technical interviewer should meet following these interviews. If both give thumbs up, proceed to step 3.
Step 3. Break down the skills to interview modules.
Interview modules represent a subject to discuss during the interview. For example:
- System + Networking
- Methodologies\Processes – Agile, CI\CD, etc.
- Software Engineering, Architecture Design
- Problem solving, debugging, monitoring
- Culture fit
- Practical\Code challenge
Step 4. Prepare a questionnaire for each module.
For each module, prepare a questionnaire that applies to different skill levels (junior, intermediate, senior and expert). This is best organized in a question and answer format as the answer section will be helpful in situations where the answer differs from what is expected.
System + Networking module example:
These questions will assess a candidate’s skills and knowledge in areas of your specific flavor of operating systems and scalable networking. It is very important that these areas are well understood to allow for effective troubleshooting and competency in a technical environment. Lacking this basic knowledge can result in a long learning curve.
- How does traceroute command work?
- What is ARP?
- What is jitter/latency/packet-loss-ratio?
- What is proxy ARP?
- How do you set up a TCP connection?
System engineering module example:
The intent of this module is to determine whether the candidate has a reliable and repeatable process for solving complex problems. The interviewer will present a subset of the problems outlined below and progressively increase the complexity of the required solution through discussion with the candidate. If the problem field of knowledge is unknown to the candidate, only the process used to arrive at a solution should be judged and not the solution itself:
You are to architect a new email system capable of supporting an expanding user base with low delivery times. The incoming mail throughput is expected to be around a million emails per minute.
- Describe in detail or draw the mail system you would implement.
- What software choices would you make for each layer and why?
- What are your kernel, networking, storage and monitoring considerations?
Step 5. Form your interviewer squad and begin interviewing.
Each interviewer should pick a module that fits with their skill set so that they’re able to ask and understand the candidate answers. Every interviewer takes notes and gathers feedback. Best of luck for a great interview!
Step 6. Closer. All interviewers meet and decide.
All interviewers meet and share thoughts and concerns. At the end of the meeting, ideally each person will be able to give a thumbs up or down. If, however, there’s a lack of clarity or unresolved concerns about the candidate, you can set up an additional interview before making a final decision.
I’ve been using this method for a couple of years and it’s proven to be very successful.
Adi Glasman is a Senior Operations Manager at APAC – Zendesk. You can find more articles by Adi, as well as the original post here.
Without a clear strategy for cost control and an architecture designed with cloud services in mind, costs and operational performance can quickly get out of control. To avoid multiple architectural redesigns requires extensive thought and planning. Boundary (now part of BMC) launched a new public-facing multi-tenant high resolution monitoring service on Amazon AWS two years ago, facing challenges and learning best practices in the early days of the new service.
Mar. 27, 2017 03:45 AM EDT Reads: 2,933
All organizations that did not originate this moment have a pre-existing culture as well as legacy technology and processes that can be more or less amenable to DevOps implementation. That organizational culture is influenced by the personalities and management styles of Executive Management, the wider culture in which the organization is situated, and the personalities of key team members at all levels of the organization. This culture and entrenched interests usually throw a wrench in the work...
Mar. 27, 2017 03:00 AM EDT Reads: 2,956
DevOps is often described as a combination of technology and culture. Without both, DevOps isn't complete. However, applying the culture to outdated technology is a recipe for disaster; as response times grow and connections between teams are delayed by technology, the culture will die. A Nutanix Enterprise Cloud has many benefits that provide the needed base for a true DevOps paradigm.
Mar. 27, 2017 12:45 AM EDT Reads: 2,010
As software becomes more and more complex, we, as software developers, have been splitting up our code into smaller and smaller components. This is also true for the environment in which we run our code: going from bare metal, to VMs to the modern-day Cloud Native world of containers, schedulers and micro services. While we have figured out how to run containerized applications in the cloud using schedulers, we've yet to come up with a good solution to bridge the gap between getting your contain...
Mar. 26, 2017 09:45 PM EDT Reads: 7,578
As organizations realize the scope of the Internet of Things, gaining key insights from Big Data, through the use of advanced analytics, becomes crucial. However, IoT also creates the need for petabyte scale storage of data from millions of devices. A new type of Storage is required which seamlessly integrates robust data analytics with massive scale. These storage systems will act as “smart systems” provide in-place analytics that speed discovery and enable businesses to quickly derive meaningf...
Mar. 26, 2017 07:45 PM EDT Reads: 9,591
DevOps is often described as a combination of technology and culture. Without both, DevOps isn't complete. However, applying the culture to outdated technology is a recipe for disaster; as response times grow and connections between teams are delayed by technology, the culture will die. A Nutanix Enterprise Cloud has many benefits that provide the needed base for a true DevOps paradigm. In his Day 3 Keynote at 20th Cloud Expo, Chris Brown, a Solutions Marketing Manager at Nutanix, will explore t...
Mar. 26, 2017 03:15 PM EDT Reads: 2,806
DevOps has often been described in terms of CAMS: Culture, Automation, Measuring, Sharing. While we’ve seen a lot of focus on the “A” and even on the “M”, there are very few examples of why the “C" is equally important in the DevOps equation. In her session at @DevOps Summit, Lori MacVittie, of F5 Networks, explored HTTP/1 and HTTP/2 along with Microservices to illustrate why a collaborative culture between Dev, Ops, and the Network is critical to ensuring success.
Mar. 26, 2017 03:00 PM EDT Reads: 10,546
With major technology companies and startups seriously embracing Cloud strategies, now is the perfect time to attend @CloudExpo | @ThingsExpo, June 6-8, 2017, at the Javits Center in New York City, NY and October 31 - November 2, 2017, Santa Clara Convention Center, CA. Learn what is going on, contribute to the discussions, and ensure that your enterprise is on the right path to Digital Transformation.
Mar. 26, 2017 01:45 PM EDT Reads: 8,512
Everyone wants to use containers, but monitoring containers is hard. New ephemeral architecture introduces new challenges in how monitoring tools need to monitor and visualize containers, so your team can make sense of everything. In his session at @DevOpsSummit, David Gildeh, co-founder and CEO of Outlyer, will go through the challenges and show there is light at the end of the tunnel if you use the right tools and understand what you need to be monitoring to successfully use containers in your...
Mar. 26, 2017 01:00 PM EDT Reads: 1,583
What if you could build a web application that could support true web-scale traffic without having to ever provision or manage a single server? Sounds magical, and it is! In his session at 20th Cloud Expo, Chris Munns, Senior Developer Advocate for Serverless Applications at Amazon Web Services, will show how to build a serverless website that scales automatically using services like AWS Lambda, Amazon API Gateway, and Amazon S3. We will review several frameworks that can help you build serverle...
Mar. 26, 2017 12:45 PM EDT Reads: 1,919
The IT industry is undergoing a significant evolution to keep up with cloud application demand. We see this happening as a mindset shift, from traditional IT teams to more well-rounded, cloud-focused job roles. The IT industry has become so cloud-minded that Gartner predicts that by 2020, this cloud shift will impact more than $1 trillion of global IT spending. This shift, however, has left some IT professionals feeling a little anxious about what lies ahead. The good news is that cloud computin...
Mar. 26, 2017 10:30 AM EDT Reads: 1,262
SYS-CON Events announced today that HTBase will exhibit at SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. HTBase (Gartner 2016 Cool Vendor) delivers a Composable IT infrastructure solution architected for agility and increased efficiency. It turns compute, storage, and fabric into fluid pools of resources that are easily composed and re-composed to meet each application’s needs. With HTBase, companies can quickly prov...
Mar. 26, 2017 08:15 AM EDT Reads: 2,916
An overall theme of Cloud computing and the specific practices within it is fundamentally one of automation. The core value of technology is to continually automate low level procedures to free up people to work on more value add activities, ultimately leading to the utopian goal of full Autonomic Computing. For example a great way to define your plan for DevOps tool chain adoption is through this lens. In this TechTarget article they outline a simple maturity model for planning this.
Mar. 26, 2017 06:00 AM EDT Reads: 4,286
While DevOps most critically and famously fosters collaboration, communication, and integration through cultural change, culture is more of an output than an input. In order to actively drive cultural evolution, organizations must make substantial organizational and process changes, and adopt new technologies, to encourage a DevOps culture. Moderated by Andi Mann, panelists discussed how to balance these three pillars of DevOps, where to focus attention (and resources), where organizations might...
Mar. 26, 2017 05:15 AM EDT Reads: 6,184
The rise of containers and microservices has skyrocketed the rate at which new applications are moved into production environments today. While developers have been deploying containers to speed up the development processes for some time, there still remain challenges with running microservices efficiently. Most existing IT monitoring tools don’t actually maintain visibility into the containers that make up microservices. As those container applications move into production, some IT operations t...
Mar. 26, 2017 01:00 AM EDT Reads: 2,970
For organizations that have amassed large sums of software complexity, taking a microservices approach is the first step toward DevOps and continuous improvement / development. Integrating system-level analysis with microservices makes it easier to change and add functionality to applications at any time without the increase of risk. Before you start big transformation projects or a cloud migration, make sure these changes won’t take down your entire organization.
Mar. 25, 2017 09:45 PM EDT Reads: 3,647
Software development is a moving target. You have to keep your eye on trends in the tech space that haven’t even happened yet just to stay current. Consider what’s happened with augmented reality (AR) in this year alone. If you said you were working on an AR app in 2015, you might have gotten a lot of blank stares or jokes about Google Glass. Then Pokémon GO happened. Like AR, the trends listed below have been building steam for some time, but they’ll be taking off in surprising new directions b...
Mar. 25, 2017 01:30 PM EDT Reads: 5,836
@DevOpsSummit has been named the ‘Top DevOps Influencer' by iTrend. iTrend processes millions of conversations, tweets, interactions, news articles, press releases, blog posts - and extract meaning form them and analyzes mobile and desktop software platforms used to communicate, various metadata (such as geo location), and automation tools. In overall placement, @DevOpsSummit ranked as the number one ‘DevOps Influencer' followed by @CloudExpo at third, and @MicroservicesE at 24th.
Mar. 25, 2017 10:30 AM EDT Reads: 10,337
Culture is the most important ingredient of DevOps. The challenge for most organizations is defining and communicating a vision of beneficial DevOps culture for their organizations, and then facilitating the changes needed to achieve that. Often this comes down to an ability to provide true leadership. As a CIO, are your direct reports IT managers or are they IT leaders? The hard truth is that many IT managers have risen through the ranks based on their technical skills, not their leadership abi...
Mar. 25, 2017 05:00 AM EDT Reads: 11,007
The essence of cloud computing is that all consumable IT resources are delivered as services. In his session at 15th Cloud Expo, Yung Chou, Technology Evangelist at Microsoft, demonstrated the concepts and implementations of two important cloud computing deliveries: Infrastructure as a Service (IaaS) and Platform as a Service (PaaS). He discussed from business and technical viewpoints what exactly they are, why we care, how they are different and in what ways, and the strategies for IT to transi...
Mar. 25, 2017 05:00 AM EDT Reads: 6,120