How to find good DevOps candidates?

Software is eating this world alive, and in 2020 it is even truer than back in 2011 when Marc Andreessen published his famous NYT essay. Unless you are a pizza house around the corner, your business has to operate some infrastructure and interact with customers and contractors online using some apps (btw, Domino’s and Pizza Hut both have huge and ultra-modern infrastructures in the cloud). 

Thus said, to remain competitive nowadays, every business has to gain access to DevOps expertise in order to manage their infrastructures cost-efficiently, and most businesses try to hire DevOps engineers in-house. However, the successful implementation of projects demands a high level of expertise, so the question is — how to find good DevOps candidates?

The basic problem with this endeavor is that there is no way to tell if a DevOps engineer is good at the interview unless you already have your own DevOps expertise at hand. The reason for this is that DevOps is not a set of rules carved in rock, which can be learned by anyone and still be the same, just like a periodical table of chemical elements. DevOps is a set of best practices of infrastructure management, ethos and culture of collaboration, quite a long list of DevOps tools and much, much more. This means that there might be a variety of talents with a wide range of skills and preferred tools, who will all seem to have the qualities you seek.

Thus said, how do you evaluate a candidate for a job that might bring your company a fortune or cost you dearly? IT Svit answers this question based on our 5+ years of experience with providing Managed DevOps Services to businesses and organizations of all sizes worldwide. Keep reading to find out how to find good DevOps candidates if you don’t have DevOps experience yourself.

DevOps bootcamp: why use DevOps at all?

Let’ start by busting some myths about DevOps processes to give you a solid understanding of what it is and what it isn’t. There is a huge number of DevOps definitions, due to the fact that all the authors define DevOps based on their experience. This means everybody has their own pretenses that differ from each other quite strongly.

DevOps is a culture of collaboration and communication between Devs, QA engineers and system (Operations) engineers, used to minimize the risks of software delivery and optimize infrastructure performance. To say it even shorter — DevOps is about sharing the goals and the risks to move new code into production faster.

Why is it so and what do we mean by sharing the goals and the risks? Commonly the Devs, the QA and the Ops engineers have siloed tasks, goals and responsibilities — and risks, if something goes wrong.

• The main goal of a developer is to release as much code as possible to finish the project before the deadline. The risks are all the reasons that prolong the code release — from manual environment configuration to slowpoke testing and bug fixing.
• The main goal of a QA engineer is to find as many bugs as possible (all of them, ideally) before the code goes into production. The risks are all the things that require time (broken builds, lengthy manual environment provisioning, merge conflicts, etc.)
• The main goal of a system engineer (Ops) is to ensure maximum system uptime and fastest recovery from incidents (and minimize the number of incidents over time). The risks are all the actions that reduce the time that can be spent on improving the infrastructure (repetitive manual environment configuration, post-release rollbacks and recovery, etc)

As you can see, the repetitive manual provisioning and configuration of virtual machines is by far the most time- and effort-consuming aspect of IT operations and one of the biggest risks for all the parties involved.

Enter DevOps — the ethos of collaboration between system engineers and software developers/QA specialists aimed at minimizing the time and effort needed to develop high-quality code, release it to end-users without risks and run your infrastructure in production without major incidents. Thus said, adopting DevOps workflows will definitely be in your best interests — but how to know if the candidate is indeed a good fit for you?

Below is a step-by-step roadmap to finding a good DevOps candidate.

Hiring DevOps: identify and fill the skill gaps

To hire a good talent you need to know what you want to get, so you can look for relevant skills and expertise. Thus said, you need to evaluate the skills, tools and workflows you already use (as they are a part of the reason why you need DevOps). For example, does your company employ a team of excellent developers who just need a good system engineer to automate the code delivery pipeline? Or do you need a skilled system architect able to fix the application Python code on the fly? Or a software engineer that will help split your monolith app into microservices and restructure it for scalability? Identify your strengths, this will help you highlight the gaps you need to fill with DevOps skills.

Once you have a full list of tools your business uses before you, ask yourself a very important question: are you sure you are going to use them in 2-3 years? For example, such container management tools as Mesos/Marathon, Docker Swarm and Kubernetes were nearly equal in popularity several years ago — but now Mesos is all but forgotten, Swarm is used in precious few projects and Kubernetes is the undisputed ruler of the scene. 

The same goes for Jenkins, Ansible, Terraform, Prometheus, ELK and other DevOps tools that are either mainstream already or are definitely will be standard system components in the future. Are you already using them, and if not — maybe DevOps transformation is a perfect opportunity to start doing so? Otherwise, you might invest huge time and money to just find out Maven, Rancher, Chef and Icinga are not needed by anyone in 3 years or so.

However, it is also important to keep in mind that the tools are used to achieve some goal, and a person must be willing to achieve it in order to use the right tools and do it correctly. The tools we use can be changed and new tools mastered — if the goal is right. Your candidate must be passionate about continuous integration and delivery of code, constantly improving the infrastructure efficiency and efficiency of operations, not about using Puppet or Jenkins or any other tool. 

Look for competencies, work experience and correct attitudes, not for specific tool names. Otherwise, you can miss a great fit because they did not have Terraform in their resume. However, while the tooling a candidate might list is not that essentials, he/she must be familiar with a wide range of tools used at various stages of your IT operations. We provide a list of essential DevOps competencies and tools that can be used to implement them.

Once you have a list of competencies you want your DevOps engineer(s) to possess, you can start monitoring the job portals, LinkedIn and other sources for the candidates with an appropriate skill set. However, skills alone won’t get the job done, and the best way to evaluate a candidate is to give him/her a test task similar to the one they will be dealing with while working for your company. 

Hiring a DevOps engineer: let them show what they can do

Naturally, a DevOps engineer must also have the same set of personal skills and characteristics as any other specialist you employ:

• Soft skills and collaboration. DevOps is a culture of collaboration, first and foremost. This role emphasizes teamwork to constantly evaluate the effectiveness of processes in place and optimize them.
• Hard skills and willingness to learn. As we discussed above, the DevOps engineer must be proficient with the tools you need and be willing/able to learn new ones as needed.
• Goal-orientedness. DevOps engineers must be the engines that propel your company to operational stability and cost-efficiency. They must be able to form a strategy, prove its viability and follow it to completion.

How to check this? We suggest that the best way to do it is by giving a candidate a task he/she must accomplish to their liking and at their own pace (but before some deadline). This means that after having a technical call to evaluate the general background and attitude of a candidate, you need to give them a task similar to the work they will be doing for you and let them achieve the result the way they want. Evaluating the results of this task will show you if the candidate is capable of the following:

• Reading the app code and grasping how it works and what runtime is needed to launch it
• Writing the scripts and code that enable the automated deployment of a simple app, along with configuring all the needed dependencies and networking
• Follow the requirements and deliver the job without trying to hide any issues or taking any shortcuts.

As an example, here is a task some advertising company offers to their DevOps candidates:

Get this simple Flask app up and running on a Vagrant VM using your known best DevOps practices as well as the instructions in the guidelines below. You must use Ansible as your provisioner to install and configure everything. When you’re done, we should be able to type vagrant up and our app will be running and reachable on HTTP port 80 at http://192.168.33.15 and http://devops.adsnative.com

Guidelines:

• The provisioner should clone this GitHub repo into the VM under /webapps/adsnative
• You should be able to find what system packages are needed by looking through the app
• You should not need to change the app code in any way
• The app should be running as a non-privileged user
• The app should be automatically restarted if crashes or is killed
• The app should maximize all of the available CPUs (have Vagrant virtualize multiple CPUs)
• The app’s logs should be captured to /var/log/adsnative/app.log, and they should be rotated hourly
• Timezone should be in US EST

Write your ansible code and templates in your own private repo and when complete, send to us the contents in a zip or tarball.

You may stick to the default Vagrant image (precise64) or use another preferred flavor of Linux.

We’ll evaluate the submission in terms of simplicity, code quality, and the use of best practices of the provisioning code. Feel free to be creative and take liberties where you feel it will improve the deliverable!

Analyzing the deliverables will help you evaluate if the candidate was able to follow the instructions if he/she is familiar with all the tools and best practices used by your company, the level of quality of the resulting solution, etc. By comparing the results of the technical interviews with the quality of test tasks delivered, you will be able to make a pretty good impression of the applicant and ensure you find good DevOps candidates. For instance, take a close look at the following points:

• Was the app launched at all (what are the results of typing vagrant up)?
• Is the app running as intended? (a tip – a 200 response alone does not mean the app works)
• Were all the task guidelines followed? If not – why?
• Did the candidates follow Ansible playbook best practices?
• Did they write their own code or found some relevant open-source solutions?
• Did a candidate walk an extra mile (like leaving comments and instruction or writing short developer documentation, did they install any monitoring service agent, are there any debugging tools configured, how are the logs stored, did they issue and use SSH keys, etc)?

Most importantly, you will be able to provide constructive feedback based on the task requirements and make a well-informed decision regarding your possible new DevOps engineer.

Conclusions: it’s hard to find good DevOps candidates, but it can be done

You might have noticed that we mentioned that not all companies go for hiring a DevOps talent in-house. As one of the world-leading Managed DevOps Services providers, IT Svit has ample experience with helping various companies enable DevOps best practices for their projects and operations. If you don’t want to spend much effort on finding, interviewing and evaluating multiple DevOps candidates — you can simply contact IT Svit and our experienced DevOps team will handle your projects and achieve your business objectives!