Transition to DevOps: 5 sure ways to fail

DevOps is like a Holy Grail of the modern software development. Many are actively searching for it, some claim they have found it and a vast majority is still waiting to begin their journey.

Unfortunately, there are no step-by-step instructions leading to a glorious and overwhelming success, as every company has a unique style of operations. However, there definitely are 5 sure ways to fail a transition to DevOps and today we are going to share them, so any business can be aware of what NOT to do.

The complexity of transition to DevOps varies greatly between a nimble startup or a train-building enterprise, and we have experience delivering to both.

At IT Svit we have more than a decade of experience in helping businesses of all size building a viable and reliable infrastructure. In addition, we have helped both startups and established enterprises transition to using DevOps methodology of software delivery for more than 5 years.

Our clients include nationwide infrastructure companies and automotive giants, multinational payment platforms and political campaigners. The complexity of transition to DevOps varies greatly between a nimble startup or a train-building enterprise, and we have experience delivering to both.

However, we’ve seen not only happy endings, as more than a couple of times we had to intervene amidst the process and rectify the mistakes made by other contractors or due to incorrect planning/vision of the process. Here is what these mistakes looked like.

Failing with determining what DevOps means for your business

There is no strict definition of DevOps, but there are multiple explanations of why did it appear and what problems adopting DevOps can help solve. There is a beautiful post back from 2010 from one of the DevOps movement founders Stephen Nelson-Smith on what is the whole DevOps thing. To put it out clear, all the members of Development, Operations and QA team should reach a consensus on what DevOps means for them. Answering these questions will be of help:

• How does YOUR team define the DevOps process? The consensus should be reached before any processes take place, otherwise unneeded tension will arise all of the time, and instead of working the team will spend time defining what must and must not be done.
• What privileges are your personnel willing to lose and what tasks are they willing to delve into? Clearly, if the developers don’t want to be involved in testing and infrastructure maintenance, while Ops engineers are not eager to begin coding themselves and QA team preferring manual testing to writing automated unit tests — such people will not prevail in building a healthy and competitive DevOps team.
• What boundaries will there be between the newly established DevOps and the rest of the enterprise departments? Obviously, the transition to DevOps software delivery pipeline cannot happen overnight and demands quite a significant investment of time and resources. During that time the pilot DevOps teams should work on separate tasks outside of the scope of the main enterprise business workflow in so-called Centers of Excellence. We explained this in more details in our article on the transition to DevOps for enterprises.

Failing to adhere to DevOps culture

DevOps methodology is actually only around about 25% using DevOps instruments like Ansible, Kubernetes, Salt, Puppet or Terraform. To a much larger extent (namely, 75%), DevOps approach relies on cultural changes within the organization, as we described in our article on why DevOps culture is a huge step for mankind.

The mottos of DevOps are quite simple: providing continuous integration, continuous delivery and building infrastructure as code. This means the DevOps engineers must not only be all-around skilled with using multiple DevOps tools for code development, building, testing and deployment; they should also be able to deliver automated testing for their code and maintain the production infrastructure, so any rising issues can be solved by any team member available.

The last, but not the least important component of a successful adoption of DevOps paradigm is building infrastructure as code, which is possible only after moving the enterprise infrastructure to the private or hybrid cloud. When all the infrastructure parameters are configured through files that can be easily accessed and modified by any team member, there are no bottlenecks and operational overhead. This corresponds to the “you build it, you run it” paradigm, forming the core of DevOps methodology.

Such cultural transition from siloed tasks and responsibilities to a unified team with common toolset and skill set, as well as a shared mindset is essential for producing a viable and durable DevOps team. Most importantly, this cannot happen as a grassroots initiative and must be backed by the wholehearted support of C-Suite and managers throughout the organization. This, however, raises the concerns about governance and security, which should be addressed.

Failing to reorganize the managerial structure and governance

One of the reasons transition to DevOps still does not happen ubiquitously is that in enterprise business the cost of error is too high. This means wrong decisions can cost the managers their positions, which is why they are really resistant to changes. “Why change it if it works” paradigm prevails. The bad thing is, following the legacy practices does not actually work quite well in the modern fast-paced world, as failure to deliver value ON TIME means failure to deliver value AT ALL.

Therefore, in order to speed up the software development, as much managerial overhead should be removed from the delivery pipeline, as possible. However, removing the need for multiple approvals does not mean removing the need for following the security and compliance restrictions. It simply means the DevOps should be responsible for the consequences of their actions and have the power to correct errors if all hell breaks loose.

The correct way of doing this is duplicating the approval functions across the software delivery pipeline with the DevOps team while leaving the final word to the manager. This way, the managers allow the DevOps to make decisions regarding software deployment and infrastructure updates, but can cancel them should the need be. If such control is absent, the unified structure of software deployment and infrastructure architecture across the enterprise might turn into a disorganized mess.

Only once the DevOps engineers develop a stable and error-proof workflow leading to 10 updates out of 10 going smoothly, the DevOps can be allowed to work on their own. Such incremental transition of tasks and responsibilities should happen across the enterprise. This helps build a strong and self-confident team that can work autonomously, while still complying with security and governance regulations.

Failing to evaluate the risks

Continuous delivery, automated testing, continuous integration, building immutable infrastructure as code — all of these DevOps benefits lead to drastically shortening the time to market and feedback loops. However, the same actions can lead to significant losses, should the things go awry. Thus said, the DevOps engineers should be trained to remove as many error-generating factors out of the equation as possible.

This generally means automating the testing procedures, but as a saying goes: “if it can be automated, it must be automated”. The cost of error on the latest stages of software delivery process is quite high for the business, which can cause a detrimental effect and burnout for the QA and DevOps teams. To avoid such results it’s better to devote significant effort from DevOps to writing automated unit tests and hone their skills in using configuration management tools like Ansible and Terraform, as well as mastering all the possibilities Docker provides. This allows deploying testing and staging environments quickly, helps with issuing rolling updates and enables rapid rollbacks to previous versions, should the need arise. Such practices help mitigate the risks and provide stable, uninterrupted service.

Failing to provide measurable statistics

Let’s be honest, while there is a bright idea in the heart of every endeavor, the core goal of any business remains the same — making the profit (or deliver the impact, if we speak of NPOs). Thus said, every change should be first and foremost feasible to be approved by the executives. While DevOps approach does help speed up the service and software delivery process, it should meet certain KPIs (Key Performance Indicators) to justify implementing it.

These KPIs will differ depending on the industry, the state of legacy infrastructure and a plethora of other factors, yet one remains the same: the adoption of DevOps should solve problems and prove tangible results. In order to provide those results, the management should audit the current state of the legacy infrastructure, systems and processes used across the enterprise. Such audit will allow determining the approximate levels of efficiency and performance, like the number of days from issuing a task for a new feature till the release of this feature, etc.

Once such parameters are identified, measuring the success becomes quite a simple task. This is not a mission for a month or two, however, as deploying a full-scale DevOps model can take up to a year, depending on the size of the organization and the amount of transformational effort needed to reorganize the workflow. Keeping a close eye on the DevOps team performance and comparing it to the performance of the conventional teams helps evaluate the progress and efficiency of the transition to DevOps approach.

Conclusions

Transition to DevOps is not an overnight effort. This should become a well-grounded decision, supported by all the managerial body throughout the organization and will still require a significant investment of time and resources. The outcomes of such transformation, however, will yield tangible and feasible benefits for the business, as stated in the Puppet’s 2016 report on the state of DevOps.

How did your transition to DevOps go? Did we miss some important phase of the process? Please share your experience, we are always open for discussion!