Geekly Articles each Day
📜 ⬆️ ⬇️

How and why we developed our tool for creating product distributions.



As the product develops and grows, various difficulties may arise. During the work on our MaxPatrol system, we faced the exhaustion of resources of the team responsible for supporting the project infrastructure. In particular, we needed to set up a deployment system — it was not all that easy. Today we will talk about exactly what the difficulties were and how we overcame them by developing our own tools for creating distributions.

A bit of history


In 2013, the infrastructure of our project looked as follows:
')

In the process of product development, it has become much larger. The number of its separate components increased, the number of installer packages increased in each of them, the complexity of each individual installer increased, and more sources of files became. It became necessary to create a special department of product infrastructure. Some numbers for the 2014-2015 year:


All this led to an increase in the labor costs of the infrastructure team - the system became more and more complicated, so when making changes it was necessary to spend time trying to figure out how to implement them correctly. As a result, the mean time to wait for changes has increased.

We spent a lot of time - every change needed to be discussed with the customer, we had to deal with bugs at the deployment stage, not all of which were related to the installer’s work, etc. At the same time, changes always have a high priority, which means they had to deal more with the raking of this “routine”, rather than the development of the project infrastructure.

As a result, the development of infrastructure aspects such as the delivery of updates, centralized deployment management and configuration, tools for creating installation packages slowed down a lot. It was necessary to do something about it.

Solution: division of areas of responsibility


To optimize the development, we decided to separate the areas of responsibility between the members of the infrastructure development and product development teams. To understand exactly how to do this, we conducted an analysis. He allowed us to break all change requests into several groups:


As a result, the distribution of various tasks has changed quite seriously - from the monopolistic “ownership” of these three classes of issues by the infrastructure team, we moved to a mixed scheme:



But it is not enough just to agree on the division of responsibility - you still need to find a way to technically implement it.

DSL to the rescue


Domain-specific language or DSL is a language that is suitable for use in working on a particular task. If we talk about our project, then with the help of DSL we were able to "agree" and got a tool that allowed all people involved in the development to solve their tasks without having to delve deeply into the details of the product (how to make changes to configuration files, etc. ) As a result, the work is greatly accelerated, and all entities can be freely expanded, which provides greater flexibility.

Here are the technologies we used at this stage:


The final scheme looked like this: we used the DSL and the script template to generate, in fact, the final script.

Using DSL (Yaml):



Description of the deployment scenario (Jinja2):



The resulting PowerShell deployment script is:



The creation of configuration files is worked out in a similar way: first, using DSL, the parameter values ​​are described, then a configuration file template is created - the output is a ready-made “config” with the necessary parameters.

We are also working on the generation of documentation - the DSL scheme is first developed, then an HTML document template is created, which allows you to get ready-made documentation in HTML on output.

Results and plans


One of the main achievements of the implementation was the fact that we finally got an adequate distribution of responsibilities between developers and project infrastructure specialists.



The second important effect: knowledge was also redistributed among the people involved in the product. In particular, developers have learned more about the deployment process — installer descriptions and configuration templates are located directly in their projects.

We also managed to significantly reduce the waiting time for changes. Previously, the process looked like this:



In this scheme, we came to a constant decline in performance due to the growth in the number of tasks. Partially, the problem could be solved by expanding the infrastructure department, but this method has quite obvious limits of applicability. And there will always be more tasks than people.

After the introduction of new approaches, the interaction scheme began to look like this:



As a result, the time for making changes is always the same and does not increase with the number of customers who need to solve a specific task.

We are not going to stop there and plan to develop our system. Here is what will be done in the near future:


PS The story about the tools we have developed for creating distribution kits was presented as part of a DevOps-mitap, which took place recently in Moscow.



The link presents presentations of 16 reports presented during the event. All presentations and video presentations will be added to the table at the end of this topic-announcement .

Author : Vladimir Selin

PPS We remind you that very soon, with the support of Positive Technologies, Moscow will take a course on asyncio + aiohttp from Core developer Python Andrei Svetlov.

We want to offer one free ticket to the seminar to the author of the best question for Andrei - the question he chooses himself and will answer it during the course. Send your questions to: asyncio2016@ptsecurity.com .

Source: https://habr.com/ru/post/315944/

More articles:


All Articles