How Ivan metrics DevOps did. Start
Once Ivan was called to a meeting to discuss DevOps metrics.
Each participant prepared for the meeting a list of some metrics that in his opinion would be worth implementing.
Listening to the reports, Ivan tried to calculate how many metrics were suggested: 5.10, again 10, and about a dozen more. It turned out 30 with something.
')
For some reason, suddenly the thought came that the gathered people simply googled and discharged those whose names seemed interesting to them. The essence of the metrics, apparently, no one thought.
Observing from the side, Ivan asked himself questions: why? Why these metrics? What will they give you? It suddenly became obvious that the meeting brought together people who were completely far from the real understanding of the nature of metrics, and that everything would end as usual, with the loss of a huge amount of time and the throwing out of work in the trash.
It became sad and offensive. It's a shame that the time and money of the company just go nowhere, and it is sad that the useful thing will not be done.
Ivan has been studying metrics for a long time and realized a long time ago that the topic is very serious and complex, and it is impossible to approach it from the bay-groove.
On that day, the meeting ended with everything and nothing - they decided to implement everything at once (no one wanted to take responsibility for the refusal, because he did not understand why these metrics are needed by another person).
Ivan decided to prepare his own vision of DevOps metrics, and to make sure that each metric was justified in it, had a specific goal, was useful and was clear.
That's what happened to him ...
The first thing Ivan decided to think about the main goal for which metrics are made.
Read books "Lean Analytics" and "Practice Tao Toyota" suggested: as the main metric, choose the number you want to manage. Next, as the metrics, select the components of this number that you can influence.
The purpose of DevOps at the last meeting declared "software quality", but the concept of quality was very vague. What is quality? How to express it in one number? What components affect it?
Unfortunately, software quality cannot be expressed in one number.
Anyway, is quality really the goal of using DevOps?
A couple of years ago, Ivan worked as a CIO at a small company that produces its own software, and there he successfully launched and used DevOps. And the purpose of this DevOps really was not at all quality. Or rather, quality too. The main and only goal was the elimination of manual labor in the form of industrial deposits.
By removing the manual labor, he then achieved such acceleration and reduction of the number of errors, that it became possible to release the modifications at least once a minute.
Thus it turned out that as a TARGET METRIC, DevOps suggested itself
It is his decrease or increase would show the effect of management decisions.
Delivery time (Time To Market, TTM) increased - bad. Diminished - great.
But the delivery time depends on the volume of the task and on the duration of the test and on many other factors! Yeah, right.
And that is why Ivan consciously decided to start the countdown not from the moment of coding and analysis, but from the moment when the assembly (distribution) has already been created and is in the repository and all that is left is to conduct a series of checks and deploy it on an industrial environment (so-called Continious Delivery, CDL).
Here it is important to first develop a principle, a concept, Ivan thought, and to further extend it to other uncovered areas, because coding, assembly, and all other development steps in the same way affect the delivery time as the CDL stage. Let's do it in one way - do it in the other.
In a big company where Ivan worked, metrics were vital. Thousands of developers sawed the code and hundreds of builds were done every day, but no one, not a single person in the company, knew how much time teams actually spent on DevOps.
Complaints poured in from all directions: DevOps is garbage, it doesn’t work, it slowed down terribly, there’s no use to it. But no one had real numbers on their hands, and it was simply impossible to prove or refute the statements of the teams.
This is the goal that Ivan set for himself - to count the time that teams spend on DevOps, and in particular on the assembly delivery stage.
This can not be, Ivan thought, if I once launched DevOps and it gave an instant effect, then why is it not here?
Creating a system of metrics has become a matter of honor for Ivan.
How to manage delivery time? Is it possible? - Ivan wondered.
If we consider the delivery time as a number, it becomes clear that there are places in the DevOps process that significantly affect this number.
In Ivan’s company there was a standard: each assembly had to pass three test benches before entering the industry where various aspects of the software were tested.
Naturally, the assemblies on them "fell off" because of errors, and new ones were created instead.
It turned out that the stands are the key elements of the total delivery time, and it is they that affect the total time, increasing it.
Delivery time = Time on the stand 1 + Time on the stand 2 + Time on the stand 3
By affecting the time spent by the teams on each of the stands, it will be possible to influence the total delivery time.
There were two simple questions:
Ivan had no choice but to plunge into the jungle of Jenkins and Nexus (software used in the CDL).
"Rolling" assembly at the stand was carried out using Jenkins. In fact, Jenkins is an ordinary sheduler, like a crontab, but with advanced features.
Jenkins can produce logs of all running jobs (tasks for rolling out the assembly at the stand) in the form of RSS, and there is a start, end, and a link to a specific assembly.
It turned out that the data on the exact time of the beginning and end of the assembly work fell into Ivan’s disposal, i.e. it was possible without effort and to within a second to count the net time spent by the teams on the stands.
And if the data from all the stands to fall into one base, then it was possible to count the downtime between the stands. It was great!
From Nexus (network storage files), Ivan was going to get the date of creation of the assembly itself, and so on. determine the moment of its appearance and point of reference.
Everything fell into place. Nearly.
- How to manage this, Ivan thought? ..
Continued. Object of influence
Each participant prepared for the meeting a list of some metrics that in his opinion would be worth implementing.
Listening to the reports, Ivan tried to calculate how many metrics were suggested: 5.10, again 10, and about a dozen more. It turned out 30 with something.
')
For some reason, suddenly the thought came that the gathered people simply googled and discharged those whose names seemed interesting to them. The essence of the metrics, apparently, no one thought.
Observing from the side, Ivan asked himself questions: why? Why these metrics? What will they give you? It suddenly became obvious that the meeting brought together people who were completely far from the real understanding of the nature of metrics, and that everything would end as usual, with the loss of a huge amount of time and the throwing out of work in the trash.
It became sad and offensive. It's a shame that the time and money of the company just go nowhere, and it is sad that the useful thing will not be done.
Ivan has been studying metrics for a long time and realized a long time ago that the topic is very serious and complex, and it is impossible to approach it from the bay-groove.
On that day, the meeting ended with everything and nothing - they decided to implement everything at once (no one wanted to take responsibility for the refusal, because he did not understand why these metrics are needed by another person).
Ivan decided to prepare his own vision of DevOps metrics, and to make sure that each metric was justified in it, had a specific goal, was useful and was clear.
That's what happened to him ...
What do you want to manage?
The first thing Ivan decided to think about the main goal for which metrics are made.
Read books "Lean Analytics" and "Practice Tao Toyota" suggested: as the main metric, choose the number you want to manage. Next, as the metrics, select the components of this number that you can influence.
The purpose of DevOps at the last meeting declared "software quality", but the concept of quality was very vague. What is quality? How to express it in one number? What components affect it?
Unfortunately, software quality cannot be expressed in one number.
Anyway, is quality really the goal of using DevOps?
A couple of years ago, Ivan worked as a CIO at a small company that produces its own software, and there he successfully launched and used DevOps. And the purpose of this DevOps really was not at all quality. Or rather, quality too. The main and only goal was the elimination of manual labor in the form of industrial deposits.
By removing the manual labor, he then achieved such acceleration and reduction of the number of errors, that it became possible to release the modifications at least once a minute.
Thus it turned out that as a TARGET METRIC, DevOps suggested itself
Delivery time
It is his decrease or increase would show the effect of management decisions.
Delivery time (Time To Market, TTM) increased - bad. Diminished - great.
But the delivery time depends on the volume of the task and on the duration of the test and on many other factors! Yeah, right.
And that is why Ivan consciously decided to start the countdown not from the moment of coding and analysis, but from the moment when the assembly (distribution) has already been created and is in the repository and all that is left is to conduct a series of checks and deploy it on an industrial environment (so-called Continious Delivery, CDL).
Here it is important to first develop a principle, a concept, Ivan thought, and to further extend it to other uncovered areas, because coding, assembly, and all other development steps in the same way affect the delivery time as the CDL stage. Let's do it in one way - do it in the other.
In a big company where Ivan worked, metrics were vital. Thousands of developers sawed the code and hundreds of builds were done every day, but no one, not a single person in the company, knew how much time teams actually spent on DevOps.
Complaints poured in from all directions: DevOps is garbage, it doesn’t work, it slowed down terribly, there’s no use to it. But no one had real numbers on their hands, and it was simply impossible to prove or refute the statements of the teams.
This is the goal that Ivan set for himself - to count the time that teams spend on DevOps, and in particular on the assembly delivery stage.
This can not be, Ivan thought, if I once launched DevOps and it gave an instant effect, then why is it not here?
Creating a system of metrics has become a matter of honor for Ivan.
Manage what you can manage
How to manage delivery time? Is it possible? - Ivan wondered.
If we consider the delivery time as a number, it becomes clear that there are places in the DevOps process that significantly affect this number.
In Ivan’s company there was a standard: each assembly had to pass three test benches before entering the industry where various aspects of the software were tested.
Naturally, the assemblies on them "fell off" because of errors, and new ones were created instead.
It turned out that the stands are the key elements of the total delivery time, and it is they that affect the total time, increasing it.
Delivery time = Time on the stand 1 + Time on the stand 2 + Time on the stand 3
By affecting the time spent by the teams on each of the stands, it will be possible to influence the total delivery time.
There were two simple questions:
- how to physically calculate the cost of teams on the stands and
- How to deal with downtime between the stands (downtime is also a component of delivery time)?
Ivan had no choice but to plunge into the jungle of Jenkins and Nexus (software used in the CDL).
Jenkins and Nexus help
"Rolling" assembly at the stand was carried out using Jenkins. In fact, Jenkins is an ordinary sheduler, like a crontab, but with advanced features.
Jenkins can produce logs of all running jobs (tasks for rolling out the assembly at the stand) in the form of RSS, and there is a start, end, and a link to a specific assembly.
It turned out that the data on the exact time of the beginning and end of the assembly work fell into Ivan’s disposal, i.e. it was possible without effort and to within a second to count the net time spent by the teams on the stands.
And if the data from all the stands to fall into one base, then it was possible to count the downtime between the stands. It was great!
From Nexus (network storage files), Ivan was going to get the date of creation of the assembly itself, and so on. determine the moment of its appearance and point of reference.
Everything fell into place. Nearly.
- How to manage this, Ivan thought? ..
Continued. Object of influence
Source: https://habr.com/ru/post/433834/
All Articles