DevOps metrics - where to get data for calculations
To be honest, Ivan often laughed at the vain efforts of colleagues from the monitoring department. They made great efforts to implement the metrics that they ordered the company management. They were so busy that they didn’t want to do anything else.
But the leadership was not enough - it was constantly ordering more and more new metrics, very quickly ceasing to use what had been done earlier.
Recently, everyone just talked about LeadTime - the delivery time of business features. The metric showed a crazy number - 200 days for the delivery of one task. How everyone sighed, gasped and raised their hands to heaven!
')
After some time, the noise gradually subsided, and the management received an order to create another metric.
Ivan was completely clear that the new metric would also die in a dark corner just the same.
Indeed, Ivan pondered, knowing the number absolutely does not tell anyone about anything. 200 days or 2 days - there is no difference, because by the number it is impossible to determine the cause and understand whether it is good or bad.
This is a typical metric trap: it seems that the new metric will tell the essence of being and explain some secret secret. Everyone hopes so, but for some reason nothing happens. Yes, because the secret must be sought not at all in the metrics!
For Ivan, it was a past stage. He understood that metrics are just a simple wooden ruler for measurements, and all secrets must be sought in the object of influence , i.e. in the fact that this metric forms.
For an online store, the object of influence will be its customers, who bring in money, and for DevOps, the teams that create and roll out distributions using the pipeline.
Once, having settled down in the hall in a comfortable chair, Ivan decided how to think through how he would like to see the DevOps metrics, given that the object of influence is the commands.
It is clear that everyone wants to reduce the delivery time. 200 days is, of course, no good.
But how, that is the question?
The company employs hundreds of teams, and thousands of distributions pass through the DevOps pipeline daily. Real time delivery will look like distribution. Each team will have its own time and its own features. How can you find anything in this mess?
The answer arose by itself - it is necessary to find problem teams and figure out what is happening with them and why for so long, and learn from “good” teams how to do everything quickly. And for this you need to measure the time spent by the teams at each of the DevOps stands:
“The goal of the system will be the selection of teams according to the time of the stands, in the end, we should get a list of commands with the selected time, and not a number.
If we find out how much time is spent on the stand in total and how much time is spent on the idle time between the stands, we will be able to find commands, call them and find out more about the causes and eliminate them, ”Ivan thought.
To count, it was necessary to delve into the process of DevOps and its essence.
The company uses a limited number of systems, and information can only be obtained from them and nowhere else.
All tasks in the company were registered in Jira. When the task was taken into work, a brunch was created for it, and after the implementation a commit was made in BitBucket and Pull Request. When a PR (Pull Request) was accepted, a distribution was automatically created and saved to the Nexus repository.
Further, the distribution kit was rolled out on several stands with the help of Jenkins to check the correctness of knurling, automatic and manual testing:
Ivan painted from which systems what information can be taken to calculate the time at the stands:
On the basis of the information available, the following scheme was drawn:
Knowing how much time distributions are created and how much time is spent on each of them, you can easily calculate the total cost of passing the entire DevOps pipeline (full cycle).
Here are some DevOps metrics that Ivan turned out as a result:
On the one hand, metrics very well characterized the DevOps pipeline in terms of time, on the other hand, they were considered very simple.
Satisfied with a job well done, Ivan made a presentation and went to present it to the management.
He went back frowning and with his hands down.
- This is a fiasco, bro - the ironic colleague smiled ...
Continued in the article " How fast results helped Ivan ."
But the leadership was not enough - it was constantly ordering more and more new metrics, very quickly ceasing to use what had been done earlier.
Recently, everyone just talked about LeadTime - the delivery time of business features. The metric showed a crazy number - 200 days for the delivery of one task. How everyone sighed, gasped and raised their hands to heaven!
')
After some time, the noise gradually subsided, and the management received an order to create another metric.
Ivan was completely clear that the new metric would also die in a dark corner just the same.
Indeed, Ivan pondered, knowing the number absolutely does not tell anyone about anything. 200 days or 2 days - there is no difference, because by the number it is impossible to determine the cause and understand whether it is good or bad.
This is a typical metric trap: it seems that the new metric will tell the essence of being and explain some secret secret. Everyone hopes so, but for some reason nothing happens. Yes, because the secret must be sought not at all in the metrics!
For Ivan, it was a past stage. He understood that metrics are just a simple wooden ruler for measurements, and all secrets must be sought in the object of influence , i.e. in the fact that this metric forms.
For an online store, the object of influence will be its customers, who bring in money, and for DevOps, the teams that create and roll out distributions using the pipeline.
Once, having settled down in the hall in a comfortable chair, Ivan decided how to think through how he would like to see the DevOps metrics, given that the object of influence is the commands.
DevOps metrics goal
It is clear that everyone wants to reduce the delivery time. 200 days is, of course, no good.
But how, that is the question?
The company employs hundreds of teams, and thousands of distributions pass through the DevOps pipeline daily. Real time delivery will look like distribution. Each team will have its own time and its own features. How can you find anything in this mess?
The answer arose by itself - it is necessary to find problem teams and figure out what is happening with them and why for so long, and learn from “good” teams how to do everything quickly. And for this you need to measure the time spent by the teams at each of the DevOps stands:
“The goal of the system will be the selection of teams according to the time of the stands, in the end, we should get a list of commands with the selected time, and not a number.
If we find out how much time is spent on the stand in total and how much time is spent on the idle time between the stands, we will be able to find commands, call them and find out more about the causes and eliminate them, ”Ivan thought.
How to calculate the delivery time for DevOps
To count, it was necessary to delve into the process of DevOps and its essence.
The company uses a limited number of systems, and information can only be obtained from them and nowhere else.
All tasks in the company were registered in Jira. When the task was taken into work, a brunch was created for it, and after the implementation a commit was made in BitBucket and Pull Request. When a PR (Pull Request) was accepted, a distribution was automatically created and saved to the Nexus repository.
Further, the distribution kit was rolled out on several stands with the help of Jenkins to check the correctness of knurling, automatic and manual testing:
Ivan painted from which systems what information can be taken to calculate the time at the stands:
- From Nexus - The time when the distribution was created and the name of the folder containing the command code
- From Jenkins - Start time, duration and result of each job, the name of the stand (in the parameters of the job), stages (steps of the job), link to the distribution in Nexus.
- Jira and BitBucket Ivan decided not to include in the pipeline, because they were more related to the development stage, and not to rolling the finished distribution kit on the stands.
On the basis of the information available, the following scheme was drawn:
Knowing how much time distributions are created and how much time is spent on each of them, you can easily calculate the total cost of passing the entire DevOps pipeline (full cycle).
Here are some DevOps metrics that Ivan turned out as a result:
- The number of created distributions
- The share of distributions that have “visited” the booth and “passed” the booth
- Time spent on the stand (stand cycle)
- Full cycle (total time for all stands)
- Jobs duration
- Idle time between stands
- Easy between running jobs on one stand
On the one hand, metrics very well characterized the DevOps pipeline in terms of time, on the other hand, they were considered very simple.
Satisfied with a job well done, Ivan made a presentation and went to present it to the management.
He went back frowning and with his hands down.
- This is a fiasco, bro - the ironic colleague smiled ...
Continued in the article " How fast results helped Ivan ."
Source: https://habr.com/ru/post/459344/
All Articles