Insight on metrics: how do I understand what metrics are and what is their main charm
Metrics are bullshit, you say, and you will be right. In something.
Indeed, when it comes to metrics, the very first first metric that comes to mind is attendance.
Many people like to meditate for hours looking at the schedule of attendance of their site.
')
How cool it is to observe how the line jumps - here and there, here and there ... And even cooler when site traffic grows continuously.
Then blissful heat spreads all over the body and the mind soars to the heavens in anticipation of the manna of heaven.
Oh, what a joy, what a bliss!
And even if the picture is sad ...
From the graph, still do not take your eyes off, so it delays.
It seems that the graph is hidden secret meaning. A little more, and the picture will reveal its secrets and tell you an incredibly simple and effective way to attract a huge number of customers. And then money will flow like a river.
But in fact, attendance is a typical “sugary (vain) metric” that does not carry useful meaning.
And most of these metrics. Basically, all the metrics you see are sugary. And it is precisely for this reason that the fame of a meaningless waste of time and effort has developed behind the metrics.
But actually this is not the case at all. Correct metrics provide extremely important, and sometimes invaluable information for business and the project.
The main bonus and the appointment of metrics is that they enable you to manage your business or project.
Let's look at a very simple example - the speed of a car.
Please tell me what speed means ...
Hmm ...
Hmm ...
So what does it mean?
I think you probably yourself guessed that ... does not mean anything!
OK. Now the second question:
100 km / h is it good or bad?
Hmm ...
Neither one nor the other?
Right!
Speed ​​is a completely useless and stupid metric. Unless, of course, use it by itself. Coupled with other metrics, she, of course, can say something, but by itself - definitely not.
Site traffic is exactly the same speed.
That is why to hang in front of the schedule site traffic there is absolutely no point. He will not reveal to you the secret of life. Do you understand now?
For example, Churn rate. This metric tells how many customers left the company / site forever over time.
Churn rate = 1% says that we are losing only 1% of customers. Those. almost no one lose.
If Churn rate = 90%, then this means that we lose almost all of our customers. It's horrible!
Do you see the difference between this metric and speed?
Churn rate is a meaningful metric that answers the question whether this is good or bad. And you do not need to guess what it means.
This is a metric that speaks for itself!
And now we are ready to take urgent action to reduce customer churn.
That is why such metrics are called effective. Because they encourage action.
There is a very simple way to determine that the metric is “sugary / vanity” (eng. Vanity).
Most absolute metrics , such as attendance, number of downloads, number of retweets, number of emails / subscribers, number of likes, etc. are sugary .
Relative, weighted, metrics are often effective. But not all!
As for the quality metrics, there is no unambiguity, because Qualitative assessment alone cannot be exact and unambiguous.
But on the other hand, it is possible and necessary to evaluate the convenience of the program precisely by the level of perception of the end users and nothing else.
The first thing you need to turn the brain.
No kidding.
Everyone (!) Who is faced with metrics, first of all begins to look into them for the reason for being. But they will not show it, unfortunately.
Metrics are just like a regular ruler with which we measure everything we want.
You are not looking for the reason for being in a normal wooden ruler, right?
The search for the meaning of life in the ruler is what is called the “bottom-up approach.”
To work properly with metrics, you need to change the paradigm and start working the other way around, from top to bottom.
Those. first, do some kind of action, and then measure the effect from it using metrics.
Metrics should be used as a common measurement item and no more.
Think about these words.
Measure with the help of metrics the effect of your actions, and not invent actions based on the readings of a wooden ruler.
This approach is also called “Hypothesis-> Measurement”.
Ok, this is understandable.
Question number 2: “What exactly is measured? How to find the right metrics? ”
Climbing on the Internet you will surely find dozens, or even hundreds of different metrics on the same topic.
For example, software quality metrics can be found around a hundred. These are GOSTR-ISO standards, and metrics calculated in SonarQube, and some self-written variants, and even “high-quality” metrics based on user feedback.
So which ones should be used and which ones should not?
The best approach is to be guided by “core value”.
Let's take an example.
It is clear that if you want to improve the quality of your software, you can measure this quality in different ways.
Quality is not only the number of mistakes. If you look at the quality in general, it is:
the number of incidents in prom
usability and ease of perception,
speed of work
completeness and timeliness of implementation of the planned functionality,
security.
Criteria a lot and work immediately with all impossible. They act very simply: they choose one, the most important criterion at the current moment, and work only with it.
This approach is called OMTM (One Metric That Matters) - One (Single) Important Metric.
OMTM software quality is logical to choose the number of serious (important and critical) incidents in the industrial environment.
For online stores, you don’t need to think about OMTM at all - it is the volume of sales or profit (depending on your decision).
This One Important Metric will be the core value for your set of metrics. And their final set will depend on it.
To make a set of metrics often begin “from the bald,” rummaging through the Internet and choosing from the found the best options on the principle: “Oh! It will suit us! ”
As you know, this is not the best way, right?
But how to decide which metric to take and which not?
For example, they often measure various kinds of user conversions.
But why measure precisely the users, and not something else? Have you thought about this issue?
Naturally, the answer is.
Let's consider the online store as the easiest to understand example.
Let's say you want to increase sales. What metrics do you need for this? How to approach this?
There is one simple, logical and working way. Everything falls into place when you answer the question:
WHO DOES A VALUE?
We're working on sales, right? We want to increase it, right?
Who and what should be affected in order to increase sales?
Of course,
need to influence the cause -
on the one who “produces” the value.
Who makes money in the online store? Where does the money come from?
Very simple: from customers.
Where exactly in the online store can you influence customers?
Yes, anywhere!
Right. At each stage of the customer's life cycle.
To represent the life cycle it is convenient to build a so-called. “Funnel” of the client’s movement through the process.
Example of an online store funnel:
Why so? Because customers are lost precisely when moving from one funnel step to another.
Increasing the number of customers at any funnel level, we automatically increase the resulting sales volume.
A simple example.
The metric “Percentage of abandoned baskets” essentially shows the conversion when moving from the commodity basket to the completed order.
Suppose, during the first measurement, you found that 90% of the baskets are lost, i.e. out of 10 baskets only 1 order is made.
There is clearly something wrong with the product basket, right?
For simplicity, we assume that the amount of one order is 100 rubles. So total sales will be only 100 rubles.
As a result of finishing the basket, the percentage of abandoned baskets decreased by 10%, to 80%. How does it look in numbers?
From 10 baskets began to make out 2 orders. 100 rubles * 2 = 200 rubles
But this is an increase in sales by 100%! Bingo!
Increasing the conversion step by only 10%. You increased sales by 100%.
Fantasy!
But this is how it works.
Do you understand now what is the beauty of properly constructed metrics?
With their help, you can achieve a fantastic influence on your processes.
With an online store, everything is quite simple, but how does this shift everything, for example, to the quality of a software product? Yes, just as well:
Here, for example, what quality metrics could have turned out (on a vskidku) ...
Value Indicator:
Metrics based on the life cycle of the source code:
Metrics based on the life cycle of defects:
Results
As you can see, the topic of metrics is really very important, necessary and interesting.
How to choose the right metrics:
Choose OMTM, think over its core value and measure the producers of this value.
Build metrics based on the manufacturer’s life cycle funnel.
Try not to use absolute metrics.
The topic of metrics has become popular in the wake of the Lean Startup movement, so it’s best to start reading from primary sources - the books “Lean Startup” (translated into Russian - “Business from scratch. Lean Startup Method” on Ozon) and “Lean Analytics” (no translation, but book in English is sold on Ozon).
Some information can be found on the Internet even in Russian, but a comprehensive textbook, unfortunately, has not yet been found even in the western segment.
By the way, now there are even individual specialists “productologists” whose task is to build the right metrics system for their product and to suggest ways to improve them.
That's all.
If the article helped you to better understand the essence of the issue, the author would be grateful for the “like” and repost.
Indeed, when it comes to metrics, the very first first metric that comes to mind is attendance.
Many people like to meditate for hours looking at the schedule of attendance of their site.
')
How cool it is to observe how the line jumps - here and there, here and there ... And even cooler when site traffic grows continuously.
Then blissful heat spreads all over the body and the mind soars to the heavens in anticipation of the manna of heaven.
Oh, what a joy, what a bliss!
And even if the picture is sad ...
From the graph, still do not take your eyes off, so it delays.
It seems that the graph is hidden secret meaning. A little more, and the picture will reveal its secrets and tell you an incredibly simple and effective way to attract a huge number of customers. And then money will flow like a river.
But in fact, attendance is a typical “sugary (vain) metric” that does not carry useful meaning.
And most of these metrics. Basically, all the metrics you see are sugary. And it is precisely for this reason that the fame of a meaningless waste of time and effort has developed behind the metrics.
But actually this is not the case at all. Correct metrics provide extremely important, and sometimes invaluable information for business and the project.
The main bonus and the appointment of metrics is that they enable you to manage your business or project.
How to determine that the metric is bad?
Let's look at a very simple example - the speed of a car.
Please tell me what speed means ...
100 km / h?
Hmm ...
Hmm ...
So what does it mean?
I think you probably yourself guessed that ... does not mean anything!
OK. Now the second question:
100 km / h is it good or bad?
Hmm ...
Neither one nor the other?
Right!
Speed ​​is a completely useless and stupid metric. Unless, of course, use it by itself. Coupled with other metrics, she, of course, can say something, but by itself - definitely not.
Site traffic is exactly the same speed.
That is why to hang in front of the schedule site traffic there is absolutely no point. He will not reveal to you the secret of life. Do you understand now?
And which metrics are good then?
For example, Churn rate. This metric tells how many customers left the company / site forever over time.
Churn rate = 1% says that we are losing only 1% of customers. Those. almost no one lose.
If Churn rate = 90%, then this means that we lose almost all of our customers. It's horrible!
Do you see the difference between this metric and speed?
Churn rate is a meaningful metric that answers the question whether this is good or bad. And you do not need to guess what it means.
This is a metric that speaks for itself!
And now we are ready to take urgent action to reduce customer churn.
That is why such metrics are called effective. Because they encourage action.
The criterion of "sweetness" metrics
There is a very simple way to determine that the metric is “sugary / vanity” (eng. Vanity).
Most absolute metrics , such as attendance, number of downloads, number of retweets, number of emails / subscribers, number of likes, etc. are sugary .
Relative, weighted, metrics are often effective. But not all!
As for the quality metrics, there is no unambiguity, because Qualitative assessment alone cannot be exact and unambiguous.
But on the other hand, it is possible and necessary to evaluate the convenience of the program precisely by the level of perception of the end users and nothing else.
How to approach the metrics in general?
The first thing you need to turn the brain.
No kidding.
Everyone (!) Who is faced with metrics, first of all begins to look into them for the reason for being. But they will not show it, unfortunately.
Metrics are just like a regular ruler with which we measure everything we want.
You are not looking for the reason for being in a normal wooden ruler, right?
The search for the meaning of life in the ruler is what is called the “bottom-up approach.”
To work properly with metrics, you need to change the paradigm and start working the other way around, from top to bottom.
Those. first, do some kind of action, and then measure the effect from it using metrics.
Metrics should be used as a common measurement item and no more.
Think about these words.
Measure with the help of metrics the effect of your actions, and not invent actions based on the readings of a wooden ruler.
This approach is also called “Hypothesis-> Measurement”.
Ok, this is understandable.
Question number 2: “What exactly is measured? How to find the right metrics? ”
How to create your own set of metrics?
Climbing on the Internet you will surely find dozens, or even hundreds of different metrics on the same topic.
For example, software quality metrics can be found around a hundred. These are GOSTR-ISO standards, and metrics calculated in SonarQube, and some self-written variants, and even “high-quality” metrics based on user feedback.
So which ones should be used and which ones should not?
The best approach is to be guided by “core value”.
OMTM (One Metric That Matters)
Let's take an example.
It is clear that if you want to improve the quality of your software, you can measure this quality in different ways.
Quality is not only the number of mistakes. If you look at the quality in general, it is:
the number of incidents in prom
usability and ease of perception,
speed of work
completeness and timeliness of implementation of the planned functionality,
security.
Criteria a lot and work immediately with all impossible. They act very simply: they choose one, the most important criterion at the current moment, and work only with it.
This approach is called OMTM (One Metric That Matters) - One (Single) Important Metric.
OMTM software quality is logical to choose the number of serious (important and critical) incidents in the industrial environment.
For online stores, you don’t need to think about OMTM at all - it is the volume of sales or profit (depending on your decision).
This One Important Metric will be the core value for your set of metrics. And their final set will depend on it.
Inside value
To make a set of metrics often begin “from the bald,” rummaging through the Internet and choosing from the found the best options on the principle: “Oh! It will suit us! ”
As you know, this is not the best way, right?
But how to decide which metric to take and which not?
For example, they often measure various kinds of user conversions.
But why measure precisely the users, and not something else? Have you thought about this issue?
Naturally, the answer is.
Let's consider the online store as the easiest to understand example.
Let's say you want to increase sales. What metrics do you need for this? How to approach this?
There is one simple, logical and working way. Everything falls into place when you answer the question:
WHO DOES A VALUE?
We're working on sales, right? We want to increase it, right?
Who and what should be affected in order to increase sales?
Of course,
need to influence the cause -
on the one who “produces” the value.
Who makes money in the online store? Where does the money come from?
Very simple: from customers.
Where exactly in the online store can you influence customers?
Yes, anywhere!
Right. At each stage of the customer's life cycle.
To represent the life cycle it is convenient to build a so-called. “Funnel” of the client’s movement through the process.
Example of an online store funnel:
Why so? Because customers are lost precisely when moving from one funnel step to another.
Increasing the number of customers at any funnel level, we automatically increase the resulting sales volume.
A simple example.
The metric “Percentage of abandoned baskets” essentially shows the conversion when moving from the commodity basket to the completed order.
Suppose, during the first measurement, you found that 90% of the baskets are lost, i.e. out of 10 baskets only 1 order is made.
There is clearly something wrong with the product basket, right?
For simplicity, we assume that the amount of one order is 100 rubles. So total sales will be only 100 rubles.
As a result of finishing the basket, the percentage of abandoned baskets decreased by 10%, to 80%. How does it look in numbers?
From 10 baskets began to make out 2 orders. 100 rubles * 2 = 200 rubles
But this is an increase in sales by 100%! Bingo!
Increasing the conversion step by only 10%. You increased sales by 100%.
Fantasy!
But this is how it works.
Do you understand now what is the beauty of properly constructed metrics?
With their help, you can achieve a fantastic influence on your processes.
With an online store, everything is quite simple, but how does this shift everything, for example, to the quality of a software product? Yes, just as well:
- We choose the main value we are working on. For example, we reduce the number of incidents in prom.
- We understand who and what produces this value. For example, the source code.
- Build the source code life cycle funnel and set up metrics at each funnel step. Everything.
Here, for example, what quality metrics could have turned out (on a vskidku) ...
Value Indicator:
- defect density prom on 1000 lines of code
Metrics based on the life cycle of the source code:
- proportion of failed compilations
- autotest coverage
- share of failed autotests,
- share of failed deployments
Metrics based on the life cycle of defects:
- defect detection dynamics
- fix dynamics,
- re-discovery dynamics
- dynamics of deviations of defects,
- average waiting time for fixes
- average correction time.
Results
As you can see, the topic of metrics is really very important, necessary and interesting.
How to choose the right metrics:
Choose OMTM, think over its core value and measure the producers of this value.
Build metrics based on the manufacturer’s life cycle funnel.
Try not to use absolute metrics.
What else to read on this topic
The topic of metrics has become popular in the wake of the Lean Startup movement, so it’s best to start reading from primary sources - the books “Lean Startup” (translated into Russian - “Business from scratch. Lean Startup Method” on Ozon) and “Lean Analytics” (no translation, but book in English is sold on Ozon).
Some information can be found on the Internet even in Russian, but a comprehensive textbook, unfortunately, has not yet been found even in the western segment.
By the way, now there are even individual specialists “productologists” whose task is to build the right metrics system for their product and to suggest ways to improve them.
That's all.
If the article helped you to better understand the essence of the issue, the author would be grateful for the “like” and repost.
Source: https://habr.com/ru/post/342330/
All Articles