Caution. How to use process metrics without harming process health

Original Be Careful How You Use Metrics
By Stuart Rance | April 10, 2018 in General IT

Unfortunately, the original is no longer available: (but now there is a translation :).

It turned out quite free and closer to retelling, i.e. the personal experience of a translator could overshadow the opinion of the author and somewhere replace the original meaning, so be careful (here and below it’s the italics of the translator’s comments).
')
If an employee is given a goal with personal gain, his efforts to achieve it will be much more than efforts to work for the benefit of customers.

Metrics for Service Desk

Most of us working in IT understand the importance of using metrics to gain confidence that quality services are provided, and I have written many times in my blogs about how to define them, for example:

• Defining Problem Management Metrics
• Defining Metrics for Change Management
• Defining Metrics for Help Desk
• Defining Metrics for Service Desk
• Definition of metrics for Incident Management

Articles are not available now, as soon as it becomes possible to give working references to originals or translations, I will immediately add them.

Each of them explains the importance of first determining the critical success factors (CSFs), and then the key performance indicators (KPIs), which allow digitizing the achievement factors. Each blog also provided examples of suitable CSFs and KPIs for a specific process.

But even if you are sure that you have the right CSFs and KPIs in the right control points, this does not guarantee the correct performance of the work itself. This time I want to talk about the general approach in the application of critical factors (CFSs) and key indicators (KPIs), which allows you to focus on the quality of service delivery.

What makes KPI useful?

When determining KPIs it is important to remember the meaning of all 3 words:

Key. Every KPI should change something really important. Resist the urge to identify hundreds of KPIs - using a small number of them more efficiently. I recently saw the phrase clearly illustrating the reason: “For the same reason, why do not put 20 devices on the panels in the car”. It is necessary to have a small number of carefully selected indicators that help preserve the focus on the really most important things now.

Result (Performance). Each KPI must measure a significant result and must be explicitly projected onto one or more CFSs. It should be unambiguously clear what the increase or decrease in the value of a specific indicator means and what value is the norm for it.

Indicator. This is perhaps the most frequent thing that we understand and use is not true. KPI does not prove that the work is done well, it merely illustrates a possible assessment of the final result. You should not use KPI to prove to customers that the work is done well, customers are quite capable of making this assessment on their own.

I often hear that every KPI must be formulated using SMART:

• Specific: clear and unambiguous.
• Measuring (Mesurable): You can evaluate the current value or history of changes in the value of a metric for a specific indicator.
• Corresponds to the task (Relevant): the achievement of the required KPI values ​​is consistent with the supported CFS
• Limited in time (Time-based): indicators are tied to the time scale, in accordance with which values ​​are measured and reporting is provided.

This is really useful, but not a guarantee of the desired result. There are service providers who have a small number of KPIs defined by SMART and supporting important CFS, but this does not prevent them from doing the disgusting job.

How does this happen? Often this happens when the indicator is used in practice. People forget that KPI is just an indicator and I work more to achieve indicator values ​​than to provide quality services to the customer. This results in that all the required KPI values ​​have been reached, but the customer is still not satisfied, and leads to a return to the working relationship “from dissatisfaction” *, and if problems are not solved, the customer will start looking for alternative services on the side.

* One of the goals and “buns” often declared during the transition to service relationships is to move away from the priorities of problem solving based on the volume of customer perturbations to priorities based on the value provided to the customer, i.e. from negatively charged relationships claims to positively charged relationships based on benefits

Do not make the mistake of believing that it is sometimes written “sometimes” that means it will not happen to you. In fact, this happens quite often and there is even a common name for this phenomenon - “watermelon SLA”. Like a watermelon, outside the indicator values ​​look green, but if you cut it, you will see that it is all red inside. *

* It may be necessary to clarify that for clarity, the values ​​of the indicators are highlighted in colors: red - the value is worse than acceptable, yellow - while the value is better than acceptable, but dangerously close to it, green - value is better than acceptable with a margin.

If you want the required knowledge of the indicators to be achieved, inform the staff that the next wage rise depends on the KPI values. And the required values ​​will be achieved, but it’s far from a fact that the required overall result will suit you.

The most common example of an indicator for the Service Desk is the number of calls received by each employee for each day or the average duration of a telephone conversation. People will do everything to achieve the goal, even if it leads to the provision of services of unacceptable quality for the customer.

One of the obvious ways to eliminate such situations is to determine multidirectional balanced indicators. When it is known that work is evaluated on two or more conflicting requirements, target values ​​are determined and set for all of them. For example, you can set target values ​​for the average time to resolve calls and the number of overlapped hits. If the contractor reduces the average time due to the deterioration of the quality of his work, this will cause an increase in the number of blocked calls, i.e. This approach allows you to balance indicators.

* Not sure of the correctness of the translation of the term “tension KPI”, namely, as “multidirectional balanced indicators”, the translation options were also “stressed indicators” or “conflicting indicators”.

Multidirectional balanced indicators can help, but they may not be enough, because It is very difficult to predict fully the consequences of the influence of all indicators on each other in advance *. And people are smart and they will always find a way to achieve their goals.
* Rather, it’s not difficult to predict how the entire pool of indicators will affect employee performance.

So how do you use KPIs?

Please do not read further on this blog, then tell your friends that Stuart Raines advises not to use KPIs. I am absolutely convinced that indicators are used incorrectly if they become targets, on the basis of which the evaluation, reward or reward of people is made. But all this does not mean that you should not get the values ​​of indicators. It is just the opposite.

In my opinion, there are two main things for which you definitely need to use for KPIs:

Trend detection

Indicators are extremely useful for determining trends. If you systematically (regularly and monotonously) measure the values ​​of indicators and plot them on a graph with a time scale, you can see how they change over time. For example, you can see that incidents for a particular service are being addressed more and more, or that a new and Service Desk employee is gradually increasing the number of solutions to appeals every day.

This data can be used in analyzing the current state of services and planning improvements. And even as a justification of needs when planning budgets for staff expansion, or for improving the quality of the self-service portal, or for additional training.

Setting thresholds for taking action

If there is a clause in the SLA that incidents of a certain category will be resolved for no longer than 72 hours, then simply measuring and reporting on the results of solving the incident that has occurred, studying and analyzing what has happened from time to time or after not achieving the required target indicators, is not enough. It is better to embed triggers in the process of work, which, for example, after 48 hours will notify, only 24 hours left to solve. Ideally, this approach allows you to turn on in advance and influence the solution of the incident before the target value becomes unachievable.

I have seen the successful application of this approach on the example of quarterly achievement of goals. When service unavailability for a period exceeds 50% of the allowed value, the provider included the use of an additional set of measures that reduced the unavailability time for the remainder of the period. This kit included the recruitment of additional personnel available on the call, more stringent verification of changes made, etc. This approach allowed him to consistently achieve the target values ​​of availability while maintaining the price and minimal effort to provide quality service.

First of all people

Like everything that one way or another provides outstanding IT service management (ITSM), the proper use of metrics is primarily about people and their relationships. A great set of indicators (KPIs) can be a very powerful help in understanding what is happening and where improvements are needed, but the values ​​themselves of indicators should not become your goals, they should remain just indicators.

Here, for example, it may well be an example when measuring body temperature in treating a disease is used as an indicator of the current state, but the goal is not to achieve a normal value not higher than 37.0 and not less than 36.0, but a cure for the disease.

The described is not specific to ITSM difficulties. In economics, there is a very similar Goodhart law: “when an indicator becomes a goal, it ceases to be a good indicator.”

If you put effort into building excellent relationships with customers and your own employees, they will begin to tell all that they need to know in order to provide excellent services. You can then begin using key performance indicators (KPIs) for what they do best: help identify trends and alert when thresholds are reached to include the required corrective actions.

If you have read this far, then the article was interesting for you. Please take another minute and leave your opinion in the comments, it is very valuable for the translator, especially if you downgraded the publication. Thank :)