Metrics in software development projects
Probably, I will not be too original, repeating that management implies measurement. This topic is particularly relevant for large companies.
This article does not claim to be absolute and absolute truth. I was recently just asked to humanly propose a list of metrics that need to be measured in projects, and think about which of them can be given to top management to assess the situation. The article can be considered as the first iteration of my sentences.
Any comments, additions and suggestions are welcome.
The first thing we need to ask ourselves when using metrics is why we do it. Obviously, then, in advance to assess the possible risks. Therefore, first you need to identify the main risk factors for your project. This can be done in various ways; I personally like three most of all: analyzing past similar projects, preparing the entire project team for a general meeting, at which everyone voices the risks that he sees in the project, and using standard lists of risks, as a rule, present in each project (industry-specific general or for your company).
Risk factors are not the risks themselves. These are the reasons behind the risk.
For your project (development of an information system for an external customer) you can determine the following main risk factors:
Not all of these factors can always be used as a basis for numerical metrics. However, to put a kind of "state of affairs indicators" in those risks that you cannot yet quantify, but you can evaluate qualitatively ("everything is still good / bad / terrible"), in my opinion, will not interfere.
In general, for a project, I consider it quite reasonable to start determining the following types of metrics:
In general, you can use the following approach to choosing metrics for a project:
Three types of metrics can be used to analyze the status of a project: metrics that work for proactive analysis, diagnostic metrics and retrospective metrics. The first we need to try to eliminate the trouble long before it happened. The second we need to see how things are going in the project. Still others are needed in order to learn from the history of their own victories and defeats.
So:
In order to understand what problems lie ahead, and what can happen in the end, you can prepare for the analysis several numbers:
Of course, all these numbers:
a) not taken from the ceiling. They are drawn on the basis of facts known to you (for example, a vacation schedule, average annual number of days spent at the hospital for each of your project team members, and other statistical data on its work).
b) regularly updated. And in general, they can be quite correct only after you have reached the end of the analysis phase.
Do you think all this is so trivial that it is not worth mentioning at all? Then those project managers who can answer at any time whether he has gone beyond the budget for unforeseen expenses on his project, let them tell how they do it. How are unforeseen budget expenditures calculated for the project portfolio under their management? Who is able to make resource plans with an accuracy of 10% - how does he do it?
Key metrics that allow you to assess what is happening:
Each of the listed quantities is measured on a regular basis. You can build graphs, and you can represent the numbers in a table form.
I think everyone understands why it is important to collect them. The distinction between diagnostic and retrospective metrics is that the former are used to analyze the situation “here and now” for the purpose of emergency response, while retrospective ones are used to assess past events and compare them with a bright future.
Perhaps that's all for today.
Does anyone else have their own thoughts on metrics related directly to the project management area?
This article does not claim to be absolute and absolute truth. I was recently just asked to humanly propose a list of metrics that need to be measured in projects, and think about which of them can be given to top management to assess the situation. The article can be considered as the first iteration of my sentences.
Any comments, additions and suggestions are welcome.
Main ideas
The first thing we need to ask ourselves when using metrics is why we do it. Obviously, then, in advance to assess the possible risks. Therefore, first you need to identify the main risk factors for your project. This can be done in various ways; I personally like three most of all: analyzing past similar projects, preparing the entire project team for a general meeting, at which everyone voices the risks that he sees in the project, and using standard lists of risks, as a rule, present in each project (industry-specific general or for your company).
Risk factors
Risk factors are not the risks themselves. These are the reasons behind the risk.
For your project (development of an information system for an external customer) you can determine the following main risk factors:
- Risks associated with staff turnover (in the industry, in the company)
- Time, budget and time constraints
- Limitations imposed by quality requirements
- Internal political factors (internal policy in the company of the customer and in your company, which may hinder the implementation of the project)
- External political factors (may be important when working with large government customers)
- Lack of technological expertise in the team
- Human factor: people avoiding responsibility, having negative experience in previous projects
- Low level of organizational maturity in the company
Not all of these factors can always be used as a basis for numerical metrics. However, to put a kind of "state of affairs indicators" in those risks that you cannot yet quantify, but you can evaluate qualitatively ("everything is still good / bad / terrible"), in my opinion, will not interfere.
Basic metric types
In general, for a project, I consider it quite reasonable to start determining the following types of metrics:
- Employee turnover rate
- Resource utilization rate
- Indicators related to the timing and budget of the project
- Indicators to assess the quality of the developed product
- Integral indicators of project progress
In general, you can use the following approach to choosing metrics for a project:
- Metrics of life cycle stages and schedule: Follow the work schedule for life cycle stages and compare actual and planned values.
- Project cost / value added metrics: Monitor the cumulative costs in comparison with the budget, as well as the total cost of the project, constantly updating the data as the project progresses.
- Requirement change tracking metrics: The number of changes in project-wide requirements.
Development process metrics: Monitor the number of requirements implemented in the model versus the total number of requirements in the project. - Failure type metrics: Track software failure causes.
- The remaining metrics for defects: A graphical representation of the number of failures per month by month throughout the duration of the project.
- Performance metric overview: Track phase error density and use charts to determine “peaks” and “dips” on a curve, as well as exceeding maximum permissible values.
Project Status Analysis
Three types of metrics can be used to analyze the status of a project: metrics that work for proactive analysis, diagnostic metrics and retrospective metrics. The first we need to try to eliminate the trouble long before it happened. The second we need to see how things are going in the project. Still others are needed in order to learn from the history of their own victories and defeats.
So:
Proactive analysis
In order to understand what problems lie ahead, and what can happen in the end, you can prepare for the analysis several numbers:
- Planned scope of work, budget , resource plan .
- Similar figures (scope of work, budget, resource plan) for other projects of the same profile - for comparison (strong variations should indicate problems)
- The complexity factor of a project is a quantity characterizing the total volume of all external artifacts of a project multiplied by the complexity coefficient determined for each of the external artifacts.
- The total risk associated with the schedule is a total value reflecting changes in the schedule in the project, taking into account the probability of their occurrence (expressed in man-hours). For example, you can estimate how much time your project team will spend at the hospital, and how much time will be on vacation. You can try to estimate how long the supplier will delay the delivery of equipment, which is always late with deliveries by at least three weeks.
- The total risk associated with the budget is the total value of all unforeseen expenses on a project, based on the budget plan for unforeseen expenses, taking into account the probability of their occurrence. For example, you can include compensation for unforeseen changes in labor costs, overhead costs, etc., arising during the work on the project.
- The complexity of the project plan is the number of interrelations and interdependencies between the various works in the schedule, referred to the total number of all works in the schedule. It affects the time reserve that needs to be laid in the project to take into account the shift in the timing of tasks that are unexpectedly “pushed” by the links. In addition, it is very difficult to add new people to a project with a large number of relationships, and this should be done (as a rule) not earlier than if the project plan can be simplified - for example, by reorganizing the business.
- Density of the project - the ratio of the total duration of all work in the schedule when they are executed in succession to the sum of its own and the total reserve (reserve) of time for all planned works. The higher the density, the more difficult the project will be to complete. Do not just perform on time - it will be harder to complete the project.
- Project independence - the relationship between the number of dependencies for internal project work and the total number of dependencies, including dependencies on external work and suppliers. I think everything is obvious here: the more independent the project, the greater the chance of success. It is a pity that such projects in the IT-industry in the past 10 years, almost never occur.
- Human resources : the maximum number of participants in the project (time to think: do we even have the necessary number of people? Are there any “spare” ones for an emergency?)
- Total time margin : total reserve (reserve) of time for all planned works.
- Evaluating SLA requirements or quality requirements that meet the minimum requirements for the product, system or service being developed.
Of course, all these numbers:
a) not taken from the ceiling. They are drawn on the basis of facts known to you (for example, a vacation schedule, average annual number of days spent at the hospital for each of your project team members, and other statistical data on its work).
b) regularly updated. And in general, they can be quite correct only after you have reached the end of the analysis phase.
Do you think all this is so trivial that it is not worth mentioning at all? Then those project managers who can answer at any time whether he has gone beyond the budget for unforeseen expenses on his project, let them tell how they do it. How are unforeseen budget expenditures calculated for the project portfolio under their management? Who is able to make resource plans with an accuracy of 10% - how does he do it?
Diagnostic metrics
Key metrics that allow you to assess what is happening:
- The budget of the work performed ( planned and actual ): a constantly changing value (with a cumulative total) of the total cost of all the planned work on the project completed so far.
- The ratio of the actual budget to the planned budget : if this ratio does not exceed 1, then the project fits into the budget in the medium term; if not, then, most likely, the budget will be overspent.
- Dispersion of costs : the difference between the planned budget of work that should have been completed at the current time according to the schedule, and the actual budget, allowing to estimate the amount of overspending or unspent budget in the project.
- Execution of the schedule : the ratio of the actual labor costs for the execution of completed works to the planned labor costs for the execution of these works (planned labor intensity). This parameter is used to assess the risk of non-compliance with the schedule.
- Schedule variance : the difference between the actual labor costs for the work performed and the planned labor costs. This is an absolute expression of the same parameter, which is represented as a coefficient.
- Schedule lag : the total length of the delay time for tasks that are not met in the schedule.
- Ratio of closed work: the ratio of the number of completed (closed) works to the total number of works scheduled for completion at the moment.
- Labor productivity , calculated as the ratio of the volume of work to elapsed time (deviation from planned values)
- The actual values ​​of the results of the verification of SLA requirements or quality requirements for the software product or service that you are developing. About SLA, I'll ever tell you more.
Each of the listed quantities is measured on a regular basis. You can build graphs, and you can represent the numbers in a table form.
Retrospective metrics
I think everyone understands why it is important to collect them. The distinction between diagnostic and retrospective metrics is that the former are used to analyze the situation “here and now” for the purpose of emergency response, while retrospective ones are used to assess past events and compare them with a bright future.
- Reliability ratio : the ratio of the product of the project cost to its duration to the cumulative value of the increasing planning error.
- The ratio of the actual labor productivity to the planned one , calculated according to the actual and planned schedule: the difference is calculated as a percentage of the planned schedule.
- Percentage of labor involved in the project phase / separate group of tasks
- The ratio of the amount of work in testing to the total amount of work
Perhaps that's all for today.
Does anyone else have their own thoughts on metrics related directly to the project management area?
')
Source: https://habr.com/ru/post/148654/
All Articles