Testing Effort Planning - Report from SQA Days 15
Presentation by Alexandra Kovaleva at the SQA Days-15 conference.
Presentation
')
I do risk assessments in the company. Over the last period, while working as a test lead, I have been engaged in global planning. However, not all companies plan a lot of time. We'll just talk about what to do about it.
Master class will consist of 2 parts:
1. Theoretical - the concept of evaluation and planning. Roles and tasks.
2. Practical - an example of resource management ( can be seen on the video )
So what is planning? This is such a global concept that it can be applied to any area of ​​life.
As an example, when you go to work in the morning and think about going to the cinema in the evening, you are planning! Also at work! Despite the fact that you are not asked about planning, you plan, as you should know about when you can finish your work and pass on the results to the team. In other cases, you precisely name the release date, having planned its dates, which are often postponed due to the fact that you do not fall on the dates indicated in the plan.
Planning is a very broad concept. And before we talk about planning at the testing stage, you need to know what this word means for other members of the company and especially for management.
But the team is involved in planning, but the team doesn’t need it, they don’t want to do it. Such a problem arises due to management. The manager constantly communicates with the customer who wants to know the time. But this is not the time spent on development, but the date when it will receive the finished product, and how much time it takes.
In this case, money and time are very interrelated, especially if the deadlines for product release are violated. Maybe he does not require reporting from you, but he is associated with other specialists: marketers (advertising company), financiers (budget allocation). The last link in the release of the product - testing, so they are trying to save on it.
We’ll talk about planning in forecasting methodologies, because they have time to plan before the project starts. In flexible methodologies, everything happens “here and now”: not only regular planning, but also initial planning, as a rule, management deals with them. This is a separate topic.
If we are talking about predictive methodologies, then there may be entire stages that you can observe on the slide. Here, for example, the first stage - Incention - it can take up to six months. And this is a period when they do not program and do not really deal with requirements. Who can agree on when we get paid, and we really do nothing? This may be the military, medical industry. These can be global projects on which people's lives depend, and the development period is approximately from 2 to 5-7 years. Usually, planning takes about 2 weeks, but often it's just “tomorrow”.
What is test planning ? For a test lead, this is not only time, but what, how and where to test.
Test Planning :
1. Determine test requirements
2. Risk assessment
3. Develop a testing strategy
4. Definition of resources
5. Developing a Test Plan
6. Creating a work schedule
Each issue is worth considering. But we will talk about a small paragraph "Creating a schedule of works . " Of course, you can just say about testing, but it is better to show and somehow formalize your argument.
Now let's talk about this. Of course, planning is not possible without evaluating the results - this is our main problem.
"We do not know how to evaluate!" - The team can tell you that the junior, that an experienced specialist. The reason is seen in the absence of certain methods. This is a very big misconception. Methods are:
Requiring a detailed mathematical study:
1. Delphi method;
2. The method of three points (sequential calculation);
3. Method of analysis of functional points / testing points;
4. The method of assessing points of use cases;
5. COCOMO (COnstructive COst MOdel) - cost model;
6. Genetic evaluation model;
These are mathematical methods, but I have not seen a single company that would use them. Their main disadvantages are :
1. They take a lot of time
2. They are labor intensive
3. They do not give a result.
Why?
Because planning must be flexible . Because a difficultly calculated plan can be broken only due to the fact that someone did not come or fell ill. Recalculate? Hardly!
Therefore, we will talk about the methods that you already use, but are not always aware of!
The most simple to use:
1. PVN (finger to the sky), or trial and error;
2. Special method;
3. Analogies and recommendations of experts;
4. Structure of work decomposition;
5. The percentage of development;
6. The method of percentage distribution;
Our favorite is trial and error . We all use it often. You need to understand how it differs from the method of expert assessments .
If you have already worked with the project, he is familiar to you and tested something, then you already do as an expert. If you didn’t do this for testing, didn’t work with the project and never came across this area or customer, then you fully appreciate your finger in the sky. This is a big difference.
My favorite method is the structure of work decomposition . This is the least time consuming but formalized method for evaluation. What is meant? You just need to split up your work to the minimum, which is very easy to evaluate and even do not need to learn.
For example, test authorization. The letter "o" does not pass - you can immediately say 5 minutes for yourself. This is the minimum unit. This can be determined even by a person who works only a couple of months. Plus in this method - it's harder for you to forget something.
You take your requirements, and a big task, you divide them into 3 parts, for example: field testing, button testing, localization testing, performance testing and testing by type, method, area. Crush further: by entering numbers, letters, symbols, etc.
Pay attention to the chart, here are statistics for sites from six months - a year. Programming takes from 20% to 40% of the development, it’s not the same as 20–40% of the project, it’s an average of 15% of the project. Testing never takes 15% of the product. If you do not lay so much time for testing, then lay at least some. It is advisable to find out statistically what percentage of the project is testing and this is applicable if you have stable versions of releases, the constant volume of products is the same.
Solution to the problem
1. We train beginners:
a. Timekeeping;
b. Analysis
2. Create a universal Estimation Check List for a project portfolio;
3. We do not criticize for errors in the estimates;
To do this, you do not need to go to trainings, somehow evaluate, write down your grades and see how much time it really takes. Different trailers / apps that help record time help. Or about days to memorize. When you understand yourself, you can teach your team. It is very important that when you try to persuade a team to evaluate, it is very important not to scold it for these ratings. Most often, people do not want to evaluate, because they are afraid that you can find fault with them. Until everything is fixed, they do not owe you anything.
Explain to your employees that any incorrect assessment is better than no one. Otherwise, you, as a test lead, know nothing, cannot relate resources, and so on. Praise if they even made a mistake. Say: “You made a mistake 2 times. and I was mistaken in 3! ". But do not miss the mistakes, sit down and compare how much was planned, how long it took in reality, where there was big fail, and what took the most time. The most important thing is this analysis, when the person himself realizes that he did not take into account. This is where the work decomposition is needed in order not to forget anything.
Here we can be helped - “Forget-me-not for the tester” .
1. Familiarization / research;
2. Specification revision;
3. Writing test documentation (checklist, test cases);
4. Data preparation;
5. Execution of tests + recommendations from programmers;
6. Buffer / Risks;
The first thing we did was hang it to everyone in the workplace. It is suitable for evaluating bugs and evaluating tasks. It is top-level and is suitable for global tasks.
And this is already for test leads and experts who evaluate very large tasks, especially when tomorrow you need to evaluate the project from the large 20 tasks.
You will not go to all testers, you will go to the first experienced and ask: "Let's estimate how much this will take." This Istimation checklist (checklist for evaluation), so as not to forget anything. For each project he has his own. Throw a list of all types and types of testing that you use, so consider what you spend time on - requirements analysis, communication with clients, programmers, documentation, creation of test cases, test plans. And then put in the check list, what you need and do not need.
The next problem - we do not want to evaluate . What to do with these people, and you really need?
Make all the estimates yourself and use them for planning. Increase your skills + it's interesting to know how well you can predict. Look at the task and evaluate, the better you estimate, the more important you are for your test lead. Surprise your team and show that you can predict what the team does not know!
Let us assume for yourself that planning is the optimal allocation of resources, apart from evaluation. Without evaluation - planning does not make sense.
Planning - the optimal allocation of resources to achieve goals, a set of processes related to the formulation of tasks and actions in the future. (c) Wikipedia .
It is important who performs the assessment. Uncertainty influences you and your test lead, the manager and the product manager, all these factors influence the project completion time.
Tell me, how am I, as a test lead, in charge of finance? Everything is very simple. If you include 4 testers in the project, then the salary must be paid to four, if three, then, accordingly, less. This is a banal level, but if your company enters into agreements on fines for an overdue project, then the financial indicators are very large. Therefore, each of these indicators affects the end time of the project.
Therefore, each of these indicators affects the end time of the project. And in such conditions I would like to have something that would help answer such a difficult question.
Here we will help the network schedule and Gantt Chart .
Here on the slide now is the work schedule, which consists of a period and tasks. This is a chart for visualizing your work so that you do not have to look at the tablet. It is more convenient to look at a beautiful chart, which is easier to work with.
All this together will help us create a critical path method.
There is a graph theory - this is the method of passing the path with the heaviest weights. In this case, “scales” are estimates. The path consists of work.
If you maximize this as project management, it will be a set of tasks from the beginning of the project to the end of the project that have no time margin. Little red is a critical path. If you violate the deadlines for the implementation of these works even for an hour, then the release will be delayed for an hour, if for 2 days, then the release will be delayed for 2 days. Why do you need it? It is important for you to understand what tasks lie on this path, because the tester asks for a day off to you in the middle of the path, but it turns out he is doing the work on a critical path and because of his absence, the release has moved 1 day. The method of the critical path allows us to anticipate the onset of such critical works, to realize what is happening in the project.
Let's describe the basic steps we take to make a plan.
1. Decide what we will test;
2. Make estimates;
3. Fill in the network schedule, build a Gantt Chart;
4. Make a logical connection between the works;
5. Assign resources;
6. Determine the critical path;
7. Add resource links;
8. Optimize resources (number of performers);
It is very important to understand. Often just such a schedule allows you to understand the relationship of resources and the impossibility of executing the 400 hour plan by hundreds of testers in 4 hours. Time is spent on data preparation, project study, requirements analysis, communication with programmers.
Variants of graphs in various programs.
Have we considered everything? If not, what remains and where to take it into account?
Do not forget:
Holidays, holidays;
Bugs
Buffer
Risks;
Performers;
If the version is not the first;
The manager takes this into account in his buffer. There can also enter the fact that there is no test environment, the programmer used a different technology, time for the establishment of bugs.
Often there is a problem when a project for testing is already given without a buffer, so count the buffer for programmers and testers separately, then it will be clear who used it.
Those who are already planning, please note that you cannot use the Gantt Chart for a group of projects if you have shared resources.
If you have a common architect, and he is needed for all projects, then this may not work.
Its advantages are when resources do not intersect at once in all projects. There are many of them, so you can manage your resources, understand where they are. But it matters to you whether it makes sense to your company.
Benefits
If the customer is interested :
If the customer is NOT interested:
You need to control it very clearly. How long does it take to modify this plan? This is a flexible plan, so changes should be made once a week. If it is automated, then your time will be spent only on optimization. In companies where this process is not automated, find out one question: "How much time is left on the task?". They are not interested in what has prevented, it is important how much time is left, and how to change the plan. Therefore, if you have 20 testers, you will have to change 20 lines.
Well, the conclusions :
1. Planning - a set of processes for:
2. Evaluation of labor costs and timing is not the same thing ;
3. Most steps can be automated .
Planning is great , because everything can be automated, remember that scheduling dates and labor costs are not the same thing.
Presentation
')
I do risk assessments in the company. Over the last period, while working as a test lead, I have been engaged in global planning. However, not all companies plan a lot of time. We'll just talk about what to do about it.
Master class will consist of 2 parts:
1. Theoretical - the concept of evaluation and planning. Roles and tasks.
2. Practical - an example of resource management ( can be seen on the video )
So what is planning? This is such a global concept that it can be applied to any area of ​​life.
– «-» .. As an example, when you go to work in the morning and think about going to the cinema in the evening, you are planning! Also at work! Despite the fact that you are not asked about planning, you plan, as you should know about when you can finish your work and pass on the results to the team. In other cases, you precisely name the release date, having planned its dates, which are often postponed due to the fact that you do not fall on the dates indicated in the plan.
Planning is a very broad concept. And before we talk about planning at the testing stage, you need to know what this word means for other members of the company and especially for management.
But the team is involved in planning, but the team doesn’t need it, they don’t want to do it. Such a problem arises due to management. The manager constantly communicates with the customer who wants to know the time. But this is not the time spent on development, but the date when it will receive the finished product, and how much time it takes.
In this case, money and time are very interrelated, especially if the deadlines for product release are violated. Maybe he does not require reporting from you, but he is associated with other specialists: marketers (advertising company), financiers (budget allocation). The last link in the release of the product - testing, so they are trying to save on it.
We’ll talk about planning in forecasting methodologies, because they have time to plan before the project starts. In flexible methodologies, everything happens “here and now”: not only regular planning, but also initial planning, as a rule, management deals with them. This is a separate topic.
If we are talking about predictive methodologies, then there may be entire stages that you can observe on the slide. Here, for example, the first stage - Incention - it can take up to six months. And this is a period when they do not program and do not really deal with requirements. Who can agree on when we get paid, and we really do nothing? This may be the military, medical industry. These can be global projects on which people's lives depend, and the development period is approximately from 2 to 5-7 years. Usually, planning takes about 2 weeks, but often it's just “tomorrow”.
What is test planning ? For a test lead, this is not only time, but what, how and where to test.
Test Planning :
1. Determine test requirements
2. Risk assessment
3. Develop a testing strategy
4. Definition of resources
5. Developing a Test Plan
6. Creating a work schedule
Each issue is worth considering. But we will talk about a small paragraph "Creating a schedule of works . " Of course, you can just say about testing, but it is better to show and somehow formalize your argument.
Now let's talk about this. Of course, planning is not possible without evaluating the results - this is our main problem.
"We do not know how to evaluate!" - The team can tell you that the junior, that an experienced specialist. The reason is seen in the absence of certain methods. This is a very big misconception. Methods are:
Requiring a detailed mathematical study:
1. Delphi method;
2. The method of three points (sequential calculation);
3. Method of analysis of functional points / testing points;
4. The method of assessing points of use cases;
5. COCOMO (COnstructive COst MOdel) - cost model;
6. Genetic evaluation model;
These are mathematical methods, but I have not seen a single company that would use them. Their main disadvantages are :
1. They take a lot of time
2. They are labor intensive
3. They do not give a result.
Why?
Because planning must be flexible . Because a difficultly calculated plan can be broken only due to the fact that someone did not come or fell ill. Recalculate? Hardly!
Therefore, we will talk about the methods that you already use, but are not always aware of!
The most simple to use:
1. PVN (finger to the sky), or trial and error;
2. Special method;
3. Analogies and recommendations of experts;
4. Structure of work decomposition;
5. The percentage of development;
6. The method of percentage distribution;
Our favorite is trial and error . We all use it often. You need to understand how it differs from the method of expert assessments .
If you have already worked with the project, he is familiar to you and tested something, then you already do as an expert. If you didn’t do this for testing, didn’t work with the project and never came across this area or customer, then you fully appreciate your finger in the sky. This is a big difference.
My favorite method is the structure of work decomposition . This is the least time consuming but formalized method for evaluation. What is meant? You just need to split up your work to the minimum, which is very easy to evaluate and even do not need to learn.
For example, test authorization. The letter "o" does not pass - you can immediately say 5 minutes for yourself. This is the minimum unit. This can be determined even by a person who works only a couple of months. Plus in this method - it's harder for you to forget something.
You take your requirements, and a big task, you divide them into 3 parts, for example: field testing, button testing, localization testing, performance testing and testing by type, method, area. Crush further: by entering numbers, letters, symbols, etc.
Pay attention to the chart, here are statistics for sites from six months - a year. Programming takes from 20% to 40% of the development, it’s not the same as 20–40% of the project, it’s an average of 15% of the project. Testing never takes 15% of the product. If you do not lay so much time for testing, then lay at least some. It is advisable to find out statistically what percentage of the project is testing and this is applicable if you have stable versions of releases, the constant volume of products is the same.
Solution to the problem
1. We train beginners:
a. Timekeeping;
b. Analysis
2. Create a universal Estimation Check List for a project portfolio;
3. We do not criticize for errors in the estimates;
To do this, you do not need to go to trainings, somehow evaluate, write down your grades and see how much time it really takes. Different trailers / apps that help record time help. Or about days to memorize. When you understand yourself, you can teach your team. It is very important that when you try to persuade a team to evaluate, it is very important not to scold it for these ratings. Most often, people do not want to evaluate, because they are afraid that you can find fault with them. Until everything is fixed, they do not owe you anything.
Explain to your employees that any incorrect assessment is better than no one. Otherwise, you, as a test lead, know nothing, cannot relate resources, and so on. Praise if they even made a mistake. Say: “You made a mistake 2 times. and I was mistaken in 3! ". But do not miss the mistakes, sit down and compare how much was planned, how long it took in reality, where there was big fail, and what took the most time. The most important thing is this analysis, when the person himself realizes that he did not take into account. This is where the work decomposition is needed in order not to forget anything.
Here we can be helped - “Forget-me-not for the tester” .
1. Familiarization / research;
2. Specification revision;
3. Writing test documentation (checklist, test cases);
4. Data preparation;
5. Execution of tests + recommendations from programmers;
6. Buffer / Risks;
The first thing we did was hang it to everyone in the workplace. It is suitable for evaluating bugs and evaluating tasks. It is top-level and is suitable for global tasks.
And this is already for test leads and experts who evaluate very large tasks, especially when tomorrow you need to evaluate the project from the large 20 tasks.
You will not go to all testers, you will go to the first experienced and ask: "Let's estimate how much this will take." This Istimation checklist (checklist for evaluation), so as not to forget anything. For each project he has his own. Throw a list of all types and types of testing that you use, so consider what you spend time on - requirements analysis, communication with clients, programmers, documentation, creation of test cases, test plans. And then put in the check list, what you need and do not need.
The next problem - we do not want to evaluate . What to do with these people, and you really need?
Make all the estimates yourself and use them for planning. Increase your skills + it's interesting to know how well you can predict. Look at the task and evaluate, the better you estimate, the more important you are for your test lead. Surprise your team and show that you can predict what the team does not know!
Let us assume for yourself that planning is the optimal allocation of resources, apart from evaluation. Without evaluation - planning does not make sense.
Planning - the optimal allocation of resources to achieve goals, a set of processes related to the formulation of tasks and actions in the future. (c) Wikipedia .
It is important who performs the assessment. Uncertainty influences you and your test lead, the manager and the product manager, all these factors influence the project completion time.
Tell me, how am I, as a test lead, in charge of finance? Everything is very simple. If you include 4 testers in the project, then the salary must be paid to four, if three, then, accordingly, less. This is a banal level, but if your company enters into agreements on fines for an overdue project, then the financial indicators are very large. Therefore, each of these indicators affects the end time of the project.
Therefore, each of these indicators affects the end time of the project. And in such conditions I would like to have something that would help answer such a difficult question.
Here we will help the network schedule and Gantt Chart .
Here on the slide now is the work schedule, which consists of a period and tasks. This is a chart for visualizing your work so that you do not have to look at the tablet. It is more convenient to look at a beautiful chart, which is easier to work with.
All this together will help us create a critical path method.
There is a graph theory - this is the method of passing the path with the heaviest weights. In this case, “scales” are estimates. The path consists of work.
If you maximize this as project management, it will be a set of tasks from the beginning of the project to the end of the project that have no time margin. Little red is a critical path. If you violate the deadlines for the implementation of these works even for an hour, then the release will be delayed for an hour, if for 2 days, then the release will be delayed for 2 days. Why do you need it? It is important for you to understand what tasks lie on this path, because the tester asks for a day off to you in the middle of the path, but it turns out he is doing the work on a critical path and because of his absence, the release has moved 1 day. The method of the critical path allows us to anticipate the onset of such critical works, to realize what is happening in the project.
Let's describe the basic steps we take to make a plan.
1. Decide what we will test;
2. Make estimates;
3. Fill in the network schedule, build a Gantt Chart;
4. Make a logical connection between the works;
5. Assign resources;
6. Determine the critical path;
7. Add resource links;
8. Optimize resources (number of performers);
It is very important to understand. Often just such a schedule allows you to understand the relationship of resources and the impossibility of executing the 400 hour plan by hundreds of testers in 4 hours. Time is spent on data preparation, project study, requirements analysis, communication with programmers.
Variants of graphs in various programs.
Have we considered everything? If not, what remains and where to take it into account?
Do not forget:
Holidays, holidays;
Bugs
- Institution time;
- Time for regression;
- Statistical approximation;
Buffer
- On a task or project?
- %?
Risks;
Performers;
- Separation;
- Experience;
If the version is not the first;
The manager takes this into account in his buffer. There can also enter the fact that there is no test environment, the programmer used a different technology, time for the establishment of bugs.
Often there is a problem when a project for testing is already given without a buffer, so count the buffer for programmers and testers separately, then it will be clear who used it.
Those who are already planning, please note that you cannot use the Gantt Chart for a group of projects if you have shared resources.
If you have a common architect, and he is needed for all projects, then this may not work.
Its advantages are when resources do not intersect at once in all projects. There are many of them, so you can manage your resources, understand where they are. But it matters to you whether it makes sense to your company.
Benefits
- Allows you to calculate the cost and timing of the project, based on numerical estimates;
- Gives insight into employment resources;
- Allows more efficiently allocate resources between projects;
- Project timing optimization tool;
- It is a visual document for management and the customer;
- If the customer is interested in using visual graphs, then you can say that it is easier for you to comply with the obligations to him, you will have a clear argument.
If the customer is interested :
- We comply with the obligations;
- We do not incur losses;
- Expanding opportunities;
- We do not save on quality;
- This will be a full time for high-quality testing, and you will give a product for which you will not be ashamed.
If the customer is NOT interested:
- We save nerves Lida;
- Develop your team;
- We introduce chips;
- We develop our initiatives;
- We enjoy the quality;
- You can improve the quality, spend time improving your team, invite someone from the test club, an internal training project, and finally do documentation.
You need to control it very clearly. How long does it take to modify this plan? This is a flexible plan, so changes should be made once a week. If it is automated, then your time will be spent only on optimization. In companies where this process is not automated, find out one question: "How much time is left on the task?". They are not interested in what has prevented, it is important how much time is left, and how to change the plan. Therefore, if you have 20 testers, you will have to change 20 lines.
Well, the conclusions :
1. Planning - a set of processes for:
- a. Creating a testing strategy;
- b. Estimates of labor costs;
- c. Forecasting dates;
- d. Assign and optimize resources;
- e. Task control;
2. Evaluation of labor costs and timing is not the same thing ;
3. Most steps can be automated .
Planning is great , because everything can be automated, remember that scheduling dates and labor costs are not the same thing.
Source: https://habr.com/ru/post/239011/
All Articles