How to calculate the time for testing

... Or, in other words, how to calculate the time for testing so that everyone believes? After all, in fact, we usually have two goals. The first is to count the time in such a way as not to make a mistake and correctly allocate resources - most likely, at first it won't be possible to do it well anyway. The second goal is more realistic: calculate the time for testing so as to prove to someone that you need more people in the team, explain why you don’t have time, etc. Strangely enough, after just 50 you do the second, then and the first will work!

Let's now look at how to count the time for testing, with specific examples.

Case 1: “Here's the TZ, how much time will it take to write test cases?”

Suppose a volume folder with a well-written technical task (TZ) comes to us. We are asked how long it will take to write test scenarios (TS) for the whole case.
')
Before we can answer this question, we would like to know:

• Who will write the vehicle?
  
  
  • What is the level of the person who will write the vehicle: does he have less or more experience than you? If he has less experience, of course, he will need more time. If he has as much experience as you have, then your estimates converge. If a person is better than you, then it is not at all clear why you were asked to count time.
  • What is the level of knowledge of technology ? Will a person be able to conduct low-level testing, it’s good to understand everything, will he understand what is happening with the base, for example?
  • What is the level of knowledge of the subject area ? This is even more important than the level of knowledge of technology, because if we, for example, need to test some sophisticated financial application, and we do not understand what bonds and options are, it will take a lot of time to understand the subject area. Without this, it is difficult to understand how difficult TZ is, how complex the product is and, ultimately, how voluminous and what a vehicle should be.
  
  
  
• Why and for whom are the vehicles written?
  
  
  • For ourselves - if we test our internal development. We have our own product, our own team, and we write the vehicle for our product.
  • For the client . Sometimes the customer orders only testing, and sometimes only the vehicle. Sometimes he asks to test once, and then leave him an artifact, according to which he can test it himself. TCs for a client may be less detailed and detailed, but differently than if we wrote a vehicle for ourselves. So, we ourselves are well aware of all the terms of testing, but we may need to write in detail in the TC all sorts of complex things from the subject area. The client may understand the subject matter perfectly, but he may not know what form validation is and he will need to explain it.
  • For acceptance testing - in this case, the TS is even more difficult to write than for the client, because it is testing, the results of which the client may not accept if he does not want. Therefore, we write a much more formal TS for acceptance testing - there should not be any statements like “everything works fine,” because for such a TS the client may not accept anything from you and never.
  • For automation. Here we have to think about data-driven testing, how to automate it, and so on.
  • For the soul . In this case, we just want to understand and there is no request from the team for testing.
  
  
  
• Where are written, in which test management system?
  
  
  • Familiar environment.
  • Unfamiliar environment. In an unfamiliar environment, of course, we will need more time.
  
  
  

From the answers to all these questions depends on how long it takes to write one positive test case - T1. It is also worth considering the time to write one negative TS (it will be denoted as T-1). My experience says that negative TSs are usually written longer - they also usually need more. Of course, it happens differently here, but it seems to me that the better we know the subject area, the better we understand the application, the better we understand where and how it can fall, the more negative we will have the TS. If you have, for example, three negative TSs have one negative, you probably misunderstand the application. If for one positive you can come up with 500 negative TS, this is a good job. My ratio of positive and negative TS is usually from ½ to 1/5.

T. h. We need to calculate the time to write a positive TS, estimate the ratio of positive and negative, and count the time entirely.

Evaluation options:

• Rough expert . For example, we know that usually we write 5 TS on one page of the TZ. Usually one vehicle we write 20 min. The specification of 300 pages. 5 x 20 x 300 = 30000 - this is how many minutes offhand we will write the TS.
  
  
  • 
    
    X test cases per page, T1 minutes per vehicle, Y pages (T-1 - negative vehicle)
  
  
  
• Rough deductive . Suppose we have a project plan, there are deadlines, there is a team. And we present what the ratio of developers to testers should be in the project. For example, the project for six months. There are five developers (which means we need two testers for these six months). We see that the TS will be writing about a quarter of the time of all the time allotted for testing. This is a rough but quick way to count.
  
  
  • Testing takes X time from project time.
  • 
    
    Writing test cases Y from time to time for testing
  
  
  
• Inductively experienced . This is a slow way to evaluate, but it works well. Here we are trying to write a vehicle for each part of the TZ. It is better to do this with the degree of detail with which we will write them further. However, you can make at least a high-level vehicle. It may take a day, two, a week, but it will help to understand how much time it will take at all.
  
  
  • We break the TZ into homogeneous parts (for example, by functional) and try to write the TS for each part
  • We look at how much time it took to write a vehicle for a certain number of pages of each part, based on this, we calculate how much time it would take to write a vehicle for each part.
  • We add the time counted for each part.
    
    
    
    
  
  

Example

Let's now see how these techniques work, with a specific example. For example, we are writing a social network for pets. We have a TK for this social network, only 47 pages. Suppose we broke it into homogeneous parts. Here is what we end up with:

• TK :
  
  
  • Host account (5 pages).
  • Animal profile (15 pages).
  • Fotochki, video (12 pages).
  • Friends (8 pages).
  • Messages (5 pages).
  • Non-functional requirements (2 pages, 12 pieces).
• Programmers : four people.
• Deadline : six months.
• A rough expert assessment (adopted in our company): two positive vehicles per page, 0.5 hours for a positive vehicle, 45 minutes. - negative

Now we will try to calculate how many hours it will take for us to write the vehicle (but, pay attention, not for testing at all):

• A rough expert assessment : (0.5 + 3 x 0.75) x 2 x 47 = 258 hours.
  
  Comment: negative TS here 3 to one positive. Two positive TS on one page, only 47 pages. 0.75 is 45 minutes.
• A rough deductive estimate : (176 x 4 x 6) / 3/4 = 352 hours.
  
  Comment: 176 hours per month, four programmers for six months, divided into three (the ratio of programmers to testers is approximately 1.5 testers on a project (4 / 1.5 ≈ 3)) and divided into four (a quarter of the time from all testing will be go to the writing of the vehicle).
• Inductively experienced rating : (1 x 10) + (3 x 15) + (6 x 12) + (2 x 16) + (1 x 10) + (3 x 12) = 205
  
  Comment: we spent an hour on the host account, managed to cover half of the page, and we have 10 of these halves (1x10). We spent 3 hours on the profile of the animal, for three hours covered one page (3x15). We spent 6 hours covering one page of photos / videos (so much because, let's say, everything is very difficult). 2 hours spent on friends and made half a page during this time (2x16). 1 hour spent on the message, made an hour half a page (1x10). Non-functional requirements, we decided to look here not by the pages, but by the requirements themselves: we spent 3 hours on one requirement (3x12). All this took us 16 hours - it means that in 2 days we estimated the time at 205 hours.

Our estimates do not agree, but this is normal - at least we were able to calculate approximately. And if we have enough time for evaluation, we can make an assessment in several ways.

We count time when using Agile

Now let's talk about flexible development methodologies, when there is no technical task. It is not very clear how to evaluate, but still some approaches can be distinguished. For example, you can:

• just walk through the backlog;
• write high-level TS;
• write down all the actions inside the sprint and try to estimate how much time they will take - this approach will be discussed below.

Retreat for QA-managers

Now a slight digression for QA-managers, project managers. We can divide all actions for creating and launching test cases into the following components:

CREATION OF THE CU	STARTING TS
Environment Preparation (TCenv)	Fulfillment of preconditions (TRprec)
TC structure preparation (TCstr)	Run Regression Tests (TRreg)
Creating a vehicle for the finished functionality (TCbase)	Running tests for new functionality (TRnew)
Creating a vehicle for new functionality (TCnew)	Build new configurations (Cnew)
Automated Test Creations (TCa)	Number of configurations (Qconf)
Change created TC (TCchng)	Retest after fixing bugs (TRrtst)
Outcome: ΣTC = TCenv + TCstr + TCbase + TCnew + (TCbase + TCnew) x TCa + TCchng	Total: ΣTR = (TRprec + TRreg + TRnew) x Qconf + TRrtst x Qconf + Cnew

How much time does the testing take, we see the table in the formulas below - everything is clear and without comments.

Case 2: "Here is a piece of functionality, how much time will it take for testing?"

How much time will it take to test one or another part of the functional? To estimate this, there are different ways:

• 
  You can see how much time it usually takes us to test similar functionality.
• You can look at the same functionality , if we have previously tested this. It is very convenient! If you, of course, recorded the last time, how much time it took.
  
  
  
• Deductive evaluation - if we know how much this functionality takes place in the general functionality (if we, for example, already know how long a complete regression takes), we simply see that testing this functionality will take, for example, one fifth of the total time.
  
  
  
  
• Inductive assessment - we remember how much time we spend on each vehicle. We look at this functionality and we understand that we need 10 - 15 vehicles to test it.
  
  
  
  

What is important to remember in this case? That the new functionality can again require time to prepare the environment. It is important to remember about the existence of a coefficient on familiarity with the system: more time is spent on tests if you run them away the first time than if you run them second or third time. But if the second time comes in a month, then you still chase him away, as if for the first time - however, this is different for everyone.

Case 3: "Why do you not have time?!"

The next, very common question asked by testers: “what do you do, why do you not have time, why do you need help with automation, why do not you want us to release new versions every day - there are no new bugs ? .. "

To show what we do, we can write out for each project all the tasks that are performed in one sprint, and paint them by day: calculate how much time each task takes on a particular day. So you can see the average, maximum and minimum load.

An example of such a table:



When I did it, I saw that the load could not always be optimally distributed - sometimes the load is large, and sometimes half a day there is nothing at all to do. This allows you to make certain conclusions and somehow optimize the process, to distribute the load - for example, to do some things in advance, even if it is not completely consistent with the logic of the process. For example, if we have a lot of free time at the beginning of a sprint, we can already start writing user stories for the next sprint. Although we don’t know the structure of the next sprint, we can most likely predict that these two or three stories will definitely fit into it - they are usually the most critical, usually they have to be rewritten anyway, so this is a good exercise.

So:

1. Defined a task list.
2. Defined "input parameters".
3. Determined time for each task.
4. Smeared the tasks of the sprint.

Received download in hours by day. And if it turns out that every day you need 24 hours of testing, and we have only two people ... So, we definitely need a third tester!

Case 4.1: "We are releasing the assembly on Friday, or even earlier - so how much time do you need for testing?"

In this case, the testing process is as follows:

1. Smoke test + test for complete regression x on the number of major platforms + test for basic functionality for all platforms + tests for complete regression on platforms (see the matrix).
2. Retesting : smoke test + validation of the whole platform.

Tests for a complete regression on the platforms are carried out in this case as follows (Cr, FF, etc., here are different platforms, in this case browsers):

	Cr	FF	IE9	IE8	Cr '	FF '	Σ
Smoke test	one	one	one	one	one	one	6
Basic functionality	one	one	one	one	one	one	6
Administration	one						one
Statistics for admins		one					one
User account			one				one
User Posts				one			one
User friends					one		one
User photos						one	one
User Files	one						one
User Settings		one					one

According to this matrix, we see that for each platform we are testing any part of the functional: as a result, it turns out that we have checked all the functionality and tried the work on each platform. We can also shift this matrix - in the end it turns out that for a certain number of runs we will check every function on each platform anyway. This is still a simple scheme - and in fact there may be 20 configurations without the ability to automate their testing - of course, in this case, we will not have time to test everything, and we will test on such a matrix.

As a result, we obtain the following formula:

ΣT = (Tprec + Treg) x (Qmain + 1) + (Tprec + Tbase + Tsmoke) x Qconf + (Tsmoke + Tvalidation) x Qconf x Qreturns

Qmain - the number of main configurations on which we test everything.

Qreturns - the number of returns.

By this formula, it is clear how much time it will take. On it, you can try to keep track of time, look at which testing takes the most time. It will also be clear where it is better to automate where - we will understand where automation will save us the most time.

Case 4.2: “Too many tests! Hurry up".

If we are required to perform testing faster, we can cut the number of tests according to the following principles:

• We look at the priorities:
  
  • test cases - if we describe the priorities of the vehicle. Then we will work only with the highest priority vehicles;
  • user stories or bugs , if our vehicles are attached to them.
• We look at the release notes, which include :
  
  • critical functionality , without which the application will not work and no one needs it;
  • declared new functionality - since it is publicly announced, users will definitely check it;
  • Correction of critical errors (most likely, errors will already be checked);
  • declared bug fixes - we stated that we fixed something, and we check it;
  • Known issues (known issues) - paradoxically, but we definitely have to retest those errors that we have stated as errors with which we have released a product. Why do this? First, you need to check that their impact is exactly the same as we described. Secondly, we check if that workaround of this error that we offer works.
• Instead of reducing the amount of testing, you can increase the command :
  
  • To add programmers is best from your own project. For example, they can write data generators, etc., all for automation or semi-automation. They do it well, and the responsibility for testing is still not on them - so they do not suffer from the fact that their code does not work well. For the same reason, some programmers can be asked to check the code of other programmers.
  • Call testers from other projects, or from this project, but doing something else.

In general, the actual testing process can be represented as a triangle. The angles of this triangle are:

1. Scope of testing.
2. The number of people in the team.
3. The period for which you want to perform testing.

The speed and quality of testing depends on these three aspects - it is on them that you need to think in order to successfully plan your work. So, for example, if we are offered to test a product for an unrealistically short period, and we definitely don’t meet or refuse such work (if we don’t want to release a low-quality product, of course), or work with the above aspects. Then we can:

1. Reduce testing.
2. Increase the number of people in the team.
3. Ask to extend the period for testing.

The latter method is perhaps the most reliable: after all, people in a team can give inept ones, and a reduction in the amount of testing can lead to the release of a raw product.

A method also works here - to ask why such terms are. Maybe you need only one function for one client - then we will make him a private release, and later we will test everything else.

And finally, the case: the revision of the already released project

If the project has already been released, but there is a demand to finalize it, how then to calculate the time for testing, for development, etc.? In this case, we count time as usual, but at the same time we multiply it by two, because all people have already forgotten everything, already in other projects - it takes time to get acquainted with the system. In addition, the addition may be outside the standard cycle, therefore, in addition to testing this addition, complete regression may be needed - then we add time to the complete regression.

Posted by: , QA Lead