Test automation from scratch. Part 1
Good afternoon, dear readers.
I want to talk about the experience of building a test automation system, when the project is either not at all testing, or its degree is minimal.
I hope the article will be useful for beginner autotesters.
')
The short answer is as early as possible.
A full one will reveal below. In any case, when the project has been working for 3 years, and everything has been checked manually, life becomes very monotonous. And a fleet of 5000 scenarios to automate in a month or two is problematic. As a rule, in such projects you will have to start working on the scripts again. Because it will be faster. And not the fact that the old will automate without significant changes.
Because autotets:
The larger the fleet of automated scenarios, the more likely it is to find a regression, especially if the data changes each time it is started.
If the autotest passed steadily and fell on some branch, then either the requirements were changed, or the bug, or the infrastructure failed.
If the requirements were changed, the autotest should go to the rework along with the reworking of the main functionality. That is why testers take part in the coordination of TK.
If the test run is automatically linked to the task, then no one can say that it has not been tested. Or vice versa, can. In general, the problems just get smaller.
If each test satisfies the boring conditions:
Precondition - Test - Postcondition
That such tests are easy to automate.
And then it is easy to parallelize. Because they are independent.
The number of threads - how many test servers will stand and not to interfere with others.
The second point is speed itself. In manual mode, click on the creation of the goods, its search and purchase in the online store is 5 minutes. 4 browsers. Total 20 minutes. On just one small script.
Autotest for this scenario takes 1.5 minutes. But on 8 browsers. (Last and next to last versions of each browser).
With custom scripts.
The value of atomic tests all the time falls, especially on microservices. And in general, in an ideal world, this is the responsibility of the programmer. Such errors should be detected at the unit testing stage.
QA should work from the user story. Because the programmer usually does not know it, and does not want to know.
Accordingly, the logic 1 test - 1 user case (business scenario) is closest to reality.
There are of course difficulties with preparing data, for example, in the case of an online store, the process of paying by card requires the presence of things in the basket, and either data for a new user, or authorization data of an existing one.
Yes, sometimes the preconditions take longer than the test, but when reusing scenarios it does not carry any difficulty.
Least subject to change in the near term. To automate at least some lived.
Or most commonly used. It makes sense to help manual testing in the generation of test data. For example, in the bulletin board it makes sense to automate the creation of ads, because Further, any script requires created ads.
Usually there are projects
And the last two points - usually live separately. Mobiles are testing on their own scripts, and very few people are able to approach the debugging of hardware using JTAG without preparation.
Therefore, I propose to do Backend and Front.
Suppose we have an online store.
There is an admin, there is a client and admin front.
There is an API that serves all of this.
Let's watch.
There are customers, there are employees.
The client has the first case - view and search for the product, add it to the cart, and design.
The employee's most common case is the addition of a product card.
The most important thing for the store is sales.
Therefore, customer history is most important for business. Therefore, finding a product, placing it in a basket and completing registration with any payment method is the first thing to do.
By API. But without a front. Here you can argue, but most likely the design will change 2-3 times more, but the backend is unlikely. So at the first stage you will have to check the interface manually.
Next, make a check on the product card editing API and its front.
And return to the client. At this stage there will already be statistics of the most frequent user actions, and the most frequent ways of the client. Yes, Yandex Metric and Webvisor help testers.
It makes no sense at the first stage to check whether the payment function works for the firm’s account through the generated payment order, if this function is used by 0.5% of visitors. Of course, you can ask the manager a question why this is needed here, but this is not about that.
Suppose we have 40% of people pay on the site with a card, 30% in cash, 20% in cash on delivery and 10% all other methods.
What type of payment will we check first? Of course the card.
That is, we must clearly understand which scenario is the most important for the business at the moment. And ensure its quality in the first place.
We created everything in the tests. We litter. It would be necessary for a clean. It is necessary to foresee in what tests we will clean up after ourselves, and in which not.
If the goods can be left in the store, and nothing terrible will happen, then the addition of administrator rights to the average user needs to be returned. And then the next test associated with the rights may fall. Or worse - go positive, although it had to fall.
There is a strange approach when automating the scenarios in which users find a bug. Usually these are very exotic scenarios, and it does not make sense to spend a resource on them. There was a situation when the functionality of updating the data of the bank-counterparty broke. Fail to synchronize with the directory BIK.
That is, the new BIC has not been updated. But the new bank started up normally.
Does it make sense to automate this script? If contractors change every day - maybe. And then it was a flaw in planning.
If this is done 5-6 times a year, can it be better to deal with a higher priority?
In the next article I will tell you how to quickly start testing the API, embed these tests into releases, parallelize tests, how to select data for tests and what it will give us.
Let me remind you about the effect of the pesticide, and how it shines to a minimum.
Technologies: Java + maven + testng + allure + RestAssured + Pict.
I want to talk about the experience of building a test automation system, when the project is either not at all testing, or its degree is minimal.
I hope the article will be useful for beginner autotesters.
')
- In the first part, let's pofilosovstvuem in general about the approach.
- In the second part ( Part 1 ), using examples, we will make a JAVA + autotest project that will teach how to quickly test the API.
- In the third part, we supplement the project for UI testing, we will do parallel execution of tests.
When to do autotests?
The short answer is as early as possible.
A full one will reveal below. In any case, when the project has been working for 3 years, and everything has been checked manually, life becomes very monotonous. And a fleet of 5000 scenarios to automate in a month or two is problematic. As a rule, in such projects you will have to start working on the scripts again. Because it will be faster. And not the fact that the old will automate without significant changes.
Why?
Because autotets:
- Accumulate scenarios for regression
- Unbiased
- Fast
Accumulated Scripts
The larger the fleet of automated scenarios, the more likely it is to find a regression, especially if the data changes each time it is started.
If the autotest passed steadily and fell on some branch, then either the requirements were changed, or the bug, or the infrastructure failed.
Impartiality
If the requirements were changed, the autotest should go to the rework along with the reworking of the main functionality. That is why testers take part in the coordination of TK.
If the test run is automatically linked to the task, then no one can say that it has not been tested. Or vice versa, can. In general, the problems just get smaller.
Speed
If each test satisfies the boring conditions:
Precondition - Test - Postcondition
That such tests are easy to automate.
And then it is easy to parallelize. Because they are independent.
The number of threads - how many test servers will stand and not to interfere with others.
The second point is speed itself. In manual mode, click on the creation of the goods, its search and purchase in the online store is 5 minutes. 4 browsers. Total 20 minutes. On just one small script.
Autotest for this scenario takes 1.5 minutes. But on 8 browsers. (Last and next to last versions of each browser).
Where to begin?
With custom scripts.
The value of atomic tests all the time falls, especially on microservices. And in general, in an ideal world, this is the responsibility of the programmer. Such errors should be detected at the unit testing stage.
QA should work from the user story. Because the programmer usually does not know it, and does not want to know.
Accordingly, the logic 1 test - 1 user case (business scenario) is closest to reality.
There are of course difficulties with preparing data, for example, in the case of an online store, the process of paying by card requires the presence of things in the basket, and either data for a new user, or authorization data of an existing one.
Yes, sometimes the preconditions take longer than the test, but when reusing scenarios it does not carry any difficulty.
What scripts to automate
Least subject to change in the near term. To automate at least some lived.
Or most commonly used. It makes sense to help manual testing in the generation of test data. For example, in the bulletin board it makes sense to automate the creation of ads, because Further, any script requires created ads.
What exactly to do?
Usually there are projects
- Backend
- Frontend
- Mobile applications
- Glands
And the last two points - usually live separately. Mobiles are testing on their own scripts, and very few people are able to approach the debugging of hardware using JTAG without preparation.
Therefore, I propose to do Backend and Front.
We choose the script
Suppose we have an online store.
There is an admin, there is a client and admin front.
There is an API that serves all of this.
How to start automation?
Let's watch.
There are customers, there are employees.
The client has the first case - view and search for the product, add it to the cart, and design.
The employee's most common case is the addition of a product card.
What case automate the first? And How?
The most important thing for the store is sales.
Therefore, customer history is most important for business. Therefore, finding a product, placing it in a basket and completing registration with any payment method is the first thing to do.
By API. But without a front. Here you can argue, but most likely the design will change 2-3 times more, but the backend is unlikely. So at the first stage you will have to check the interface manually.
Next, make a check on the product card editing API and its front.
And return to the client. At this stage there will already be statistics of the most frequent user actions, and the most frequent ways of the client. Yes, Yandex Metric and Webvisor help testers.
It makes no sense at the first stage to check whether the payment function works for the firm’s account through the generated payment order, if this function is used by 0.5% of visitors. Of course, you can ask the manager a question why this is needed here, but this is not about that.
Suppose we have 40% of people pay on the site with a card, 30% in cash, 20% in cash on delivery and 10% all other methods.
What type of payment will we check first? Of course the card.
That is, we must clearly understand which scenario is the most important for the business at the moment. And ensure its quality in the first place.
Postconditions
We created everything in the tests. We litter. It would be necessary for a clean. It is necessary to foresee in what tests we will clean up after ourselves, and in which not.
If the goods can be left in the store, and nothing terrible will happen, then the addition of administrator rights to the average user needs to be returned. And then the next test associated with the rights may fall. Or worse - go positive, although it had to fall.
Oddities
There is a strange approach when automating the scenarios in which users find a bug. Usually these are very exotic scenarios, and it does not make sense to spend a resource on them. There was a situation when the functionality of updating the data of the bank-counterparty broke. Fail to synchronize with the directory BIK.
That is, the new BIC has not been updated. But the new bank started up normally.
Does it make sense to automate this script? If contractors change every day - maybe. And then it was a flaw in planning.
If this is done 5-6 times a year, can it be better to deal with a higher priority?
What will happen next?
In the next article I will tell you how to quickly start testing the API, embed these tests into releases, parallelize tests, how to select data for tests and what it will give us.
Let me remind you about the effect of the pesticide, and how it shines to a minimum.
Technologies: Java + maven + testng + allure + RestAssured + Pict.
Source: https://habr.com/ru/post/426487/
All Articles