Automation of functional testing: what it is and how it can be useful

The fact that software testing is necessary, fortunately, no one doubts. But since the testing industry itself is fairly young, it has not yet formed generally accepted methodologies and rules, as in the software development industry. As usual, functional testing of applications, systems, etc. is performed. The process can be divided into the following stages:

1. Collecting requirements (valid for "external testing")
2. Creating a test model (what and how we test)
3. Testing
4. Testing report (defects, problems, etc.)

Each stage includes exclusively manual labor. And for each new version of the application it is necessary to conduct regression testing - repeat the following stages:

1. Addition of test model
2. Testing
3. Report

Moreover, the stage “Testing” includes testing all the functionality - both the old and the new. Thus, it turns out that as the functionality grows, the amount of manual testing grows, and the testing of the “old” functionality rightly causes the tester to reject - “I have already watched it 10 times”. Consequently, the quality of testing (loss of attention) and the speed of a complete system test also fall. Regression testing leads to regression of testers. Automation is designed to speed up the testing process and avoid degradation of testers.


Consider a test scenario consisting of entering data into a form of 10 fields and controlling the result — a table of 20+ rows in which we are interested in one row containing the data we just entered. Calculate and compare the time spent on the launch of the autotest and manual test.

Manual testing:

1. Data entry, ~ 10 * 5 seconds = 50 seconds
2. Table display - 2 seconds
3. Finding the right line - 5-15 seconds
4. Verification of compliance of the entered data and the string - 10 * 3 = 30 seconds

Total for one launch - 87-102 seconds
Autotest:

1. Data entry - ~ 10 * 0.1 = 1 second
2. Table display - 2 seconds
3. Finding the right line - 0.5 seconds
4. Verification of compliance with the entered data - 0.5 seconds

Total for one launch - 4 seconds.
')
Thus, the time saving on launches is 83 seconds or 25 times (autotests work almost at the speed of the server response). And this is the simplest one-step test case. And if we have 100 test cases of 10-15 steps in each? Then man-months turn into one calendar day. Impressive? And it's not just about saving money, but also what’s never enough — time.

But there are "pitfalls". It is not by chance that I’m focusing on saving on launches - I’ll also have to write the autotest myself. Usually this is the main reason for the reluctance to implement test automation. The arguments are as follows: “Now I have 10 testers of low qualifications for n-money, and automation will require two, but for n * 4. There is practically no savings, and there are a lot of problems with the implementation. ”But if the product grows, then after a while, not 10, but 15-20-30 testers will be needed, but the number of automatics, if it grows, is insignificant. Auto tests do not need to rewrite everything, you just need to create new ones and keep up to date those existing in case of a change in the functionality covered by them. And the savings will no longer be so ephemeral, even excluding the costs of managing all of these 15-20-30 manual testers.

The most difficult period in test automation is its implementation, since it takes a very large amount of autotests to be written in a short time, and it is at this stage that, unfortunately, most of the aspirations to introduce automation stumble. But if we apply test automation in the early stages of a project, then this problem can be avoided.

Take the classic for "I am promoting" example - a startup. The nature of such projects is very rapid development. If the introduction of new features will start to fail those that were originally and are, in fact, the highlight of the project, users will leave. If you test new features and existing highlights, it will take significant time and users will leave. And in this case, automation can act like a magic wand.

Testing automation is not a scary and expensive something that is accessible only to the elect, but a real means to reduce entropy :)

PS As an automation tool, I can recommend:

• For Web applications - Selenium
• For Win-applications - White Framework
• For large budgets - HP Quick Test Pro