Automation testing software systems

To date, few people doubt the feasibility of the process of testing the developed software products. The goal of any testing project is to ensure the quality of the product being developed. Automation increases the efficiency of testing and, consequently, improves the quality of the software being created. The main objective of the article is to create in the reader a fairly clear picture of what the testing automation is.

Basic concepts

The first thing I would like to start with is to refresh the main terms and concepts that will be used in this article in the future:

Testing is a process that contains all life cycle activities, both dynamic and static, related to the planning, preparation and evaluation of a software product and the work related to these results in order to determine their compliance with the described requirements and to demonstrate their suitability for the stated purposes and to determine defects;
Test automation — using software to perform or assist in conducting specific process tests, for example, test management, test design, test execution, and validation of results;
Automated software testing - software testing process in which the basic functions and steps of the test, such as starting, initializing, executing, analyzing and issuing the result, are performed automatically with the help of a certain technology stack, in which the tool for automated testing is in the main place;
A tool for automated testing is software, through which a specialist in automated testing performs the creation, debugging, execution and analysis of the results of a run of test scripts;
An application programming interface (sometimes an application programming interface) (eng. Application programming interface, API [hey-pi-ai]) is a set of ready-made classes, procedures, functions, structures and constants provided by an application (library, service) for use in external software products. Used by programmers to write all sorts of applications;
Load testing - a type of performance testing conducted to assess the behavior of a component or system under increasing load (the number of concurrent users and / or the number of transactions) to determine the maximum allowable load level for the component or system under investigation;
Death regression spiral - this effect occurs when more and more time is spent on project testing, since the finished functionality in the product becomes more and more and it is necessary to constantly monitor that it still works;
Regression testing - testing of an already tested program, carried out after the modification to ensure that the modification process did not introduce or did not activate errors in the areas that were not affected. It is carried out after changes in the code of the software product or its environment;
A test script is a set of instructions for automatically checking a specific piece of software;
A test suite is a combination of test scripts to test a specific piece of software combined with common functionality or goals pursued by launching this kit;
Test data - data that exists (for example, in the database) at the beginning of the test \ test script and affect the work, or are influenced by the system or component under test;
Framework (born framework - framework, structure) - the structure of the software system; software that facilitates the development and integration of various components of a large software project. (The word " frame " is also used, and some authors use it as the main one, including not based on the English-language analogue at all );
Functional testing - testing based on analyzing the specification of the functionality of a component or system.
')

Types of testing

Having a little understanding of the basic concepts, I suggest you consider the main types of testing:
• The first item on this list is load testing . Without automation, its execution is difficult to imagine ( software products simulate the load as follows: virtual users are connected that execute various scripts (actions) according to different scenarios. );
• Next comes regression testing . Errors that occurred after making changes to the program are called regression errors (eng. Regression bugs). It is carried out with a regular frequency set depending on many conditions: it can be carried out with each new build of the project or with each version for the customer;
• Functional testing is carried out in order to verify the feasibility of functional requirements, that is, the ability of the software in certain conditions to solve the tasks needed by users. ( One of the main tasks of automating regression / functional testing is to save the project from the regression spiral of death. )

Advantages and disadvantages of automated testing

But is everything as good and beautiful as described above? To answer these questions, I propose to “weigh the pros and cons” in order to draw appropriate conclusions for yourself.

Benefits:
• “Human factor” is excluded at runtime: the test script will not make mistakes due to carelessness;
• Fast execution;
• Automatically generated and saved reports on test results;
• Running in the background - during the execution of tests, you can engage in other tasks or run test scripts during off-hours.

Disadvantages:
• Same type - all written tests will always be performed strictly according to the algorithm implemented in them, while the tester, performing a manual test, can pay attention to some details and find the defect. ( For example, after the next update of the project, an optional field was added to the operation form, but the developer made a mistake and the data format for the input was not correct. During the functional / regression testing of the application, the test script will work without errors, because in its algorithm the interaction with this field is not implemented. );
• Support costs — the more often an application changes, the higher they are. ( As a result of improvements, the specific functionality may change, which will lead to partial or complete unsuitability of test scripts. The automated testing specialist will be faced with the task of bringing the test script (s) to the current state. );
• The high cost of developing a test framework for a specific project ( in fact, an application is being developed that is testing another ).

The introduction of automated testing

Before thinking about the implementation of test automation, you need to make sure that the quality control process on your projects is built, documented and works like a clock. Remember that the introduction of automation is not a fad. This is a task designed to raise the quality control on your project to a new level, in view of which the efficiency of project testing increases (without an additional headache).
The second step will be an appeal to specialists who will help you to put the automation process on a professional level and in a short time. Undoubtedly, you can try to develop the direction on your own, but without experienced specialists, this process will probably result in substantial time, will be tried and error, will affect a larger budget, and, ultimately, it may well discourage you from automated testing.
When deploying to test automation projects from scratch, in most cases, the development of a framework is recommended. This approach will allow your specialists to independently develop the framework and develop coverage with test scripts. Today it is the most technologically advanced solution in terms of price / labor / efficiency.

The features of this framework include:
• Maximum code reuse: an API is actually created that provides control over the execution process;
• Application of Data Driven methods (testing according to the same scenario, carried out with different sets and / or values ​​of the source data);
• All test cases (test cases) and launch packages (test suites) are also described in external files, which makes it easy to manage the launch parameters;
• The framework has maximum flexibility: you can easily add, delete, edit existing test scripts and startup packages, while this task does not require additional qualifications, you only need the ability to work with the framework;
• New operations can be easily added to the system, or existing ones can be modified; it does not require any complex actions, you will only need to write a new function. This allows you to easily and painlessly expand the framework itself;
• For many testing departments, this solution may be a panacea, since based on the results of the development of the framework and brief training on how to work with it, the requirements for the qualifications of specialists covering the auto test system are significantly reduced, and XML skills are sufficient.

Results

Finally, it is worth noting that the goal of automation is to increase the efficiency of the process (in this case, testing) by releasing specialists and, therefore, reducing costs.
When considering the issue of automation is worth remembering the cost of implementation. Most test automation tools are paid, in addition, additional labor costs for adaptation are required. Finding a balance between manual and automated testing of any software product is an important task of the testing unit in any organization.