Automation testing software systems

Greeting

Hello, dear users! I want to bring to your attention an article in which we will focus on testing software systems, its automation, as well as the means used for this.
To date, there are few who doubt the feasibility of conducting the testing process of developed software products, but, unfortunately, not everyone clearly understands how to correctly implement and use testing. My article will not bring practical benefit to the luminaries of testers, but there are something that are interested in newbies interested in topics.

Initially, I do not aim to cover the entire problem area. This is not enough and a series of books, which I, besides, do not have enough knowledge and experience to write. The main objective of the article is not spreading the idea of ​​the tree, create a clear enough picture for the reader of what testing automation is and in general, and when, and with what it is eaten.

Basic concepts

I'll start with a little theoretical excursion.
So, under testing is commonly understood the activities performed to assess and improve the quality of software. In general, testing is based on the detection of defects and problems in software systems.

Automated software testing is the process of software testing, in which basic functions and test steps, such as starting, initializing, executing, analyzing and issuing a result, are performed automatically using automated testing tools.
')
In turn, an automated testing tool is software that is used to create, debug, execute and analyze test run results (Test Scripts are sets of instructions for automatically testing a specific piece of software).

Testing of software systems consists of dynamic verification of the behavior of programs on the final test suite. In this case, the tests are selected from the usually performed actions of the application area and provide verification of compliance with the expected behavior of the system.

Application of automated testing

The first item on this list is performance testing. Stressful, stress-resistant, stability testing ... Without automation, its implementation is difficult to imagine. For this reason, there is a wide range of products from different manufacturers and equally high prices, even in the case of an inconvenient and weakly functional tool.

Next comes regression testing. It means that the software is checked for the correctness of the functionality released and tested in the previous version. It is performed with a regular frequency set depending on the conditions: someone with each new build, and someone with each version for the customer.

Configuration testing - performing the same tests in different conditions. That is, when one or more components of the system architecture are required to be tested in a different environment, usually stated in the initial requirements. For example: support for DBMS from different manufacturers, work in different client browsers, use in several operating systems, etc. That is, a certain analogue of regression testing, but within the same version of the system.

Functional testing. It is clear that here we are talking about testing a new functional. Sometimes it happens that without automation can not do. Even if you need to perform testing only once. Usually, these tests are subsequently used for recourse.

Installation testing is performed to check the installation conditions (and settings) of the product, taking into account those or other system requirements from the customer.

"What for?"

... someone will ask. The question is very reasonable. You can test, as you understand, manually, or you can use automation tools. To make a choice in the direction of a particular approach, it is necessary to understand its pros and cons.
What are the benefits of testing automation? Here are some:

• “Human factor” is excluded. Strong dignity. We are all human and none of us are immune from mistakes. The executed test script will not miss the test by carelessness and will not confuse anything in the results.
• Fast execution - an automated script does not need to be checked with instructions and documentation.
• Lower support costs - when the scripts are already written, they usually require less time to support and analyze the results than to conduct the same amount of testing manually.
• Reports - automatically sent and saved reports on test results.
• Run without intervention — during the execution of tests, the tester may be engaged in other useful things, or tests may be performed during off-hours.

However, there are a number of disadvantages, such as:

• Repeatability - all written tests will always be executed in the same way. This is both a disadvantage and an advantage, as the tester, performing a manual test, can pay attention to some details and find the defect. The script of this, alas, can not do.
• Support costs - the more often an application changes, the higher they are.
• High development costs — the development of automated tests is a complex process, since an application that actually tests another application is actually being developed.
• The cost of a tool for automation - in case licensed software is used, its cost can be quite high. Freely distributed tools, as a rule, are more modest in functionality and less user-friendly.
• Skipping minor errors - the automatic script may skip minor errors, for which it is not programmed to check.

How to automate testing?

It would be more correct to say this: how to approach the implementation of the test automation process in your activity?

First, you should pay attention to how well the automation tool recognizes the controls in the application. In the case when the elements are not recognized, it is worth looking for a plugin, or an appropriate module. If there is none, it is better to give up the tool.
Secondly, you need to keep in mind how long it takes to support scripts written using the selected tool. To do this, you can write a simple script that selects a menu item, and then imagine that the menu item to be selected has changed. If you need to overwrite the entire script to restore the script to work, the tool is not optimal, since the real scenarios are much more complicated.
And the last point to which attention should be paid is how convenient a tool is for writing new scripts. How much time is required for it, how much can the code be structured, how much code is read, etc.

At the finish line

In order to make a final decision on the expediency of using automation, it is usually advised to answer the question that arises naturally in a given situation: “Do advantages prevail over disadvantages in our case?”. If the disadvantages in a particular case are unacceptable, then it is worth refraining from automation.

Tool selection

Most often depends on the object of testing and requirements for test scenarios, because, of course, testing tools cannot support the full range of technologies used in developing applications. Thus, the choice of the tool is reduced to the banal method of trial and error. As a result, testers often choose several tools to test the functions of the application.

It is now logical to take a closer look at some popular test automation tools and give an example of writing a simple test script.

HP QuickTest Professional

Automation tool from Hewlett-Packard campaign. Distributed on a paid basis (8000-10000 USD). It is the main automation tool for functional testing from this manufacturer. Allows you to automate functional and regression tests through the recording of user actions when working with the application under test, and then perform the recorded actions in order to test the software.
Recorded actions are saved as scripts.
Scripts can be displayed in the tool as VBScript (expert view), or as visual sequential steps with actions (keyword view).
Each step can be edited and you can add checkpoints to it, which compare the expected result with the obtained one.

IBM Rational Functional Tester

Also paid, but not as much (“only something” 6000 USD).
Rational Functional Tester provides testers with automated testing tools that enable functional testing, regression testing, user interface testing, and data-driven testing.
I will not give much descriptive information about him, but rather I will give a practical example.

Usage example

Integration of IBM Rational Functional Tester with the Microsoft Visual Studio development environment will be used. To create a functional test, you must perform the following steps:

1) In the Microsoft Visual Studio development environment, create a new project “Functional Test Project”:


2) Record user actions with the application under test:


3) Create a test point during the recording process. The check point will also check the values ​​in the drop-down list:


4) Save the recording results:


Next you need to create a bat-file that will call the test script for execution and check the result:
rational_ft.exe -datastore “< VS>\DemoTestRFT” -playback uml2cqtestscript
findstr failed “< VS>\DemoTestRFT_logs\uml2cqtestscript\rational_ft_logframe.html”
if %errorlevel% == 1 goto end
exit -1
:end
exit 0

The bat file performs the following actions:
Called IBM Rational Functional Tester with the following parameters:
-datastore “< VS>\DemoTestRFT” - path to the directory with the project.
-playback uml2cqtestscript - run the test script.
IBM Rational Functional Tester records its results in an HTML report. In order to determine whether at least one step was failed during the execution of the test script, it is necessary to find the word “failed” in the report.
Depending on the search result, the result is 0 or -1.

Execution result

Selenium

And this is a free package from the company OpenQA.org.
Selenium is based on an environment for testing web applications, implemented in JavaScript and performing checks directly by the browser. Within the framework of the Selenium project, 3 tools are produced, each of which has its own characteristics and scope: Selenium Core, Selenium IDE, Selenium RC and Selenium GRID.
Supported technologies: DHML, JavaScript, Ajax
Supported OS: Mac OS, Microsoft Windows, Linux, Solaris
Test Language: HTML, Java, C #, Perl, PHP, Python, and Ruby
Tested applications: web applications.