Translation of Hans Buvald’s article “Action Based Testing”
We have prepared a translation of an article by Hans Buvalda, CTO of LogiGear, in which he dispels the myths and fears associated with testing automation in Agile projects.
For his professional career, Hans Buwalda (Hans Buwalda) has gained vast experience as a software developer, manager and chief consultant at leading companies and organizations in different countries of the world. His proposed testing methods (action-based and soap opera-like) have helped many customers develop scalable and easily supported solutions for a large number of complex test tasks. Hans often speaks at international conferences. He also co-authored the book Integrated Test Design and Automation .
How does functional testing automation fit into Agile projects? Today we are constantly faced with this issue. Agile's strategy is becoming more widespread, but functional testing is usually still performed manually, because during the Agile iterations and sprints, there is simply no time left to automate it. Unlike functional testing, effective unit testing automation has become commonplace. You can answer the question by solving the following tasks:
')
In this article, I will show how an action-based testing method can help accomplish these tasks. To do this, I will briefly present an action-based testing method, and then I will tell you how it can be used to design test cases and automate them in accordance with the requirements of Agile projects.
There are many sources where you can learn more about action-based testing methodology. Here I only summarize the basic principles underlying this method.
Typically, testing and automation tasks are included in the system development life cycle, regardless of whether it is cascading or agile. However, there are three life cycles in ABT, which, although interdependent, are planned and managed as separate entities in ABT projects:
The main place is occupied by the design of tests. This is the main factor in successful automation, much more significant than the automation technology used. In ABT, it is important to create a high-quality design of high-level tests, within which so-called test modules are defined. Each test module is developed as a separate mini-project and must have well-defined content that is different from other modules.
The test module includes test objectives and action lines. The objectives of the test set the content of the test module in the form of individual verbal provisions that determine what exactly should be tested within this module.
Tests in a test module (which is arranged in the form of a table) are determined by a series of lines of action, which are usually organized in the form of one or several test scenarios. Each action line defines an action and includes a work word defining the action, as well as arguments defining the data for the action, including input values and expected results.
It should be noted that in ABT the test script does not have the same meaning as in some other testing methods. We believe that the test script is a small separate component that is not decisive for test development. Instead of creating a predefined list of test scenarios, we create a list of test modules in which test scenarios are included as a result of test design, and not as its initial elements.
Efficiency is provided by varying different test scenarios. In addition, each test case can provide a prerequisite for the next scenario, which gives a good flow of test execution.
In ABT, automation work is done separately from test design. For designing tests and for automation, different skills and abilities of developers are required. Perhaps there are specialists who are engaged in both, but, judging by my experience, they are not often met. In addition, these two areas imply different responsibilities for ensuring the execution of tests.
In ABT, automation engineers are engaged in automating actions and creating “interface definitions” to control the interaction of the system under test with various interfaces (user and others). To perform these automation tasks require special skills and experience.
When using ABT with two separate life cycles — test development and test automation — you should pay attention to two topics that are important for including automated testing in Agile projects:
When using the Scrum sprint methodology, testing in Agile projects is divided into three processes:
The most common practice is the development and execution of tests in the framework of sprints. In the sprint, the functionality is determined on the basis of a conversation with the customer and user histories to identify exactly what needs to be tested. Testing can be carried out through the developed tests, similar to the ABT test modules, as well as through research and interactive testing. In addition, it will be very useful to conduct at least some of the most "interesting" interactive tests in test modules for later use.
Unit testing is extremely important in itself, but in ABT it makes sense to consider the possibility of re-using such tests and their application in various areas for testing individual functions.
By defining test modules for unit testing and assigning them to selected actions, you can easily combine them to test a wider range of values and the various elements of the system under test during or after the sprint.
When using the ABT method, the use of actions, in particular, high-level business actions, allows you to develop tests that focus not on technical aspects, but on business functions. Such tests are commonly referred to as high level tests. They do not relate to user interface elements, but are aimed at business transactions, for example, making a request for a mortgage loan or renting a car.
Higher level tests can be developed at the initial stage of the project. For their development, it is not necessary to wait for the completion of the system development sprint, since in this case there is not enough time to define business functions and create suitable tests to verify them.
The number and the very need for business-level tests depend on the specific situation. In general, I would recommend the following:
After completing the sprints for individual parts of the system and after combining these parts, additional testing is usually required to ensure the quality of the entire system and its compliance with the requirements. In addition, additional tests may be needed to retest the elements of the system that were not affected by the changes to confirm that the new elements interact well with the old elements of the system. This may be necessary, for example, in regression sprints or resilience-enhancing sprints.
In my opinion, such “follow-up testing” is the main area in which it is advisable to have pre-designed test modules and fully automated actions. This will save valuable time, especially when the time of product release is approaching.
The term “timely automation” is usually used to describe test automation in Agile projects. When using the ABT method, this concept changes and sounds like “timely development of tests”. In any case, high-level automation plays an important role in increasing productivity and speeding up sprints.
In order to provide automation quickly and in time, the following rules should be observed:
Successful automation starts with creating a solid foundation for developing follow-up actions. Such a framework should provide various capabilities: performing all necessary operations on all classes of user interfaces, access to APIs, the ability to perform database queries, compiling and processing messages using the appropriate protocols, etc.
Although many technical capabilities are available using TestArchitect's tool created by LogiGear, most of our projects start with research and development related to specific technical problems of the customer, for example, modeling devices for POS-terminals, working with three-dimensional graphics for oil exploration, testing mobile devices, accessing firmware in diagnostic equipment, etc.
It is important to start creating such a technical basis as early as possible and pay close attention to it, identify and solve all technical problems. This will help implement low-level actions, which can then be used for higher-level actions, for example, in development sprints. In addition, the creation of a technical basis at the initial stage reduces subsequent risks.
The essence of Agile projects is that many elements of the system being tested are clarified only when they are implemented in a program as part of iterations, such as sprints in Scrum. This primarily applies to areas where automation is particularly important, for example, when testing a user interface. These elements may undergo many changes in the design process. At the same time, automation should not become a “bottleneck”. In this case, the main thing is flexibility.
The action-based testing model is able to provide the necessary flexibility, since it allows you to hide parts in separate actions, which can then be quickly reconfigured if necessary. However, other points must be taken into account. Most often in our projects there is a problem of “timing”. Often, to perform automated actions you have to wait for the system under test to react to one or another action and be ready for the next one.
We concluded that to solve this problem, an automation engineer should make the most of the active timing capabilities. To implement it, it is necessary to set a criterion in the system under test - a predetermined maximum value. When this value is reached, an automatic transition to the next action takes place immediately. By paying due attention to these and similar issues, you will ensure the reliability and flexibility of automation.
When preparing for automation, system developers should indicate which elements of the system under test provide convenient access for automation equipment. If such elements are identified at the initial stage and are defined as requirements, they can easily be included in the sprints.
A good example is the use of values for certain identifying properties that exist in various platforms for screen controls or HTML elements. The user does not see these properties, but they are available for automation tools. Providing such values will make it easy to automate testing of controls or other elements, and, as a rule, irrespective of any design changes.
If these values are defined in the early stages of a project, then using, for example, a tool like TestArchitect, you can create “interface definitions” and use them even before building the system under test.
Examples of such useful properties are the “id” attribute in HTML elements, the “name” in Java / Swing and the “accessibility name” in .Net and WPF. They have no effect on user experience, but are available for automation tools. In addition, the use of such properties helps to solve localization problems: the "OK" button can always be found, even if the inscription on it is in another language.
Automation should be tested. In ABT, this means that you must test actions and interface definitions. They represent a kind of product that automation engineers provide to testers; this product must be of high quality. It is necessary to have at least one folder in each project (in the TestArchitect test tree) with test modules intended for testing actions and interface definitions, and not for the system being tested.
Like the test development process, automation work must be well planned and organized with the participation of experienced professionals. In this case, a combination of careful test design planning and automation planning will help to successfully meet all the requirements for Agile projects.
November 22, Hans Buvalda arrives in Moscow with a master class " Five key factors for successful test automation ."
More articles from Hans Boovalda can be found here .
For his professional career, Hans Buwalda (Hans Buwalda) has gained vast experience as a software developer, manager and chief consultant at leading companies and organizations in different countries of the world. His proposed testing methods (action-based and soap opera-like) have helped many customers develop scalable and easily supported solutions for a large number of complex test tasks. Hans often speaks at international conferences. He also co-authored the book Integrated Test Design and Automation .How does functional testing automation fit into Agile projects? Today we are constantly faced with this issue. Agile's strategy is becoming more widespread, but functional testing is usually still performed manually, because during the Agile iterations and sprints, there is simply no time left to automate it. Unlike functional testing, effective unit testing automation has become commonplace. You can answer the question by solving the following tasks:
')
- well-planned and organized test design and automation processes are required;
- these design and automation processes should be organized on the basis of individual life cycles.
In this article, I will show how an action-based testing method can help accomplish these tasks. To do this, I will briefly present an action-based testing method, and then I will tell you how it can be used to design test cases and automate them in accordance with the requirements of Agile projects.
Action Based Testing
There are many sources where you can learn more about action-based testing methodology. Here I only summarize the basic principles underlying this method.
1. Not one, but three life cycles
Typically, testing and automation tasks are included in the system development life cycle, regardless of whether it is cascading or agile. However, there are three life cycles in ABT, which, although interdependent, are planned and managed as separate entities in ABT projects:
- System development: is carried out in accordance with any traditional or flexible model of the system development life cycle;
- Test development - includes test design, test execution, analysis of test results and test support;
- Automation - the focus is on action keywords, interpretation of actions, comparison of user and other interfaces, the study of technological problems, etc.
2. Test Design
The main place is occupied by the design of tests. This is the main factor in successful automation, much more significant than the automation technology used. In ABT, it is important to create a high-quality design of high-level tests, within which so-called test modules are defined. Each test module is developed as a separate mini-project and must have well-defined content that is different from other modules.
The test module includes test objectives and action lines. The objectives of the test set the content of the test module in the form of individual verbal provisions that determine what exactly should be tested within this module.
Tests in a test module (which is arranged in the form of a table) are determined by a series of lines of action, which are usually organized in the form of one or several test scenarios. Each action line defines an action and includes a work word defining the action, as well as arguments defining the data for the action, including input values and expected results.
It should be noted that in ABT the test script does not have the same meaning as in some other testing methods. We believe that the test script is a small separate component that is not decisive for test development. Instead of creating a predefined list of test scenarios, we create a list of test modules in which test scenarios are included as a result of test design, and not as its initial elements.
Efficiency is provided by varying different test scenarios. In addition, each test case can provide a prerequisite for the next scenario, which gives a good flow of test execution.
3. Automation
In ABT, automation work is done separately from test design. For designing tests and for automation, different skills and abilities of developers are required. Perhaps there are specialists who are engaged in both, but, judging by my experience, they are not often met. In addition, these two areas imply different responsibilities for ensuring the execution of tests.
In ABT, automation engineers are engaged in automating actions and creating “interface definitions” to control the interaction of the system under test with various interfaces (user and others). To perform these automation tasks require special skills and experience.
Flexible test development
When using ABT with two separate life cycles — test development and test automation — you should pay attention to two topics that are important for including automated testing in Agile projects:
- Test design and development;
- Automation.
When using the Scrum sprint methodology, testing in Agile projects is divided into three processes:
- Testing in regular system design sprints;
- Development of tests before the development of sprints;
- Testing after completion of development.
1. Testing in regular sprints
The most common practice is the development and execution of tests in the framework of sprints. In the sprint, the functionality is determined on the basis of a conversation with the customer and user histories to identify exactly what needs to be tested. Testing can be carried out through the developed tests, similar to the ABT test modules, as well as through research and interactive testing. In addition, it will be very useful to conduct at least some of the most "interesting" interactive tests in test modules for later use.
Unit testing is extremely important in itself, but in ABT it makes sense to consider the possibility of re-using such tests and their application in various areas for testing individual functions.
By defining test modules for unit testing and assigning them to selected actions, you can easily combine them to test a wider range of values and the various elements of the system under test during or after the sprint.
2. Test development before sprint development
When using the ABT method, the use of actions, in particular, high-level business actions, allows you to develop tests that focus not on technical aspects, but on business functions. Such tests are commonly referred to as high level tests. They do not relate to user interface elements, but are aimed at business transactions, for example, making a request for a mortgage loan or renting a car.
Higher level tests can be developed at the initial stage of the project. For their development, it is not necessary to wait for the completion of the system development sprint, since in this case there is not enough time to define business functions and create suitable tests to verify them.
The number and the very need for business-level tests depend on the specific situation. In general, I would recommend the following:
- The number of business-level tests should be as large as possible, since they ensure the overall quality of the product and do not depend on changes in the system that do not affect the tested functions.
- It is necessary to use the design stage of high-level tests in ABT (where test modules stand out) to determine which functions can be tested first when conducting business-level tests and which needs can be closed in detailed tests as part of development sprints.
3. Testing after the sprints
After completing the sprints for individual parts of the system and after combining these parts, additional testing is usually required to ensure the quality of the entire system and its compliance with the requirements. In addition, additional tests may be needed to retest the elements of the system that were not affected by the changes to confirm that the new elements interact well with the old elements of the system. This may be necessary, for example, in regression sprints or resilience-enhancing sprints.
In my opinion, such “follow-up testing” is the main area in which it is advisable to have pre-designed test modules and fully automated actions. This will save valuable time, especially when the time of product release is approaching.
Flexible test automation
The term “timely automation” is usually used to describe test automation in Agile projects. When using the ABT method, this concept changes and sounds like “timely development of tests”. In any case, high-level automation plays an important role in increasing productivity and speeding up sprints.
In order to provide automation quickly and in time, the following rules should be observed:
- Creating a framework at the initial stage;
- Increased automation reliability;
- Ensuring the testability of the tested system;
- Automation testing.
1. Creating a foundation at the initial stage
Successful automation starts with creating a solid foundation for developing follow-up actions. Such a framework should provide various capabilities: performing all necessary operations on all classes of user interfaces, access to APIs, the ability to perform database queries, compiling and processing messages using the appropriate protocols, etc.
Although many technical capabilities are available using TestArchitect's tool created by LogiGear, most of our projects start with research and development related to specific technical problems of the customer, for example, modeling devices for POS-terminals, working with three-dimensional graphics for oil exploration, testing mobile devices, accessing firmware in diagnostic equipment, etc.
It is important to start creating such a technical basis as early as possible and pay close attention to it, identify and solve all technical problems. This will help implement low-level actions, which can then be used for higher-level actions, for example, in development sprints. In addition, the creation of a technical basis at the initial stage reduces subsequent risks.
2. Improving automation reliability
The essence of Agile projects is that many elements of the system being tested are clarified only when they are implemented in a program as part of iterations, such as sprints in Scrum. This primarily applies to areas where automation is particularly important, for example, when testing a user interface. These elements may undergo many changes in the design process. At the same time, automation should not become a “bottleneck”. In this case, the main thing is flexibility.
The action-based testing model is able to provide the necessary flexibility, since it allows you to hide parts in separate actions, which can then be quickly reconfigured if necessary. However, other points must be taken into account. Most often in our projects there is a problem of “timing”. Often, to perform automated actions you have to wait for the system under test to react to one or another action and be ready for the next one.
We concluded that to solve this problem, an automation engineer should make the most of the active timing capabilities. To implement it, it is necessary to set a criterion in the system under test - a predetermined maximum value. When this value is reached, an automatic transition to the next action takes place immediately. By paying due attention to these and similar issues, you will ensure the reliability and flexibility of automation.
3. Ensuring the testability of the system under test.
When preparing for automation, system developers should indicate which elements of the system under test provide convenient access for automation equipment. If such elements are identified at the initial stage and are defined as requirements, they can easily be included in the sprints.
A good example is the use of values for certain identifying properties that exist in various platforms for screen controls or HTML elements. The user does not see these properties, but they are available for automation tools. Providing such values will make it easy to automate testing of controls or other elements, and, as a rule, irrespective of any design changes.
If these values are defined in the early stages of a project, then using, for example, a tool like TestArchitect, you can create “interface definitions” and use them even before building the system under test.
Examples of such useful properties are the “id” attribute in HTML elements, the “name” in Java / Swing and the “accessibility name” in .Net and WPF. They have no effect on user experience, but are available for automation tools. In addition, the use of such properties helps to solve localization problems: the "OK" button can always be found, even if the inscription on it is in another language.
4. Automation testing
Automation should be tested. In ABT, this means that you must test actions and interface definitions. They represent a kind of product that automation engineers provide to testers; this product must be of high quality. It is necessary to have at least one folder in each project (in the TestArchitect test tree) with test modules intended for testing actions and interface definitions, and not for the system being tested.
Like the test development process, automation work must be well planned and organized with the participation of experienced professionals. In this case, a combination of careful test design planning and automation planning will help to successfully meet all the requirements for Agile projects.
November 22, Hans Buvalda arrives in Moscow with a master class " Five key factors for successful test automation ."
More articles from Hans Boovalda can be found here .
Source: https://habr.com/ru/post/336740/
All Articles