Introduction to frameworks (Part 1)

Automated testing (AT) is most effective when implemented using the framework. Despite the fact that in AT, the term framework is often used to describe the set of objects that forms the unit testing tool, this article will be mainly focused on other types of frameworks. We will discuss the types of frameworks that can be defined as a set of abstract concepts, processes, procedures, and environments through which automated tests are designed, created, and implemented. In addition, this definition of the framework includes physical objects used to create tests and their implementation, as well as to organize the logical interaction between components.
Automated testing (and, therefore, frameworks) has evolved over the years, evolving and becoming more complex with each new phase of evolution. These phases can be described in terms of three generations, each of which has a set of disadvantages and advantages, due to which each of them remains relevant, despite new developments. The concepts below are commonly used to automate functional testing, but in some cases they can be used to solve unit testing problems.

First generation frameworks

The first generation of frameworks in test automation is initially based on a linear approach to the development of automated tests. A linear approach usually results in a one-dimensional set of automated tests, in which each autotest is considered simply as the implementation of its “manual” equivalent. All components that are used by the autotest are generated for the most part using the “record-play” technique and are mainly located inside this autotest. At the same time, there is practically no modularity, code reuse, and other elements that determine software quality in scripts. The scope in which such tests can be useful is quite small.

Linear frameworks

An example of a small linear script is presented below. Please note that there are no calls to external modules and access to external data.

1. Input “John” into Username textbox
2. Input “JPass” into Password textbox
3. Click Login button
4. If “Welcome Screen” exists then
5. Pass the test
6. Else
7. Fail the test
8. End if

In most cases, the use of the “record-play” technique cannot be considered as a framework or a serious approach to automation. Nevertheless, many would agree that this technique may in some cases be useful.
')

Advantages of linear frameworks

• The speed of creating scripts - not much planning is required, little attention is paid to the quality aspects of autotests as software. Considering this, as well as the fact that the development of linear scripts is often based on the use of the “record-play” technique, it is easy to understand that the cost of such development in terms of time, resources and money can be relatively small.
• Short learning curve w - test automation usually requires knowledge of the automation tool, the programming language, and how the tool interacts with the application under test. Using the “record-play” technique when creating linear scripts automatically generates code that corresponds to the actions that must be performed in the application under test. This code can then be studied by the automation to get answers to questions about the syntax of the automation tool language, as well as how the tool interacts with the application.
• Quickly obtaining information about changes in the application under test is an advantage closely related to the previous one. Using the record-play technique provides information about changes in the application under test. For example, how properties of objects change after actions to change an application.
• Script independence - given the fact that all the components used by the script are inside the script itself, there is no danger of changing the script in such a way that it accidentally affects the work of any other scripts.
• Simplicity of error detection - the more advanced the framework is, the more complicated it is. Together with the complexity come the problems associated with finding the source of the error, especially for those who are not familiar with the structure of the framework. Given the fact that all the components of a linear script are in it, finding out where the error occurred is not a problem.

Disadvantages of linear frameworks

• Incorrect reproduction - in the case when they mainly rely on the “record- play ” approach, the recorded scripts are often not reproduced correctly. This is due to the fact that this approach is not analytical. At its application, the application’s objects and its behavior are not investigated, the decision is not made on how best to interact with these objects and behavior. When trying to fix playback errors, a series of temporary solutions is often used, which then fail the test of time.
• Redundancy - linear scripts do not have the advantage of reuse, so if several scripts perform similar or similar actions, then the functionality will be duplicated in each of the scripts. In case of a change in the tested application in subsequent releases, we will receive as a result an increase in script maintenance costs.
• One-dimensionality - low flexibility is attached to the load to linearity. Scripts can be easily run on one level only, in one place and in one way. If the test manager wants the automator to perform part of the tests based on certain priorities or risks, or in a different environment, or in a different sequence, then the automator will have to spend a lot of time analyzing the scripts to perform the required. Consequently, the time available for running scripts or analyzing results is reduced.
• Scripts are difficult to read - due to the lack of reusable components, linear scripts are overloaded with code. This makes scripts difficult to read and, therefore, difficult to analyze and difficult to maintain. In addition, when changes need to be made to the script, it becomes more difficult to determine in which part of the code this should be done.
• Requires a higher level of knowledge to support - this is directly related to the lack of division into modules. Modularity helps to deal with the question for which each of the code blocks is responsible. Free declarations in linear scripts make this process not so easy. So, when it comes to analyzing or debugging a linear script, the automator must first spend a significant amount of time understanding the code in order to effectively support the script.

Part 2.