Manually or automatically: A couple of words about application testing
Automated testing is one of the most talked about topics among quality control engineers. New software testing systems appear constantly, and each new framework gets the title of the best.
However, automated testing is based on the assumption that the program is used as it was intended by the developers. If you rely only on automated tests, then soon errors will appear in the code, and at the same time you will get a weak idea of ​​the quality of the application interface.
As for manual testing, they pay less attention to it, since such a process is exhausting employees, and only an expert with a special mindset will approach the role of an executor. However, the "manual" tests are not inferior to automated ones. The point here is that the approaches have different areas of applicability, so today we will look at some of the advantages and disadvantages of each solution.
')
/ photo verkeorg CC
According to Sarah Fittje, a quality control engineer at Project A Ventures, if the test involves performing a large number of repetitive tasks, then it makes sense to automate it. A classic example is regression testing, which allows detecting serious bugs, so it is important to regularly conduct it in full before launching the final product (by the way, if you're interested, you can read the manual for QA specialists from Sarah, you can find it here ).
You should not think that setting up automatic tests will save a lot. Writing and maintaining written scripts requires certain QA skills that not all testers possess, therefore, the company will have to pay its own specialists more (however, the staff of testers will still be reduced).
It is also worth noting that it is rather difficult to implement automated tests for a large application with great functionality - this is how you risk losing the vision of the overall picture and cannot cover all the functionality with tests. However, setting up an automated testing system from the first days of the project, you get the opportunity to speed up quality control cycles and establish a consistent test of the application.
However, it is important to understand that automated testing will not be able to help identify deficiencies in the appearance of the interface and the quality of user interaction with the product. Consider the directions for the application of automation:
This type of testing is aimed at checking the front-end of the application. Without the help of automation, it is very difficult to take into account all possible combinations of user interaction with the interface in web browsers, mobile devices and other systems (it is important to understand that such a test will not be able to evaluate the UX, as noted above).
Using this type of testing, developers find bugs caused by improvements in application functions. Under the pressure of frequent updates, developers have to work at a frantic pace and sometimes make compromises, which are surely used by bugs, “creeping in” into the code of the software. Manually track down the shortcomings of each release is quite problematic.
This testing is to verify that the functional requirements have been met, that is, if the application is operating as expected. It determines how accurately the conceived functions are implemented, analyzes for compliance with the standards, analyzes the security of the solution. It takes a long time to repeat functional testing, even with a slight change in the application, so it can be expensive to perform all the actions with “hands”.
Automated testing helps to achieve high quality, but at the moment it has not completely replaced manual tests. This is due to the fact that users can perform completely unexpected actions that are difficult to take into account in automatic tests.
While the user gets used to the application, he is able to make mistakes, not complete the necessary steps, enter unexpected values, and so on. Users know much less about your product, but in the end they will use it. Therefore, client approval is the most valuable indicator of interface quality.
When the behavioral characteristics of a person and intuition begin to play a key role in evaluating the work of a program, you should pay attention to manual testing - here it is more important to put yourself in the user's place. A tester can be creative in the process of verifying functionality: come up with unexpected ideas, mimic unusual use cases — this helps to uncover critical errors that “automated testing” will not notice.
A couple of manual test cases:
This testing serves to test the usability of the application from the user's point of view. Such a computer task that is vague in its goals cannot be solved - the machine needs clear wording.
One-time testing aimed at testing one aspect of the program. There are no formally defined rules for this type of testing, it is conducted improvisationally - the tester can use any available means to search for bugs. In fact, ad hoc tests are an attempt to “guess” a possible error.
As we noted above, different types of testing have different areas of applicability. Automatic tests will not replace manual ones, but they can be used to save working time when working on large monotonous sets of actions. Manual testing is carried out for a long time, but you can not do without it if you want to achieve a high quality product. At the moment, only a combination of these approaches is able to provide high standards in relation to the functionality and usability of the product.
As an example of such an approach, Sarah Fitji cites the testing process of the natue online store, when, after successfully introducing new functions in a test project, the QA team launched an automated regression test.
“While the automated test was being performed, we simultaneously conducted manual testing of new functions, taking into account the interim results of the regression test,” says Sarah. “If bugs were found during manual or automated testing, they were corrected, and the tests were restarted.” If everything went well, then Sarah’s team repeated the test just before the final version was launched.
Another example is provided by Steven Allen (Steven Allen), an engineer at TestGrid.io. He says that in iOS, full-fledged automated testing has long been unavailable. A few years ago, Apple began to produce automation tools, but they are not perfect yet. Therefore, it is impossible to use only automation tools - you have to resort to manual testing.
Developers automate tests in order to work more efficiently and keep up with more, but you cannot use scripts to describe all possible uses of the application and take into account all errors. Some things are very easy to miss - for example, if an extra input field for a username appears on the login screen, but everything else works correctly.
“It’s very hard to write the perfect code for automated tests,” said Joseph Millar, a specialist in quality control at Lucid Software. - If the developer makes a mistake in the scripts, he risks missing a large amount of errors, whereas with manual testing these defects will most likely be detected. Therefore, it is important to use both methods when developing applications. One to save time, the second for "grinding".
PS Our other materials: Digest about clouds, network technologies and service development .
However, automated testing is based on the assumption that the program is used as it was intended by the developers. If you rely only on automated tests, then soon errors will appear in the code, and at the same time you will get a weak idea of ​​the quality of the application interface.
As for manual testing, they pay less attention to it, since such a process is exhausting employees, and only an expert with a special mindset will approach the role of an executor. However, the "manual" tests are not inferior to automated ones. The point here is that the approaches have different areas of applicability, so today we will look at some of the advantages and disadvantages of each solution.
')
/ photo verkeorg CC
Automated Testing
According to Sarah Fittje, a quality control engineer at Project A Ventures, if the test involves performing a large number of repetitive tasks, then it makes sense to automate it. A classic example is regression testing, which allows detecting serious bugs, so it is important to regularly conduct it in full before launching the final product (by the way, if you're interested, you can read the manual for QA specialists from Sarah, you can find it here ).
You should not think that setting up automatic tests will save a lot. Writing and maintaining written scripts requires certain QA skills that not all testers possess, therefore, the company will have to pay its own specialists more (however, the staff of testers will still be reduced).
It is also worth noting that it is rather difficult to implement automated tests for a large application with great functionality - this is how you risk losing the vision of the overall picture and cannot cover all the functionality with tests. However, setting up an automated testing system from the first days of the project, you get the opportunity to speed up quality control cycles and establish a consistent test of the application.
However, it is important to understand that automated testing will not be able to help identify deficiencies in the appearance of the interface and the quality of user interaction with the product. Consider the directions for the application of automation:
GUI testing
This type of testing is aimed at checking the front-end of the application. Without the help of automation, it is very difficult to take into account all possible combinations of user interaction with the interface in web browsers, mobile devices and other systems (it is important to understand that such a test will not be able to evaluate the UX, as noted above).
Regression testing
Using this type of testing, developers find bugs caused by improvements in application functions. Under the pressure of frequent updates, developers have to work at a frantic pace and sometimes make compromises, which are surely used by bugs, “creeping in” into the code of the software. Manually track down the shortcomings of each release is quite problematic.
Functional testing
This testing is to verify that the functional requirements have been met, that is, if the application is operating as expected. It determines how accurately the conceived functions are implemented, analyzes for compliance with the standards, analyzes the security of the solution. It takes a long time to repeat functional testing, even with a slight change in the application, so it can be expensive to perform all the actions with “hands”.
Manual testing
Automated testing helps to achieve high quality, but at the moment it has not completely replaced manual tests. This is due to the fact that users can perform completely unexpected actions that are difficult to take into account in automatic tests.
While the user gets used to the application, he is able to make mistakes, not complete the necessary steps, enter unexpected values, and so on. Users know much less about your product, but in the end they will use it. Therefore, client approval is the most valuable indicator of interface quality.
When the behavioral characteristics of a person and intuition begin to play a key role in evaluating the work of a program, you should pay attention to manual testing - here it is more important to put yourself in the user's place. A tester can be creative in the process of verifying functionality: come up with unexpected ideas, mimic unusual use cases — this helps to uncover critical errors that “automated testing” will not notice.
A couple of manual test cases:
Usability testing
This testing serves to test the usability of the application from the user's point of view. Such a computer task that is vague in its goals cannot be solved - the machine needs clear wording.
Ad hoc testing
One-time testing aimed at testing one aspect of the program. There are no formally defined rules for this type of testing, it is conducted improvisationally - the tester can use any available means to search for bugs. In fact, ad hoc tests are an attempt to “guess” a possible error.
Combined version
As we noted above, different types of testing have different areas of applicability. Automatic tests will not replace manual ones, but they can be used to save working time when working on large monotonous sets of actions. Manual testing is carried out for a long time, but you can not do without it if you want to achieve a high quality product. At the moment, only a combination of these approaches is able to provide high standards in relation to the functionality and usability of the product.
As an example of such an approach, Sarah Fitji cites the testing process of the natue online store, when, after successfully introducing new functions in a test project, the QA team launched an automated regression test.
“While the automated test was being performed, we simultaneously conducted manual testing of new functions, taking into account the interim results of the regression test,” says Sarah. “If bugs were found during manual or automated testing, they were corrected, and the tests were restarted.” If everything went well, then Sarah’s team repeated the test just before the final version was launched.
Another example is provided by Steven Allen (Steven Allen), an engineer at TestGrid.io. He says that in iOS, full-fledged automated testing has long been unavailable. A few years ago, Apple began to produce automation tools, but they are not perfect yet. Therefore, it is impossible to use only automation tools - you have to resort to manual testing.
Developers automate tests in order to work more efficiently and keep up with more, but you cannot use scripts to describe all possible uses of the application and take into account all errors. Some things are very easy to miss - for example, if an extra input field for a username appears on the login screen, but everything else works correctly.
“It’s very hard to write the perfect code for automated tests,” said Joseph Millar, a specialist in quality control at Lucid Software. - If the developer makes a mistake in the scripts, he risks missing a large amount of errors, whereas with manual testing these defects will most likely be detected. Therefore, it is important to use both methods when developing applications. One to save time, the second for "grinding".
PS Our other materials: Digest about clouds, network technologies and service development .
Source: https://habr.com/ru/post/311896/
All Articles