Disclaimer

Reading the book on automated testing found a rather interesting description of some of the ways that are used to automate the process. Since the book is written in English, I present my free translation of one part. When reading a book, I often recalled my path to automation. This article discusses what cannot be automated in any way.



Arbitrary, spontaneous (I think this is a more accurate translation of Ad hoc) testing implies simply sitting in front of a computer and trying out different things. The tester may or may not have a test plan or checklist. The tester thinks that such a test and just clicks here and there trying different scenarios, values ​​and thinking “What will happen if you do it like this.” All the ideas and actions of the tester are not documented and are more spontaneous. And some scenarios can not be later reproduced.
This state of affairs usually happens when a software development project is launched late, little time is devoted to planning a test. For such a situation, the lack of specifications is typical, the requirements are still in development and constantly changing, and the PM says: “There is no time for writing, just test it.” Work in such a project is not particularly pleasant and deserves sympathy.

Often, the automation of this type of testing is as follows:

1. We think what to do, what to test
2. We think over specific inputs
3. Enter the data just invented
4. Check that the program works correctly by watching the answers that appear on the screen.

It’s hard to come up with many advantages of spontaneous testing. The only argument that is mentioned is that it should save time (we do not waste time on planning, design tests, we are just starting to test). This is an analogue of the syndrome “Why is it still not programming?”, Which characterizes the simplest software development. This approach is always more expensive for long-term development (and in the development and testing itself) because much more mistakes will be made, and they will cost more to solve them.
')
The disadvantages of this type of testing include:

1. Many parts that need to be checked can simply be skipped.
2. Some parts can be tested more than necessary.
3. Tests are not repeatable, the solution of errors cannot be reliably verified (in some cases, errors may simply not be reproduced)
4. This is usually inefficient and unproductive.

Automating such testing depends on the tester's decision on what to check. This means that the tester must have the necessary knowledge to make good tests. Otherwise, it depends on the individual design and implementation of the tests without any independent testing of the quality of the tests.