The point of testing is in the process, not in the remaining artifacts. Michael Bolton and Rapid Software Testing

The IT environment has its own legends, whose names are known today by almost everyone and whose (more important) achievements in the profession have shown others a new path to development. One of such figures for the world of software testing was and remains Michael Bolton , whom we are waiting for at the near Heisenbug 2018 Piter. In this article we will talk about Rapid Software Testing , about Michael and his reports.

Having 20 years of experience in testing, developing, managing and writing software, Michael not only does not get tired of teaching others how to do testing most effectively, but also develops new approaches to the seemingly familiar art of software testing. He flew more than a million miles on an airplane, having spent hundreds and hundreds of consultations, lectures, training and master classes, and is still full of energy to instruct specialists all over the world on the right path - to carry out testing quickly and efficiently.

Michael, along with his friend and colleague James Bach, is the co-author of an excellent methodology and set of views, which they call Rapid Software Testing and which I decided to talk about today. Rapid is not about how to quickly run a test suite, no. Quite the contrary, talk about removing the routine and feeling of an endless stream of new bugs from the work of the tester.

The authors of RST describe it in the following way: it is a set of approaches and skills, honed to do testing faster, cheaper, and with excellent results. This is a methodology that takes into account how people learn and organize themselves under the pressure of circumstances. RST comes from interesting packages:

1. The process of creating a software product is a relationship between people, both emotional and rational.
2. Each project is under conditions of uncertainty and pressure of the time factor.
3. Despite our best expectations and intentions with respect to the project, some degree of inexperience, carelessness and incompetence is normal.
4. A test is an activity; its meaning is in the process, not in the artifacts.
5. The purpose of testing is to identify the status of the product being produced and evaluate what may threaten its usefulness, so that our customers can make informed decisions about it.
6. We are committed to conducting reliable and cost-effective testing, and we will inform our customers about something that threatens this commitment.
7. We will not consciously or carelessly mislead our customers and colleagues.
8. Testers are responsible for the quality of their work, although they cannot control the quality of the product.

Thoughts interesting, but even more interesting to get acquainted with the arguments leading to them. They are simple, but sometimes they force you to take a fresh look at the usual, in particular, the very understanding of what is the goal of our work.

Why, in principle, conduct software testing? No matter how we want to say that we do it in order to reduce the number of errors in each successive release of our program, the correct answer sounds different: with the help of testing, the project management receives an assessment of the risks that the software being developed does not behave as expected. As a result, we are not testing with testing so that there are no errors in the program at all, but we estimate how much worse it will be for us and users if we release a product with the current number of shortcomings and bugs.

Many people perceive software testing simply: a small test is written for each function of our code, making it possible to make sure that with some specific input values, the output is exactly what we expect to receive there - and now we are sure that the whole project does that should! Oh, it would be good, but no, it does not happen in life, especially if the project is large.

For a start, the larger the project, the more checks there will be. Very soon, with the growth of the code, we will find that there is not enough power to cover it all with tests, and some tests, as we decide at that moment, will not have to give up small things, but put them aside for later ”(This doesn’t make our strengths and hands any longer, we’ll just get not one, but at least two lines of work:“ important ”and“ do it when there is time ”- and the debt in the form of this second line will be accumulated and saved). We get an endless battle, the result of which is clear in advance: to embrace the immense will not work (although we can certainly dream that one day we will do it). At the same time, not every code can be tested in isolation, and bugs can get out in real working conditions, even with a fully successful test suite.

Further, sometimes it may turn out that the checks written according to the available documentation for the project (and the project code) refer to the already irrelevant version of the code — that awkward moment when “the code was corrected, and the document was not updated”. As a result, a specific test will “glow green”, but in fact it does not make sense to trust the result of the test. In addition (even if we will not wait for the documentation update), a correctly written and relevant test today may become irrelevant tomorrow, when the code under test will be completed / redone, and the test will be forgotten, and it will not be able to answer any reasonable questions. about the quality of the code being tested.

Tedious? You can also recall the bureaucracy, which eventually gets divorced around the results of a more or less extensive test coverage. Unsuccessful tests (especially automated ones) lead to the creation of tickets, which, in an amicable way, also need to be investigated - although it is not always clear whether this makes a lot of sense.

This is me to the fact that even in the bright matter of reducing the number of bugs in software, you can find many controversial points, which over time create a certain entourage of activity, but do not bring any benefit. Such a “motion for the sake of movement”, giving the impression that everything is under control and everything is under control, it is not useful for the project in terms of increasing the quality of the code, nor for its employees regarding their personal development.

The most important thing to keep in mind is the idea that testing is not at all about the quality of the code, but about estimating the probability of getting this or that problem with this code. Based on this assessment, the project management can make more reasonable decisions, and it is for this, generally speaking, that it is ready to pay testers for their work. By the way, we can note that for a probabilistic determination of the presence of problems in a project, an exhaustive (full) test coverage is not at all necessary; it will only affect the credibility of the assessment obtained.

I perfectly understand testers who work systematically, day after day covering still uncovered pieces of code with checks, constantly chasing autotests, dealing with “reddened” checks and triggering tickets for every suspicious red tag: in other projects, it is only that you can save your soul equilibrium. The only trouble is that as the volume of the code grows and the number of checks grows, the whole variety of tests is not completely banished, so the feeling of the dynamics of the project’s life somehow becomes dull. As a result, we have a feeling of endless struggle with windmills.

What does Michael Bolton offer us Rapid Software Testing? To begin, instead of full coverage of tests with these endless runs of a large number of checks, we turn to quick testing of the next place, which we, by virtue of work experience, knowledge of the project and some professional flair, see as containing potential problems in the code. We try to make the check short (in time), non-resource-intensive (in all senses), so that its results can be trusted and so that these results can be presented in an understandable way. Perhaps, at the same time we will move from automatic start of tests to manual testing and make the number of checks less, but we will check in those places where they are really needed, where, as our experience tells us, the most likely serious bug may happen.

And here we come to the fine line separating the worlds of developers and testers. The motto of the RST in this case is communication, communication and again communication. The idea is simple: if team members do not share with each other knowledge about how the project is developing, testers are much harder to keep track of what and how to check, since they can only work according to constantly outdated documentation. And we all know how sometimes the documentation can lag behind the real state of affairs, even if you really want to minimize this gap.

At first glance, RST does not look like an effective methodology: too much of it is built on the belief in a living person, and we, having spent half our lives behind the terminal, today have lost the habit of doing so. But it is worth thinking a little detachedly, and a lot falls into place - enough to understand why the methodology has been living and being used for years.

Putting a hand on the heart, let's say: a square-cluster approach to testing, clearly visible when creating the traditional full test coverage of the entire code, works well for proving one's own participation in the process, but does not really affect the result. In other words, we need to choose: we try to be “in business”, or we try to help the project. And the decisive factor of choice, I would call the idea that in IT (as in life) you can not stop on the spot; you either grow or lose in knowledge - and, as a result, in the prospects for further movement. Another thing is that you can mechanically cover everything with tests, not really including your head, and with RST you will have to think and think intensively, but the result is worth it!

Remember the Cheburashka from the cartoon, when he gave a speech at the completion of the house: "We built, built, and finally built"? The authors of the RST represent the development team going towards one common goal (which, of course, is more often true; It is puzzling, but at the same time, hope that he himself would like to get more involved in the work process - and together with the others, bring closer the moment of rolling out the next version of the software.

There can be no “right” and “wrong” testing. What is suitable for a bank will not be suitable either for software development in the military industry, or for a small software company, so one can speak of one approach or another only in context, from which we can obviously conclude that it is impossible to isolate the notorious “ best practices ”for testing in general - you need to choose techniques and approaches that are consistent with the situation and environment. As an example: so popular among start-ups, Agile is quite agreeable in the matter of testing meticulousness, believing, on the contrary, that the code is in any case “more alive than dead”. There is even such a joke: Agile was invented by dismissing all testers from the project. Well, the logic of the approach can be understood - Agile was made by developers for developers, with no goal of improving code testing. The reasons for this are rooted in the culture of start-ups: where a tester will doubt the efficiency of his work, an entrepreneur who has invested money in a startup will say: “Ok, so be it, but it can also earn!”

What is really needed for the success of the RST implementation (and, according to its views, for subsequent improvements in the life of the project and its team) is the experience, knowledge and passionate desire on the part of the participants. To clarify: talk about real experience, and not about crusts and certificates of passing courses and exams. The latter in no way guarantee that, after passing an exam of 40 questions with ready-made answers that do not involve performing any real work on testing, a person will be able to turn on intuition in any life situation. Undoubtedly, it is possible to put on testing for almost any person who is able to perform actions according to the standard algorithm - but there will not be much benefit to the project from such checks.

A special way of thinking is important for RST: uncertainty in the obvious, the habit of asking questions about things that seem clear and unchanging to others, automatic addition of “what if?” At the beginning of each phrase we read or hear, the ability to see complexity behind apparent simplicity, search for alternative interpretations of everything we encounter. This is a kind of internal belief that bugs can be in everything; if the tester thinks differently, it will be difficult for him to do his job.

Michael Bolton speaks modestly about himself: “I teach people how to answer the question more effectively whether the product they really received is what they really expected to receive.” In fact, his field of knowledge (and the involvement of listeners) is much wider, and it is thanks to him that many realized the whole scope (and elegance) of the profession of the tester in principle.

This time, Bolton arrives with two reports: “The logic of verification” (about the verification phase in the testing process and about avoiding wrong assumptions and false conclusions) and “Testers as their own worst opponents” (about how testers can raise their reputation and the image in companies where the attitude of developers to them has traditionally been from a position of excellence). We invite you to visit the Heisenbug 2018 Piter to enjoy the communication with this wonderful speaker.