Tours in exploratory testing. Personal translation from D. Whittaker's book "Software Research Testing"

Tourist metaphor

Imagine a typical tourist visiting a tourist town. There is something to see here. Here is what to do. Let the difficult "work" begin! It is the same with test engineers who aim to fully explore their city - a software product.

There are a lot of questions. What transport to take? How to meet the time? How to get through the route? How to behave in the absence of a good place for selfie? Do I need to deal with it myself or call a police officer?

Strategy and goals. If they are absent, it is better to stay at home. Goals essentially determine the plan of the tourist: time and place. And they will be completely different, for example, for sailors who have been sailing for half a year, and students ... Well, maybe not the best example.
')
After all, how can it happen. Without a plan and strategy you can meet interesting places. Sometimes interesting places are hard to miss. But what are these places, what value do they have, what is their story. This can be missed. If you drive the subway without a map, you may not realize at all what distance was covered: 5 stops - is it a lot or a little? Cognitive psychologists would say: “Your cognitive map is not adequate,” and neurophysiologists would invite you to study LCS (this is a safe method for studying the brain). For a test engineer, such a map (coverage) is dangerous.

For Whittaker, this is a typical description of how testing happens. This is a freestyle. And, perhaps, there will be no other chance to visit these wonderful places. This will be the first and last time. That is why you should avoid aimlessness in the study and risk the fact that we can skip the essential functionality or an important bug.

But do not write off freestyle testing. It acquires significant advantages in conjunction with the scenario approach, since the scenarios do not always describe all the interesting “lanes and alleys” (for seafarers and students this is very important).

In this sense, tourism is a combination of Kantian freedom and Tatlin (and for some people a Fuller) structure. The same is true of testing.

There are many tourist metaphors that help us in research testing and complement the freestyle. This allows us to make testing faster and more consistent. Here they are:

No, not yet. It is important to understand that the functionality rarely works independently of other functionality. They share resources, process common input, and work on the same internal data. This means that testing the functionality separately can eliminate the errors of their interaction.

To our common happiness, tourist metaphors do not deal with such a decomposition. They decompose, but in a different way. A real tourist will choose a mixture of marks to be seen and places to visit. The test engineer will do the same: make a plan to visit various features with the intention of doing some appropriate (this product) operation. And, of course, it can involve a lot of functional blocks.

Tourist metaphor in testing - these are areas. For software, this is a logical separation, these are ways for the application. Here are these tours.

Tours

1. Business areas

These are the places where business is done. They start with running the code and ending it. They contain functions for which users use this application. This is the “back wall of the box” (read applications), which is central in demonstrating the commercial value of the product, as well as the code that supports it.

2. Historic areas

This is a favorite tourist destination in historical places, places surrounded by history. This includes the “legacy” code and functions and features that historically contained a large number of errors. "Legacy" code is often poorly understood. Its use and modification implies a large number of assumptions. This tour is focused on this component.

3. Tourist areas

In many cities there are places that are interesting only for tourists and novice users. Local guys avoid such places because they are crowded there. That is, users who already have experience in interacting with the application almost do not use these functions.

4. Entertainment areas

After exploring historical and tourist places, it is sometimes necessary to have a rest, a little fun. The software also has something similar - additional functionality. And you need to check it out.

5. Hotel areas

In each tourist city there must be places where you can relax, rejuvenate, wait out bad weather. But the software does not rest at all during the moments of rest. And it would be good to check what and how it does.

6. Rough areas

This is an unsafe area, which, however, is of some interest to someone. There are bad things happening, and, apparently, it would be better not to go there. Everyone except test engineers. These sites should be tested as they may contain product vulnerabilities.

1. Business areas

From morning to evening, work is in full swing here. It is here that “business is done”. And if you ask why users use your application, they will tell you about this part.

Guide Tour

Travel guides often contain a lot of information about places that can be viewed. They talk about the best hotels, shopping malls and attractive places without detailed details. The experts who visited these places, will kindly tell the tourists exactly how to enjoy these places. It should be beautiful, clean, safe, so that tourists can safely spend their money. These are very important places, and they should be included in the test strategy of research testing. We need to ensure that users get the benefit and pleasure of working here.

A variant of this tour is the F1 tour. Namely, strict adherence to the user manual provided (if provided) by the manufacturer, which describes the key functions. The goal is to go through all the scenarios that are described in the user manual, as fully as possible. Often, here you can find various tips on input data and navigation through the interface in the process of using functions. In addition to the functionality of this tour allows you to check and how accurately and accurately compiled user manual.

A variation of the tour is also a blogger tour, which contains third-party advice, as well as an expert tour, which reviews feedback from dissatisfied users. Various forums, news, communities or even books (in case the application is large) can help. Another useful tour is a competitor's tour, which allows you to consider an application from a competing system.

The above options of the tour of the guide allow you to evaluate the application in the context of its advertised functions. These are straightforward tests, and any deviation from the manual should be perceived as an error. This tour allows us to use the functions in the order and interrelationship in which the users themselves will do it.

Money tour

Each application has functionality that, in the eyes of the user (customer), will bring them money. “This is the most attractive for me,” the user will say in this case. “Monetary” functionality carries with it “monetary force”. This includes various nuances of using the product, since it is they who make money. Sales professionals can direct a product demonstration towards the specified scenarios, affecting what was not clearly documented in the product documentation. Such a “selling vision” is an excellent basis for a cash tour.

In the specified tour, it is important to hear how sales specialists present the product, and walk through exactly what the emphasis is on. During such testing, you not only find errors, but also preserve the vision of the “cash vein” of the application.

Variety of money tour - skeptical customer tour. Walking around the tour, you suddenly stop and ask: “What if I do like this?” Or “How can I do this?”. “What if I now complement the passage with such scenarios that go beyond the originally conceived? I’ll take a few more features here! ”

Once again: observation of product demonstrations in the context of “monetary value”, as well as good relations with sales specialists, provide an excellent opportunity to conduct cash tours much more efficiently.

Tour by marks

We choose any reference points and jump to one of them through the entire application. For example, we select the key function as the mark we defined in the guidebook tour or in the cash tour. Next we compose a set of such marks, determine their order and examine the application from mark to mark until we reach each of them. You can then create a coverage map with tick marks. Different order, different marks - this is the field for the variability of such a tour.

Intellectual tour

Ask your application complex, intelligent questions. How can you make an application work as hard as possible? What features will bring it closer to its own limit? What input and input will make it strain? What data can fool error checking procedures? Obviously, this directly depends on the features of a particular application.

Option - tour arrogant American who uses stereotypes about foreigners. Ask stupid questions, try to annoy, draw attention to yourself. Think of obstacles and see how the application handles them. Does it make sense? Maybe not. But we can do that.

Intellectual tour and its variants can help to find both really difficult and priority mistakes, and stupid mistakes. You need to be able to distinguish them. To do this, it is necessary to create real cases in order to make it easier in the future to argue that this is important and that it needs to be corrected.

FedEx Tour

Think about the data that must go through the app. The data begins its journey with input, then stored in internal storages and variables, where they are often processed and modified, and then they can be subjected to calculations. At the end, most of this data is provided to the user or anywhere else.

Need to concentrate on the data. You need to identify the saved data and "trace" their path through the application. Here you entered, for example, your email address. And where is it displayed? In what places? Try to find all the areas with which the data is in contact. Determine their entire life cycle.

Tour "after work"

After a monetary activity of the application, it still continues to work. This support tasks, archiving data and files. This may be an automatic process, but it can also be enforced. This tour reminds us to do it.

Variation of the tour - the morning inclusion. Task - testing initial procedures and scripts.

Janitor's tour

Cleaners are guys who know the neighborhood even better than living or local cops here. Day by day, street by street.

We can plan places for a methodical interface check: screen by screen, dialogue by dialogue (mainly, by a short way) without detailed testing. Only obvious places.

This tour includes the selection of a goal (all menus, errors, dialog boxes, for example) and a visit to each of them in the shortest possible way.

2. Historic areas

Such areas represent legacy code, features introduced in older versions, and bug fixes that still need to be checked.

Tour "bad neighbors"

Bad neighbors - those sections in which a large number of errors is found. How can these sections be defined? No advance. But on the experience this is done on the basis of already discovered bugs and their number. There is an opportunity to make some assumption about the places of their appearance. Since mistakes can accumulate locally, this tour is worth taking. After working with such errors, it is recommended to go through the janitor’s tour through nearby functions to make sure that the corrections did not affect them.

Museum Tour

Museum antiquities here - legacy code. It can be determined by the dates specified in the repository. Such code, used in the new environment, is prone to poor performance. It is difficult to investigate this code: the programmer may no longer be in the company, and the documentation may be scarce. The tester should identify such a code and pay attention to its work.

Tour of the old version

After updating the previous version, it is advisable to conduct tests that were written for the previous version. This will make sure that the functionality that users used previously can still be used is useful in the new version of the product. In the event that the functionality has been removed or updated in the new version, testing should be done in the appropriate way.

3. Tourist areas

This tour concentrates on several points:

- short, fast test cases;
- longer script checklists;
- talk not about how to make the application work, but how to quickly check the functionality ... and say, "And I was there!".

Collector's Tour

Collect output, and the more the better. The idea is to go where you can and document everything that you see. Make sure you see all the output that the application can generate. For a text editor: print, spell check, format, various structures, tables, graphics. For the store: the possibility of buying from any valid department, successful and unsuccessful card transactions. You need to pursue all possible imprints until you can confirm that you were everywhere, saw everything and completed your collection.

This is a long tour and it would be better to go through group activity.

Tour lonely businessman

The idea of ​​the tour is to check the furthest corners of the application, which are possible. What feature requires the most clicks? Choose such a "click" path to it and test it. What feature requires the largest number of screens so that it can be used? Select and test. The idea is to travel as long as possible to the final destination.

Supermodel tour

For this tour, think superficially, do not go beyond what is outside. Take the lessons that modernity teaches us. This tour is not about functions or something significant, but about how the application looks and what the first impression makes.

Focus on the interface. He is good? Well done? If I make changes, is it displayed on it? Is this happening correctly and are there any artifacts on the screen? Is everything in its place? Does it violate any convenience requirements or standards?

Perhaps this should not be touched, but what a damn good look.

Tour "test one, the other - for free"

This tour in order to test the multiple launch of the application at the same time. Start your application, and then again, and more. And now use features that relate to memory or disk space. Make all copies of the application do something with the same file or transfer any data.

Scottish Pub Tour

This is especially true for large applications. There are places in them that are quite difficult to find without any help. They are not difficult to use, they are just hard to find. Such places can be really a lot. The problem is to find out about them. How? Chat with users, browse any blogs and just dive into the app.

4. Entertainment areas

This is an opportunity to have a little fun with the app: make it more attractive, play with patterns, colors. It gives some break in hard work. The connection between the backbone and the supporting side of the application is important: it must be useful and meaningful.

Tour actor from the support team

Whenever salespeople demonstrate a product, the user is prone to being tempted by the features that lie next to those that are highlighted. The tour of the support team actor focuses on those features that share the screen with the main ones. Their proximity to the cores increases their visibility, and we need to pay them proper attention. Shake your head - right and left - and see if they paid proper attention to the actors who support the major ones.

Tour of the back alley

Tour “behind the scenes”. All that receives the least attention, that which is used less often, that which is used most rarely. The very bottom, the farthest shelves, what is behind the facade.

An interesting variation is a mixed-use tour. Mix the most popular and least popular features, visit them. It's like a mark tour with a mix of the biggest and smallest marks. It may happen that the features will interact in a way that was not intended by the developers.

How to determine that features can interact with each other? Whitaker offers three questions:

- The question of entry. Are there input data that is processed by two features in one thread?
- A question of exit. Do the functions operate on the same parts of the graphical interface? Do they generate or update the same output?
- Do the functions operate on the same internal data? Do they use or modify the same internal data?

Tour "All night long"

Club tour - never stop, another club, another cocktail. Can you survive all night?

You can, and the application? How long can it work and process data before it rolls up? The accumulation of data in memory, writing (rewriting), repeated reading of variables - this can lead to application crashes: memory leaks, data breaches, race conditions, and more. The main condition - never close the application. Open files and do not close them. Do not save files. Connect to remote resources and do not disconnect. And while the above conditions apply, you can go through other tours for the application and data processing in it. When you restart the application, some of its sections are cleared - do not allow this.

Not seldom, such a tour takes place automatically, looping the script. Long work is especially important for mobile devices - these are their usual conditions of use.

5. Hotel area

This is a place that is far from the hustle and bustle, a place for easy rest and relaxation. This is a place where you can move away from the main functions of the application, popular features, and test secondary, supporting functions that may not be reflected in test plans.

Rain tour

The idea is to start the operation and then stop it abruptly, as if it was raining at that moment and you don’t want to go anywhere at that time. Start entering your purchase information in order to immediately complete this procedure and exit; start printing and cancel it before it prints. If the functionality has a cancel button or if it runs for longer than a few seconds, use this tour.

Look for labor-intensive operations - there is a place to play. Search engines are a great example: set the words that load a search, make it long. There is a button "Back" - click. No - press ESC or even the “Back” button in the browser. Start the operation, and then again without stopping the first one.

The reasons for possible errors are that the application cannot clear the storage after it. Something may remain open - files, data, variables. There may be a condition that contradicts the normal flow of the application. So after various cancellations, go through it and see if everything is in order. Make sure that all canceled procedures can also be run correctly.

Tour couch potato

There is always someone who crosses his arms over his chest. He is bored and shows no activity. However, it often happens that the guide tries more than usual to interest such a person.

The same applies to applications: if a person does not show high activity, it does not mean that the application is also inactive. On the contrary, often this loads the application, since the else command is used in the if-then-else construction. The application has to figure out what to do if the user leaves the fields empty. In moments when the user does not take the initiative, "default logic" is executed.

The idea of ​​the tour is to make as little effort as possible. Leave the fields blank, accept the default values, fill out the forms with the least amount of data, do not click on anything extra, skip forms without clicks, or submit whatever you like there. If there is a choice, choose the path of least resistance.
Again, lazy user interaction does not mean that the application is lazy: it must substitute default values, process empty forms, etc.

6. Rough areas

Input must break application.

Tour saboteur

Try to break the application in any way possible. Ask the application to read any data from the disk, and then sabotage this attempt, damaging the operation and the file. Ask for an operation that uses memory intensively when the application is installed on a machine with a small amount of memory, or when another application that consumes a large amount of resources is running at the same time.

The concept is as follows:

- force the application to perform an action
- Determine what resources are required for this action.
- remove or limit these resources.

You will understand that there are many ways to weaken the application, adding and deleting files, changing access rights, disconnecting the network cable, running other applications in the background, deploying the application on a machine with known problems, etc. Also useful is the concept of fault injection (here, in Whittaker, a reference to his other book, How to Break Software).

Antisocial tour

In this area there are attempts to destroy the application. Being nice, polite and following the crowd is not the best way to do it.

This tour includes the requirement to enter the least attractive data and / or negative input. If a typical user does "A", this tour implies that the test engineer will never do "A", you need to find something more meaningless.

Three ways can be distinguished in order to realize this antisocial behavior:

- Tour of the opposition. Enter the most incredible data that is out of context, stupid or completely meaningless. This is where the error handling capability is tested. If this helps, you can check the "patience of the application."
- Tour of crime. The idea is to provide data that should not appear at all ... well, should not. Don't you expect a tourist to steal a pint of beer in a pub? These are not just antisocial acts, they are crimes.

Breaking the law, the tourist will get into trouble. Breaking the law, the test engineer may encounter error messages. The data in this tour causes error messages, and if not, you have found a bug. Enter incorrect data, wrong format, too long, too short, etc. Think "What restrictions are associated with such input?", And then break the restrictions. “Want positive values? So on you negative. And pay attention to errors, they may be needed for other tours.

- Tour "Turn the wrong way." The idea is to do something in the wrong order. Take a set of correct actions and perform them in an incorrect order. Try to place an order before putting something into the basket. Try to return the order before you bought it. Change delivery settings before the purchase is completed.

Tour of obsession and even obsession

In this round, you need to enter the same data again and again, to perform the same action again and again. Copy, paste, cut, replace, and then do it again. The name of this is usually rehearsal. Select a product, and then again the same product to check whether discounts appear.

Developers often think that users will use the application in the right order, purposefully. But users make mistakes and come back; they don’t always understand the “right” way and build their own.

Using tours

Tours bring structure into testing. They help test engineers discover more relevant and interesting scenarios than with freestyle testing. By providing test engineers with targets, the tours push them to a certain path, which is more difficult than classical testing based on the decomposition of functionalities.

Tours - a tool for learning the features in their combination. In addition, tours are something that is fixed, they can be repeated. “You have such a tour, and you like this” - this already excludes many problems of the intersection of test engineers.

Some tours will be more successful, some less. Empirically it can be determined. And then they can be ranked in various ways (by errors found, by runtime, by code / feature / interface coverage). Tours can be managed, and this is important. You can learn from them. They can be distributed in the team. You can determine their effectiveness. They help to give structure to testing and knowledge.