Once again about the unbearable ease of testing

We continue the conversation.

In my recent post I gave a few examples of how a tester can sometimes miss a defect that he had almost in his hands. By thoughtlessness, inexperience, naivety, fatigue, in a hurry - it does not matter. Himself. The programmer and the general development culture can, of course, help - they say, this is not a bug, but still, in the cases described, their role is auxiliary.

mtikhon in his article “Easy way to get tested” perfectly complemented that list with “external” problems affecting the test result. External - in the sense that they originate not in the testing department, but in other divisions, and even more often - somewhere at the junctions of the divisions, in the interaction of departments. (I understand that a special department is not always formally allocated for testing. But this is a cosmetic difference, in fact it does not change: here it’s rather a matter of separation of roles)
mtikhon 'y chided a bit in the comments that the list of problems was set out, but the easy way to get around them was not. He, in turn, has rightly noted that "methods tend to vary greatly." It is on this thought that I want to tread a little more detail.
')
Perhaps, I will go directly on the same points.

1. The tester should be aware of changes in the product.

Actually, the solution here is set out in the title itself: find a way to notify employees about important changes in time, and the project’s life will become easier. I would even add: not only the tester, but also the programmer, and the analyst, and all the rest - they should also be aware of, yes. Especially if we are talking about a distributed project, in which the work of one team significantly affects the work of another.

The first problem is to find a working way for such alerts.

On the one hand, all the information is already there - everything is recorded in the task tracker. In addition, the version control system also sends change messages - the most accurate and detailed. On the other hand, it is clear, in general, that a lot of garbage messages pass through the task tracker, and no one ever reads something that does not apply to his work. And the messages from the version control system are, well, still too detailed and atomic, and even in a not quite human language. Finally, one interesting change for all can correspond to several related tasks and commits.

Well, let's say we agreed when and about what changes (in business logic, architecture, etc.) to notify, and even somehow decided how much detail these changes to describe in the notification itself (who is interested in the details - will clarify).

The second problem is how to deliver these notifications.

You can get up and say out loud. You can write to everyone a letter. You can put on the wiki or some other internal resource. You can even come up with something exotic. Ideally, of course, it would be immediately added to the right part of the brain to the participants of the process, but for now it is impossible. In general, it is worth a couple of minutes to think about how to be. There is no need to think long, it is better to start at least somehow; and then you can adjust. Moreover, there is a third problem.

As they say, we can bring the donkey to the river, but even Allah will not force him to drink water. That is, no matter how I was informed about the changes, there is always the possibility of not knowing about them: leave the room, do not read this spam, especially those who flew in from another city, etc.
That is, we must somehow show the recipients the benefits of reading these messages. And the senders, by the way, too, otherwise they will forget to report.

In general, there are plenty of opportunities to under-implement this practice. Although if it works out, the benefits are enormous, right up to the rejection of insane changes in the early stages.

2. Others bugs

The policy is such a policy. Honestly, I did not come across such that a person was punished for the fact that in “his” piece of work someone discovered additional problems; but apparently it happens.
Perhaps, I vaguely put it in the original article. Speech in any case does not mean that suddenly take on someone else's work or go into another monastery. It's just that if I work on my task, and I see a problem in an area for which another city is responsible, I will usually ask if we already know about it. Especially if I know that we use different test configurations. Especially, if I suddenly know that testing of this feature has already been completed. Moreover, if the problem is not the best way affects my own work.
It’s not necessary to start a bug right away - for a start, you can simply ask: is it here like this - on purpose? And listen to the answer, and explain your point of view, if that. And let the bug let that party turn on, if this is important, I don’t feel sorry. At the same time, horizontal ties will strengthen.
By the way, an important point, underlined Rascko in the comments: A person should know "whose it is," if it is not his. It helps a lot: you can contact the person directly, and not write to the general newsletter or contact through the hierarchy of managers.

3. Minor bugs and interface bugs / implementations flying into reject

Unpleasant, very, I know from experience. On the one hand, it is our business to report a problem, on the other hand, the work turns out to be in vain.

Here is the question: who is the rejection and why? If a programmer is one thing, if a project manager is another (or even several other things, the manager may have a lot of reasons why this defect is not critical now; there was a recent remark on this topic in the article “Testing: out of business” ).

And solutions may be different. It is possible to agree that the decision on whether to fix or not for all defects was taken by the project manager, or, for example, the analyst responsible for the feature. You can simply escalate the problem and make a suggestion to a specific programmer. It is possible to achieve an obvious hint that we are not starting such specific defects (on usability, on performance, on localization), for they will not be fixed anyway - and this is not necessarily a disaster; Perhaps this is the best solution at the moment, allowing you to focus efforts. It is only necessary to determine up to what point such an agreement is in force and clearly to brush it back when the term expires.

What to choose? Depends on the specific situation. This is where the most difficult part begins - to identify the problem and find out what the situation is with us.

4. Laboratory

The test environment is important, there’s nothing to argue with again. And the decision, in general, is the right one: to look for a middle ground between the ideal and the rough reality.

I just want to emphasize that the testers themselves should speak first and foremost about the shortcomings of the test environment and how to correct them. Because the others do not know anything about this environment, especially the authorities. Until they came and did not say that there is a problem with iron, there is no problem, and no one will solve it.

Came once and calmed down - forget or ignore with a high degree of probability.
Only regular reminders will deprive the management of peace and make you believe that the problem is not contrived, and also has a solution. Well, also distinct examples and figures in the hands.

To transfer part of the testing out to customers / users (alpha, beta) is an option, but there is enough underwater rake for a separate article. But you can try.

5. Timing and reporting

The eternal problem is how to evaluate the work of testers, and even working in a short time. And programmers, by the way, too.
Either look at the number of bugs, or criticality, or user reviews, or just look at it. There are a lot of variants in literature and life.

If we talk about the result / terms of work on the product - evaluate the work of programmers and testers, and the rest of the team, too, together. We worked on the product all together, the result is also common.

If we are talking about evaluating a specific tester, then everything sounds easy. We give him some tasks for implementation. So we agree on the shore, that this and that is expected from you, they say. Performance criteria - such here. Timing ... Well, how much do you want? Ok, come on. Report problems, huh? And then - we look, what he did and estimate.
In practice, this is all much more difficult, especially if you are not used to it. Well, that’s what a speech is - these problems are not easily solved.

Returning to the deadlines: see that the deadlines are unrealistic? Talk about it. To test all that is needed - we need so much time. During the time that is, we can test so much. Let's prioritize to test the more important.

6. What happens to bugs like: not reproducible, observed and etc?

Here, in general, everything is about the same as in paragraph 3. Or am I missing something?

7. Automation vs manual testing

Well described problems. In this case, the solution is so strongly tied to a specific project / product that it is difficult to give any general recommendations.

Is that one: automation And manual testing. These concepts should not be at war; rather, on the contrary, to supplement and help each other. Moreover, automation is not necessarily about direct testing of the product. We can automate some actions / preparation for manual tests and perform the latter better and more efficiently. But by hand.

8. Position tester in the overall structure of the company

And here everything is again in the hands of the tester.

If testers in the company are highly valued, and they are engaged in nonsense, then very soon they will be at the level of a cleaner.
If testers bring real benefits to the project and its participants, authority will grow.
Of course, the upward movement is longer and harder to fall from the throne, but it is everywhere.

As for the recruitment of commands from the "garbage" of programmers - show that such people are not suitable for complex tasks facing the testing department. Prove that these tasks are complex and require high qualifications. Explain that losing the project, giving up good frames.

Complicated.