What does the DevOps methodology mean for testers?

We offer to get acquainted with the translation of the article by Paul Gerrard "What Does DevOps Mean for Testers?".
Paul Gerrard is Technical Director of TestOpera Limited and Head of the UK Test Management Forum, winner of EuroSTAR European Testing Excellence and European Software Testing Awards (TESTA).
Translated and published by permission of the author.

Background

In this article I want to discuss the implementation of the DevOps methodology from the point of view of testing and tester. The “DevOps Movement” (there is no better name yet) is developing rapidly. The rate of innovation in this methodology, as well as many other changes that occur in this industry, is very high. At the same time, there is still no clearly formulated name for the movement itself.

Depending on the structure of your team, who you are talking to, the DevOps methodology can be both a solution to the problem and an independent goal. In some enterprises, the goal is to transition to digital technology, and the DevOps methodology is part of an overall approach to ensuring regular and high-quality deployment. It is in this sense that I view DevOps in this article. However, when marketing technologies and services related to the DevOps methodology, this goal may not be completely clear. The problem of cultural change (or behavior change) required to achieve success is often underestimated. Another assumption from which I am repelled is that for testers working within the DevOps methodology and under its influence, this methodology is new.
')
I built this article as an introduction to the DevOps methodology for such testers, as well as a discussion of its impact on testing methods. If you are an experienced practitioner using the DevOps methodology, this article may also be useful to you. If you are not a tester, you will at least have a look at the question from the point of view of the tester.

For the uninitiated reader: what is the DevOps methodology?

In simple words: DevOps is a concept that designates a development group and a system operation group that cooperate more closely with each other. In the so-called deployment pipeline (from the source code to operating in a production environment), developers automate any actions of the operation group. The operations team has the ability to track the actions of the developers and have some influence on them. The goal is to accelerate the deployment and implementation of software. Combining the operations group (Ops) and the developers (Dev) (actually as a team using Agile methods) allows for an approach that can be called “operation based on Agile Operations” (Agile Operations).

The obvious result of the successful implementation of the DevOps methodology is to reduce the amount of time it takes for a software change to go through all the stages from idea to operation in a production environment. When the developer says that the software change is “done”, the transition to using this change in the production environment is done using comprehensive automation. Automated tools and processes are used during system configuration, during the assembly process, during testing, deployment in a test, intermediate and production environment, during monitoring after deployment is completed, during evaluation and operation.

So, are DevOps just tools?

On the one hand, the goal of the DevOps methodology is to eliminate bottlenecks in the deployment pipeline through the use of automation. But the automation of multistage processes still needs to be managed. Many automated processes are in fact not autonomous — they cannot be performed without human intervention (for performing maintenance and handling non-standard situations). The fully automated DevOps process does not make sense without considering the influence of the human factor. Despite the fact that automation tools perform a lot of rough work, it is the people who control the process that ensures its successful (or unsuccessful) performance.

So, are DevOps just developers (Dev) and an operation group (Ops) working closely with automation?

No, again wrong. Transferring control between automated processes often involves other processes — usually testing of one kind or another. Automated tests are created by developers and testers. The results of these tests are aimed at providing sufficient information needed to perform other processes or, more often, people who are moving from one stage of the pipeline to another. Testers and developers who carry out testing ensure the successful and reliable execution of the DevOps process.

Head spin! And what does DevOps mean? This is a progressive new discipline. This question is discussed in detail in the article “What Is DevOps?” , Which appeared a few weeks before writing this article. As you already understood, the definition of the concept DevOps has not yet been finalized. Perhaps never will be.

What does this mean for testers? This means that there is still no “only true way” and that your role in the DevOps mode, which is constantly evolving (and each mode is constantly evolving), has not yet been definitively established. There are two things you can do:

1) pay attention to pain points and make efforts to reduce their negative impact;
2) identify opportunities that allow you to optimize the DevOps process.

If there is a single mantra that best describes the movement towards the DevOps methodology, then it sounds like this: “If this is unpleasant, do it more often.” This may seem like a cliché, but I will use this expression as a context for introducing and improving my testing skills using the DevOps methodology.

If it is unpleasant, do it more often.

Difficulties or unpleasant feelings that we experience when performing a certain work affect us negatively. If we do not like any task, then we want to postpone it. When we finally take on this unpleasant thing - it becomes even more unbearable. A visit to the dentist, cleaning the garage, software integration, testing, etc. Experience shows that the less often we perform such tasks, the more unpleasant they are to perform. Martin Fowler (Martin Fowler) suggested three reasons why the frequent (or even continuous) performance of certain tasks reduces discomfort.

The first reason is that large and complex tasks are difficult to plan, they are difficult to manage and difficult to control. Splitting large tasks into several small ones makes it easier to accomplish them and reduce the risk, and if something goes wrong, it will be easier to return everything to its former state. The second reason is that many tasks (and testing in this regard is the best example) provide the possibility of feedback. This feedback, if it arrives at an early stage and often enough, allows you to quickly respond to problems that arise before time is wasted. And the third reason: if we carry out any activity more often, we better cope with it. We learn to do this job effectively. In addition, we find opportunities to automate it in one way or another.

From the point of view of the tester, this mantra forces us to take the concept of automation more seriously when carrying out the testing process. If manual intervention is performed (usually between the automated stages in the DevOps process), then it should be considered a pain point: bottlenecks, causes of delays, and potentially unreliable and error prone aspects of the process.

Manual testing is unpleasant. Yes, you can be pleased to conduct research testing, you can assume that only you as a person can find those intricate errors that automation never finds, that only you as a tester can be trusted to prevent a catastrophe.

As a tester, you may be concerned about whether the testing work will be done properly, if it is entrusted to developers or automated tools. If this is unpleasant for you, do it more often.

Tests, automation and trust

There is a lot of controversy surrounding the value, such as testing and testing , and how much we can rely on testers, checks and automated tools ( The New Model and Testing v Checking ).

I am not saying that we should rely on automated checks. We will need a more complex approach. However, for the purposes of this article, we can at least divide the tests and testing activities into four components:

1) checks that can be automated by developers as part of component-based checks and continuous integration processes;

2) checks that can be automated (usually by system testers) to perform transactions at the API level, communication lines, or the entire system;

3) tests that include compatibility checks and demonstrate compatibility with browsers, operating systems, and platforms;

4) tests that only a person can perform.

In this article I can only make an assumption how to make such a division - each environment has its own characteristics. The most relevant question for this article is how can a tester “free himself” from late manual checks? Earlier, I wrote about the exclusion of late manual checks. It requires forethought and trust.

Focus on the following points:

1. If possible, manual checks that can be performed at the component level should be assigned to the developers. As a tester, you can offer to perform these tests during sessions of pair or teamwork. You can write them yourself and turn them into continuous integration mode.

2. User testing and complex testing may require the use of automated tools. Such testing should be minimized, since it takes a long time to complete, is unstable, and often requires maintenance. Consider whether you need to run it every time you check the code, or you can postpone it for use only on larger, less frequent releases.

3. What non-automated tests can be performed on components that are not yet integrated into the pre-release version? Can I perform manual testing in pairing sessions with developers? Are there alternatives to such testing? Can working with archival documents, prototyping using the BDD approach? Is it possible to perform user interface checks on mockups or frame structures?

4. What checks need to be performed only manually, as opposed to checks that should be left for regression testing (they are candidates for automated testing)?

Above, I mentioned trust. Another question is how to reliably test the system if there is no late manual testing. Imagine an environment in which all testing is done by automated means. Do you believe developers will properly test? Offset testing concerns left should dispel your doubts. If you, as a tester, act like a pathfinder: in order to identify and evaluate risks, select tests and ensure that they are built into the development and automation processes, then your concerns can be minimized.

Naturally, you must stop considering yourself as a guard of quality, the last line of defense, the only person who cares. You have to think more like a visionary, a risk identifier, a risk manager, a ranger, a coordinator, and an instructor or mentor.

Practice, monitoring and improvement

With all the desire, if you stop relying on late manual checks, errors can still be. When software is released into production, problems are discovered. One of the key disciplines of the DevOps methodology in terms of exploitation is deeper monitoring.
Monitoring at every level - from components and simple transactions in applications to integration and messaging and, of course, the infrastructure itself. One of the goals of monitoring is to generate fault messages before users experience their consequences. This is a rather ambitious statement, but it defines the ultimate goal.

When problems are encountered in a production environment, the challenge is to use the analytical information obtained from monitoring to not only track the cause and eliminate it, but also to refine the testing process (automated or manual) and reduce the likelihood of such problems in the future. The role of testing and analytical information in the operation of the entire pipeline is defined and discussed in the article “Thinking Big: Introducing Test Analytics” .

You can call automated testing of the DevOps process by monitoring. When monitoring is integrated with the production environment, it can be said that monitoring throughout the entire DevOps process with entry into production expands the scope of testing. Thus, the DevOps methodology does not reduce the role of testers.

Conclusion

I was recently asked: “In what case should you not try to apply the DevOps methodology in an organization?”. This is a good question. I think it is based on the concern of whether the DevOps methodology should become the norm and whether testers should pay attention to this. My answer is simple.
Why don't you make it so that developers and maintenance experts talk to each other? Why not start getting more reliable builds and deployments in a test and production environment? Why don't you use the best technology for a more accurate, efficient, and informative pipeline? The DevOps methodology is a good thing, but it is not always easy to achieve the required level of its use. Not to mention the fact that it requires a change in culture, and this is not so simple.

From the point of view of the tester, the DevOps methodology influences more in the early stages of projects, forcing us to think more seriously about the automation of testing, providing information and making decisions. Testers should strive to use the DevOps methodology, as it provides opportunities to perform actions proactively, gain more authority and respect in the project team.

On November 11, Paul Gerrard will give a master class in Moscow on the theme “Testing in the Digital Age: DevOps, ChatOps and Automation to Support Testing” .