Testing Development (TDD) is a great way to improve code quality and reliability. The same approach can be extended to the development of requirements. It is called "Development through acceptance tests" - acceptance test driven development (ATDD). At first I used to look at this approach, then I tried to apply it, then I tuned it for a long time to adapt it to my needs, and now I want to share my thoughts. And for myself once again put everything on the shelves.

In this article I will tell a little introduction to the topic. The example will be quite simple and rather for illustration. And in the next article I will try to share the story of how I used ATDD in practice when developing a real feature in a real product.

Instead of introducing

When I worked as a programmer at an outsourcing company for one bank, I had to study the specifications of requirements and evaluate the complexity of the tasks. It was necessary to evaluate as accurately as possible, we worked on the payment model for the project (Fixed Price), and all the deadlines were on our side and were not paid. Every time I read the specifications, I understood everything; I did not notice illogical points, omissions, and oddities in them. But as soon as the development began, all the shoals of requirements came out, and it was amazing how I missed them at the beginning. Despite my best efforts, I still could not think of a way to read the specifications and find problems in them before implementation.

Then I went to work in a large company, which was engaged in the development of a large and complex boxed product, on which a huge team worked. Analysts communicated with partners and customers and recorded their wishes. Then these specifications, before being taken into work, passed the review process, in which the developers also participated. In order not to waste time at the meeting itself, it was necessary to first read the requirements and prepare questions. As in the previous project, most of the questions about the content of the documents arose later - during development, and not when they should have appeared, that is, at the review stage.

Then I went to my startup. Naturally, there were no analysts, specifications and reviews. We received feedback from users in the form of emails or calls, immediately turned it into features and included in the development plan. Despite the lack of documented requirements, it was still necessary to evaluate the complexity of the tasks. What at first glance seemed very simple was in fact becoming a headache and vice versa. When the context was quickly switched from one problem to another, already implemented solutions flew out of my head and it became harder and harder to combine them in one product. We needed some kind of technical documentation, test plans and requirements. And to cost inexpensive.

What is ATDD

Acceptance test driven development (ATDD) is a development of the idea test driven development (TDD). The general sense is that before you do something, you need to come up with a criterion of the work done and a criterion that the work was done correctly. Why is it important? Because these criteria make it possible at the very early stage to understand what needs to be done, how to do it, what exactly to consider as a good result. Those. do not figure out the details along the way, building prototypes, and immediately approach the goal, since the goal is already defined, and quite formally.

These criteria are described in a language understandable to the customer in the form of ready-made scripts. Scripts model how the designed feature will be used in the future. If the script is implemented and the expected result in it can be obtained in practice, then the problem is solved correctly and the work can be considered complete. A set of such scenarios is called acceptance tests. Acceptance tests focus on the behavior of the system from the point of view of a person, and not on the internal structure and technical details of the implementation.

Despite the common name, this approach refers to a very specific part of the process - the one where the requirements are developed and their specifications are formalized. In this process, people are often involved from both the business and the technical side, i.e. people with different competencies and world views. Customers at the intuitive level understand what they want to see in the product, but not all can formulate and list requirements briefly (and fully). Developers (representatives of the performer), in turn, often do not know what exactly the customer forgot to tell, and how to find out.
To solve these problems, the Given - When - Then framework is used.

The Given - When - Then Framework

The point of the acceptance test is to show what happens to the system in a particular scenario. To do this, the script should describe what the system looks like at the beginning of the test, then some action is described that changes this system. This may be the impact from the outside, and maybe some kind of internal trigger. As a result, the system slightly changes its state, which is the criterion of success. Important point: the system is considered as a black box. In other words, when formulating a test, we do not know how the system is arranged inside or with which it interacts outside. There is one feature. Sometimes a system change is not available for direct observation. This means that the acceptance itself will not work. There are two ways out - either try to determine the state indirectly, through some neighboring signs, or simply do not use such a test. An example would be changing fields in database tables, pending changes in some inaccessible files, etc.

The unit tests use the Arrange - Act - Assert (AAA) pattern. This means that there should be explicit parts in the tests that are responsible for preparing the data - arrange, the action itself, the result of which must be checked - act, and the actual verification that reality coincided with the expectations - assert. For acceptance tests, the Given-When-Then (GWT) approach is used. The essence is the same, only from a different angle.

• Given describes what is "given", i.e. system state at initial time
• When sets the trigger directly, which should lead to a result. Most often this is some kind of user action.
• Then determines the result of this action, i.e. is the acceptance criterion.

Given part can contain both one and a set of states. In the case when there are several of them, these states must be read through the "AND". You can merge some states through "OR", but then it will be two different tests. This is possible to simplify recording. I recommend avoiding this until all possible combinations of states are described. Then you can be sure that nothing is forgotten and merge several tests into one to simplify reading and understanding. The same is true for Then - outcomes that need to be checked may be several. When there must be one. Triggers are better avoided.

GWT tests can be read out loud: "Let (given) A and B, and C. When (when) D happens, then (then) turns out to be E and F.". They can be used for documentation or formulation of requirements. When I say “read,” I don't mean that this is how they should be written. In reality, such tests are very ambitious. If you write them in plain text, then looking at them systemically is very hard. And without a system, it’s easy to miss some important scenarios.

A very important point: the recording format you need to choose the one that best suits your task, which is more convenient to work with. There are no restrictions here. Given, when, then - this is the general structure of the record, that is, something that must be in the test, and the direct presentation can be anything - even sentences, tables, charts.

ATDD does not dictate rules, but provides a framework for composing its specification through examples. There is a black box model, GWT and their combination. Everything else is the application of these mechanisms in practice, some of which can be considered established.

Example

For example, take something simple and clear, for example, a traffic light. How can I describe the requirements for the development of traffic lights using GWT notation? First you need to understand what exactly at the traffic lights you can call Given, what is When, and what Then.

For the state of the traffic light, you can take information about which section is now lit. The traffic light switches (changes state) at some intervals. So the trigger is a timer, more precisely, the timer operation. The result of a trigger is a transition to one of the states. Those. we can assume that in the example with the Given and Then traffic lights the same set:

1. Lights red
2. Lit red and yellow
3. Green
4. Flashing green
5. Lit yellow

We describe the behavior of traffic lights in GWT notation:

1. Let it burn Red . When the timer fires, then the traffic light switches to the simultaneous Red and Yellow mode.
2. Let it burn red and yellow . When the trigger is triggered, then the traffic light switches to Green .
3. Let it burn Green . When the trigger is triggered, then the traffic light switches to flashing green .
4. Let Green blink . When the trigger is triggered, then the traffic light switches to Yellow .
5. Let it burn Yellow . When the trigger is activated, then the traffic light switches to Red .

Here are 5 scenarios, after reading which, you can understand how a traffic light works. Naturally, the traffic light has a bunch of modes, for example, the yellow flashing mode (when it is faulty), or the manual control mode (for example, in the case of an accident), etc. But let's not complicate the illustration.

To describe tests with words seems redundant to me. Moreover, only the name of the color changes in them. Here a state diagram or a simple table is better suited:

The example shows one of the main advantages of acceptance tests: they allow you to communicate with business users in almost their language. A nice bonus is a ready-made set of scripts for testing and subsequent automation.

Completeness assurance

The Given - When - Then notation structures the test creation process and makes sure that the tests describe all aspects of the system's behavior. You do not have to sit and constantly ask yourself: “What scenario have I not yet described?”.
So, the algorithm is as follows:

1. Determine all the states that can be set, i.e. all given.
2. Define all triggers, i.e. When.
3. Then determine everything, what exactly can happen.
4. Now these lists must be multiplied multiply.
5. The result is a test suite.

At each of these stages, the participation of the customer or the person who plays his role is required, because it is he who best represents what should work in the end.

Why is it useful?

As already mentioned, such an approach, despite its redundancy, makes sure that none of the scenarios will be omitted. This is perhaps the main advantage of such formalization. Often, the business user sees the process only in general terms and he does not see the details. I am sure that you constantly hear from a customer or even an analyst of a phrase like: "We need such and such a feature, I thought of everything, see the picture", or "Here we need such and such a button, we already have similar functionality in in another place, do it there. " If, before starting development, sit down and estimate possible scenarios, then immediately a lot of details will emerge, in which, as is well known, lies the devil.

A similar approach is also useful in the case when the specter comes from the analyst and needs to be read, to give his own estimates of complexity and effort. When reading all the details slip away, but if you follow the GWT form in the course of reading, then it becomes immediately clear which scenarios are bad or inaccurately covered in the requirements and require clarification.

In addition to analyzing requirements to develop a solution, GWT scripts can also be used to collect requirements. Suppose that there is some kind of functional area and a person who understands it, but the time for communication with him is very limited. If you prepare in advance and parse the scenarios with the help of the GWT framework, then the interview itself will only need to learn that we have not forgotten anything from the Given, When section and clarified what exactly should be in the Then section.

There are special tools for automating GWT scripts, including those recorded in natural languages. An example is cucumber. I didn’t work with them, so I can’t tell anything but the fact of their existence.

Underwater rocks

The reverse side of GWT power is redundancy. Suppose you have defined N pieces given, M pieces when and K pieces then. In the worst case, the number of tests will be huge - N M K. And with this we must somehow live. This is the upper bound for complexity; in reality, not all of these scenarios will be feasible, and some of them will duplicate each other, and some can be skipped due to low priority or obviousness.

The second disadvantage is the format. In my experience, I can say that GWT recordings always strive for minimalism. During their development, you do not want to spend time on detailed descriptions, because often the scenarios are similar to each other. The result is a hard-to-read structure. After a break, to understand it, you have to restore the context, and re-recall the conditional abbreviations and notes. It also makes it difficult to pass the document to someone for review, since, most likely, the author himself will be required to read it.

The following disadvantage explains why ATDD is rather related to the field of requirements formalization with a free bonus in the form of test scenarios, rather than testing itself. Such scenarios cannot describe composite (large and complex) scenarios. Testing the perfect black box is primarily based on the axiom of its ideality. In reality, black boxes are never black; they always interact with something outside of themselves, being part of a more complex system — a product. It is easy to over-complicate the requirements if you try to include all the links within the product in one document at once. Such a set of acceptance tests will be so huge and complex that it will be of little help. Therefore, in real life end-to-end scenarios as acceptance tests do not apply.

A bit of historical background

According to Wikipedia, the idea to formulate specifications through specific scenarios was first described by Ward Cunningham in 1996 , and the term specification by example was introduced by Martin Fowler in 2004 . Further development of the idea is formulated in the book "Bridging the Communication Gap: Specification by Gojko Adzic 2009". In 2011, he also released another book on this topic: "Specification by Example: How Successful Teams Deliver the Right Software." I recommend these books to refer to the source.