Specification By Example - BDD for Pragmatists

Habré has a lot of references to BDD. Unfortunately, the articles that I read did not give me an answer to the question “why do I need all this?” The answer came from an unexpected side. When I seriously took up the question of automating acceptance testing, I came across the book Gojko Adzic (not sure about the transcription, so I did not translate the author's name) Specification By Example .
Reading it, I did not get surprised to be surprised: each new chapter described the bumps that I stuffed on my personal experience, and offered solutions similar or better than those I came to myself by trial and error.

This article is the first in the series “BDD for Pragmatists”. It describes the key elements of the most effective, in my opinion, the process of developing commercial software in modern conditions. Two continuations will be devoted to working with SpecFlow and automating acceptance testing.

In order for the product to “take off” in the conditions of the modern IT market, we, developers, need to achieve two goals. Make the right product (exactly what the market wants to buy) and make the product correctly (without spaghetti code, crutches and other technical debt).

The right product and the right product are not the same thing!

Failure to comply with the first condition will lead to financial collapse, and therefore with a high probability of closing the project. Failure to comply with the second - the fact that one day we find ourselves waist-high in ~~g ...~~ code. Probably, it is not necessary to explain all the charms of supporting legacy-systems without tests and specifications.
')
Specification By Example is a process that allows you to regularly achieve both items. The process is based on agile, tdd, bdd, continuous integration and test automation.

Key elements of the Specification By Example :

Derive the main (deriving scope from goals)
Write the specification together (specifying collaboratively)
Give examples (illustrating using examples)
Clear the specification
Automate testing without changing the specification (automating validation without changing specification)
Embed test execution into the build process and develop documentation (validating frequently, evolving a documentation system)

Derive the main (deriving scope from goals)

An overwhelming minority of people are able to articulate what they really need. In most cases, you will come across the fact that the client will formulate the requirements not in the form “I have a goal, how can we achieve it?” , But: “make me a vundervaflu like a neighbor, but with mother-of-pearl buttons” . The client will tell you how he sees the solution to the problem, and go deeper and deeper into the implementation details.

Here lies the first and most important mistake . Usually, the same problem can be solved in several ways, and in the form of requirements, we are already asked to implement one of the possible solutions. But not all ~~yogurt~~ solutions are equally ~~beneficial~~ effective.
Our task, as engineers, is to propose a solution that is better and cheaper than the client requests. This principle was formulated by Henry Ford:

"If I asked people what they want, they would ask for a faster horse."

F-16 fighter - one of the most successful fighters in the history of the US Army. The initial development requirements were to reach the speed of Mach 2-2.5, which, together with other requirements, made the development and production of such an aircraft very expensive. Harry Hillaker - the lead designer of the F-16 - clarified why this speed requirement is so important, and received the answer: “the fighter must roll back if it gets really hot . ” Hillaker offered an alternative solution and designed the fighter, surpassing others in maneuverability at that time.

More than 30 years have passed, and these fighters still produce. 4400 aircraft sold in 25 countries. It is on the F-16 fly in most Hollywood films: from "Independence Day" to "Transformers" ... F-16 still can not reach speeds higher than Mach 2.
The F-16 was so successful because the engineer offered a better and cheaper solution than the client requested.

Pay attention to goals, but do not go straight to the solution. Reach the goal in different ways.

How to highlight the main

Albert Einstein once said:

"Formulating a problem is often more important than solving it."

Before encoding, make a specification. Yes, we - the developers - do not like paperwork, formalism and multipage Talmuds, which no one in the end reads. But think about how often you cursed the one who made up the requirements and acceptance criteria. Take the responsibility and make the specification yourself in the form in which it will be convenient for you to work.

Here are some examples of failed items that fell into the specification:

All pages should be displayed in 0.1 seconds. The word “all” should be avoided in the specification in principle. Yes, the main page should open as quickly as possible. But you spend weeks or months trying to push into this framework a summary report that is run once a year. Such rare and resource-intensive operations can take a long time. Nothing wrong with that.
The user-interface should look like OSX. Fashion has spawned thousands of websites with the replacement of native controls, and with them the pain of thousands of web designers and web developers. Let the interface look like OSX on a Mac and in a Safari browser. In other systems, leave the native controls.
On the main page of the website should be a flash banner. Perhaps, this banner does not need a flash at all, and everything can be done using html / css / javascript, and you already have a similar module out of the box. Avoid excessive specification of technological aspects in the specification.

Use user-stories

User-stories are one of the best ways to understand what is really needed. User-stories may take shape a little differently, but must contain 3 points:
In order to - why?
As a - who?
I want - what?

Having worked through each such story, you will be able to understand whether what you are asked for is really needed, or in fact there is a better solution to the problem, about which no one simply thought. If I want is already formulated, but As a and In order to - no, this is a reason to think. Perhaps you are going to develop functionality that no one needs.

This is how user-stories of ~~spamming~~ programs for increasing the loyalty of an online store look like (I apologize to readers who do not know English well, but I am convinced that the documentation should be in English):

Products order to existing customers,
As a marketing manager
I want customers to register personal details by joining a VIP program.
In order to entice your VIP program,
As a marketing manager
I want to order.
In order to save money,
As an existing customer
I want to receive special offers.

Write the specification together (specifying collaboratively)

The specification drawn up alone, without a command, is the second big mistake and a potential space for misunderstanding and subsequent edits. Instead of relying on only one specialist, involve the entire team in the specification. If you are working on a scram, a rally meeting is a great time to do it.
Developers understand the infrastructure better and know the technologies that can be applied to solve a problem. QA specialists will indicate where errors may occur. Product-owner is a subject matter expert. All of this information is useful for making specifications. Working together allows you to:

Better deal with the problem and find the best solution.
Make the whole team understand the new requirements in the same way.
Involve all participants in the process

At the planning stage, it is better to connect the whole team. After the main scope of work is clear, pair work or small meetings for 3-4 people are more effective to clarify difficult points.

Give examples (illustrating using examples)

Natural languages play a cruel joke with us. They leave room for interpretation, misunderstanding, and sometimes require knowledge of the subject area and / or specific jargon in order to understand what is being said. A small misunderstanding can lead to deadlines and a large number of revisions, or even rewriting of entire modules. Instead of long and painful formulation of requirements, describe them with examples. Together with the client you can find key examples . The help of developers and testers is invaluable in this case, because they will be able to point out in advance potential problem areas and technical limitations.

Here’s what a collaborative specification process might look like. For example, take the online store "Horns and Hoofs."

Varvara - customer representative
Vasya - developer
Olya - QA

Varvara: So, we signed an agreement with the publishing house ABC Press that we will deliver the books of the publishing house bought from us for free.
Vasya: Across Russia?
Barbara: No, that you. Only in Moscow.
Olya: Is the right publisher everywhere in our admin panel? Varya, you can give us a complete list of books, I'm afraid we can make a mess of it, in the DB the “publishing” field is a string.
Barbara: Yes, of course, there are not so many books there.
Vasya: And how many books can we deliver for free? We have some restrictions: 5-10? What happens if there are books from other publishers in the basket?
Barbara: It doesn’t matter as long as there is at least one book in the basket, the delivery is free.
Vasya: Well, what happens if a customer orders a book and a refrigerator? Such delivery will be expensive ...
Barbara: Yes, we somehow did not think about it. Let me write and clarify, and we will discuss this in a week, is there enough information to get started?
Vasya: Of course, I'll start sketching architecture.
Olya: And I’ll check with the publishers.
In 3 days.
Barbara (by phone): We talked and decided that free shipping would be available only for books. If there is something else in the basket - only regular delivery. And we decided to limit the number of books from above - no more than 10.
Vasya: Ok.

As a result, we get key examples :

Customer type	Cart contents	Delivery
VIP	1 book	Free
VIP	10 books	Free
VIP	11 books	Standard
Regular	10 books	Standard
VIP	5 washing machines	Standard
VIP	1 washing machine	5 books Standard

Clear the specification

During the joint discussion you can achieve a common vision of the goal. You can compare the previous stage so that you mined the diamond. The diamond itself is quite valuable, but after processing its value will increase many times.

During the discussion, each team member will pull the blanket over himself .:

Business representatives tend to pay too much attention to the UI (because they are not technical experts, the UI is the only thing that they can “touch”).
Developers tend to go into the details of implementation: which framework to choose and which technology to apply.
Testers will be paranoid looking for errors and vulnerabilities.

Thus, the original version will certainly be contradictory and redundant.
The purpose of this stage is to separate the wheat from the chaff and provide the necessary amount of details. The specification should be a single document for:

Acceptance criteria
Acceptance tests
Future regression tests

In order to minimize the chances of misunderstanding, we will write scripts in terms of Given When Then . If you are not familiar with this form of recording, remember the lessons of mathematics at school: Dano, Find, Solution . It's about the same here.

Given - initial context (precondition)
When - event (which is the script trigger)
Then - the result we want to get

For our book example, it would look like this:

Feature: Free delivery In order to save money As a VIP customer I want the system to offer free delivery on certain items to me Scenario: Free delivery Given I am a VIP customer And I am on product detail page And There are only books in my shopping cart And There are <= 10 books in my shopping cart And I have added 'ABC Press' book to my shopping cart When I press 'Go to checkout' button And I have chosen 'Moscow' in 'Ship To' dropdown Then I can choose free delivery

On the one hand, such a form of a record remains readable for ~~mere mortals,~~ not technical specialists (managers, client representatives, business analysts), but on the other, it is strict enough to avoid ambiguity.
We can change the script a bit to make QA happy.

 Scenario Outline: Free delivery Given I am a VIP customer And I am on product detail page And There are only books in my shopping cart And There are <bookQuantity> books in my shopping cart And I have added 'ABC Press' book to my shopping cart When I press 'Go to checkout' button And I have chosen 'Moscow' in 'Ship To' dropdown Then <deliveryType> is available Examples: |bookQuantity|deliveryType| |5 |Free | |10|Free | |11|Standard |

In this case, the script can be used as a list of test cases.
To write this script, I used SpecFlow, a solution for .NET platform. Similar tools are for Java, Ruby, PHP.

I do not focus on the instrument. How exactly to write tests using Given When Then is too broad a topic, which we will consider in the next article, but for now we will limit ourselves to basic information. Tests are created in a declarative style using “steps”. Note that the parameters from the Examples table will be passed to the arguments of the methods.

 namespace ProjectName.Specification { public class FreeDeliverySteps { [Given("I am a VIP customer")] public void GivenIAmVipCustomer() { throw new NotImplementedException(); } [Given("I am on product detail page")] public void GivenIAmOnProductDetailPage() { throw new NotImplementedException(); } [Given("I have added \'ABC Press\' book to my shopping cart")] public void GivenIHaveAddedAbcPressBookToMyShoppingCart() { throw new NotImplementedException(); } [Given("There are only books in my shopping cart")] public void GivenThereAreOnlyBooksInMyShoppingCart() { throw new NotImplementedException(); } [Given("There are (.*) books in my shopping cart")] public void GivenThereAreBookQuantityBooksInMyShoppingCart(int bookQuantity) { throw new NotImplementedException(); } [When("I press 'Go to checkout' button")] public void WhenIPressGoToCheckoutButton() { throw new NotImplementedException(); } [When("And I have chosen 'Moscow' in 'Ship To' dropdown")] public void WhenIHaveChosenMoscowInShipToDropdown () { throw new NotImplementedException(); } [Then("Then (.*) is available")] public void ThenDeliveryTypeIsAvailable(string deliveryType) { throw new NotImplementedException(); } } }

You can read more about BDD on the site of the founder of this paradigm .

Automate testing without changing the specification (automating validation without changing specification)

The finished specification will serve you at the same time as the technical specifications for the development of the functional and test scenarios to verify the correctness of the work. Do not delay testing automation “for later”. Tests will detect errors at an early stage of development, which means you do not have to redo the functionality again and again.

If When in the future requirements change, you will not need to figure out which tests are no longer needed. Once the requirements have changed, then the specification also needs to be changed.
Tests covering part of the functional code that “fell under the knife” either turn red or stop running altogether (become Ignored ), and the process will have to be repeated from the beginning: write tests, rewrite the code, check that all tests pass.

With this organization of work, you will change the specification and tests in one place. That is, the specification itself will become executable .

Embed test execution into the build process and develop documentation (validating frequently, evolving a documentation system)

Much has already been written about the benefits of Continuous Integration on Habré. We will look at this practice from our bell tower. So, we have an executable specification (a specification with examples and automated tests associated with it). Unfortunately, most of the team still can not "touch" the result of the work done. Managers will not install IDE, hung with plug-ins, and deploy the entire system on their machine.
You need to build test execution into your build system (if there is no build server in the organization, maybe this is a reason for it to appear?)

As soon as tests are performed regularly, you will notice that automated tests are the most reliable source of information about the state of affairs in a project at the moment. The graph below is an example of a successful development iteration. On the first day, a small part of the acceptance tests was written and they are all red (which is understandable, because we still don’t have any functionality). Gradually, the number of tests increases to cover the entire development scopes. In parallel, the number of green tests is growing - the functionality began to be developed.

The number of green tests is the only reliable criterion for assessing what has been done and what is to be.
Unfortunately, practice shows that developers, albeit out of good intentions, can report “yes, yes, everything will work tomorrow” for months. The schedule will not lie: the feature is ready when all its tests are green .

Too many red tests, and release soon? Most likely, you need to take action, otherwise the functionality will not be ready in time with the proper level of quality.

With the development of the product, the specification will grow, and the requirements will change following the requirements of the market. Even this specification will have to be maintained and structured so that all information is quickly available. I am willing to pay this price for the confidence that each next release will pass without delay, overtime, and the product will work like a clock.

Why am i

The world around us is changing very quickly. And now the world of business and development have turned to each other, if not the face, then at least half a turn. But the gulf of communication has not yet been completely overcome.
I began to implement the Specification By Example practice much earlier than I read the book. Many of the recommendations described come with experience, much can be learned from other sources. It is really valuable that Gojko Adzic was able to systematize this information and suggest the process.

My experience completely coincides with the author's experience: projects on which the principles of Specification By Example are implemented are developed faster, and the number of bugs and edits decreases.
I hope that the article will be useful to young teams who are in search of a suitable process.