Automation on the methodology of BDD. Our experience of successful implementation
The article is published on behalf of Vadim Trubanov, @vonaburt
The BDD methodology is increasingly gaining the attention of the IT industry as a logical step in the development of traditional approaches to project testing, including approaches to test automation. The current era of information technology dictates its own rules, and in this technology race, the one who knows how to react to any changes quickly and efficiently wins. This is especially true of companies associated with banking activities, such as our bank, where each delayed hour before a release can affect the overall picture of the quality of services compiled by our customers. When used correctly, the BDD methodology allows us to reduce the time spent on testing manufactured products, improve the quality of testing and make the process itself transparent and understandable to all, which prompted us to use it. At the moment, the BDD methodology is being implemented on our two web-products, is actively developing and is already bearing fruit. I would like to share our experience in the implementation of BDD by testing automation and talk about the basic principles that will allow you to implement this methodology without serious consequences, quickly and, most importantly, to make its use effective.
The framework on which BDD cases will later be written is the heart of the whole methodology, its main element. That is why the framework should be given increased attention. This is your starting point . Our solution is written in Java and is based on the jBehave library. Since BDD is used on web projects, Selenium WebDriver is nowhere. When using the PageObject-pattern, a large role is given to working with elements and pages. Serenity was chosen as a report generator. Let us name three obvious main principles that your framework must adhere to. First, it must be reliable . Take the time and write a good core of the project. Unspent 30 minutes of work can result in hours of downtime for your BDD team. Secondly, the framework should be as convenient as possible to use . The main users of the framework are not you, and not the automation team! The framework should be friendly, pleasant and understandable to those who are going to write their BDD-cases on it. Third, the framework must be flexible . Returning to responding to rapid changes, you understand — it is not known how the project under test will change tomorrow. Initially, build your BDD framework so that you always have the opportunity to rebuild the basic mechanisms of the framework.
A good BDD automation framework is associated with the following concepts:
')
All three of the above items should be as user friendly as possible for the framework. Remember that users are not developers , and things that are obvious to you, developers, may be far from obvious to them.
The mechanism for writing BDD cases is quite simple. IDEA Intellij CE is installed to each user of the framework, a plug-in is installed to it, which gives the user the ability to autocomplete existing BDD instructions. Almost every instruction has its own aliases, allowing users to write their case in the way they are comfortable.
Most instructions have parameters: names of pages, elements, previously saved values, functions. All of them allow us to make cases flexible and efficient.
Surely, many wondered how to make work with elements, their locators, collections of elements, pages with elements more convenient and transparent. We stopped at the description of the full hierarchy of pages and their elements. Such an approach should be familiar to everyone who used the PageObject pattern in their projects. The entire portal is divided into pages within which all the elements on them are described. Repeating blocks of pages are rendered in “widgets” or “sub-pages”, for example, such as “Menu” or “Right Board”. At the moment, we store more than 2,200 locators of elements of the main portal of the bank and more than 1000 elements of the portal for legal entities, and these figures are growing restlessly with the release of all new and new functionality. Automators and functional testers involved in the methodology are wondering - how to work with so many elements? The solution was found: write your own plugin for IDEA Intellij, parsing the data file into a tree structure and allowing you to add and edit new pages and elements.
The instructions are structured in such a way that the user can work with all elements of the loaded page in advance by “loading” a certain page. Thanks to this, users of our framework avoid confusion with the choice of the necessary element, page or widget.
To increase the capabilities of BDD instructions, we implemented the following functionality:
The launch of BDD-cases is implemented using Apache Maven and a modified standard StoryRunner`a, allowing you to run BDD-cases in multi-threaded mode. Functional workers can run the necessary tests directly from their working machines and view the progress of the tests. All autotests are run through TeamCity, manually - quite rarely.
Testing and self-testing are closely related to test data (TD). It is one of the fundamental pillars of any quality assurance process. High-quality test data and their competent use will allow to bring testing to a new level. Returning to the BDD instructions: it is through them that the user of the BDD framework will transfer test data. "Hardcoding" test data directly in the instructions - moveton, condemning the difficulty in updating the TD. As a result, we faced a task:
- take storage of TD in a single place of storage;
- to realize the possibility of convenient work with stored TDs in BDD-instructions;
- to realize the possibility of convenient updating of TD and control over them;
How we solved it:
- all test data are entered into the database;
- implemented an internal test API that allows working with data in the database;
- implemented the mechanism for working with the test API within the BDD framework in the format of instructions;
- created a web portal that gives users a framework interface for working directly on updating, updating and editing test data;
The web portal for working with test data is written in Skalovsky Play, divided into our web projects that use BDD automation. There is also an internal division by test data categories: fields, values, their parameters for correct / incorrect validation, test users.
Test users are replenished due to the implemented mechanism for generating test data , the details of which may be a separate article. The mechanism allows testing on clients with any set of products and with any necessary parameters.
This is, in our opinion, a brief description of the necessary technical elements of a convenient and effective BDD framework. Briefly summarizing: do not forget about who you are writing the BDD framework for, how you will work with page elements, how the user will interact with instructions and test data.
Perhaps we will tell you more about the framework as a separate article.
After the framework is written or is in the final stages of writing, it's time to think about how users of your BDD framework will work with it. Issuing the necessary rights to install software and network accesses can turn into an epic with a hazy future in many large companies. Fortunately, the bureaucratic delays in our company are minimal, and we were able to quickly install all the necessary software for a selected group of future BDD-automators. Of course, sometimes there were specific problems, but they were quickly resolved. For colleagues from outstaff / outsource, a simple and effective solution was found in the form of pre-prepared virtual machines with the full necessary “combat pack”, which also quickly “multiplied” if necessary. If you have any problems, you can always go to the virtual machine and see what's wrong.
The moral of this brief, but very important chapter: make sure in advance how future users of your BDD framework will work with it, whether it is full installation of the necessary software on local machines or pre-configuration and subsequent copying of virtual machines with the framework. The faster your colleagues start using the framework, the faster you will receive from them the long-awaited feedback that will be especially useful in the beginning of the development of BDD automation.
Without aligned workflows, automation using the BDD methodology is unlikely to be beneficial. It is worth talking about how you can embed the use of this methodology in your projects. The options are quite flexible and depend on how much you want to involve all participants in the development of the desired product. Perhaps this will be the initial incomplete version involving only testers and automation engineers. In this case, you will gain time by removing the “layer” between your automators and functional testers in the form of test cases for automation.
Product managers with analysts can indirectly participate in this process, indicating which functionality you need to pay particular attention to and automate primarily in terms of the business significance of the functionality.
The next use case is full. This option assumes full participation in the BDD methodology of all participants in the development process - from product managers to testers and automators. Each participant in a particular development stage participates in the formation of the final BDD-cases, which together provide a full-fledged automated regression of the product. Also, this option smoothly leads you to full acceptance testing (ATDD), when all the criteria for “acceptance” of a product are formed not only by testers, as is often the case, but by all participants in the development process. This approach is most useful and effective from the point of view of automation, it gives a quick and very high-quality replenishment of the final autoregressive.
We started with the first, incomplete version, involving only functional testers and automation engineers in the methodology, largely as an experiment that would show us in practice the strengths and weaknesses of the methodology. We realized that such an approach to testing is bearing fruit, and we began to think about how we would involve all the participants in the methodology. The classic {x} DD option, when all the tests are initially written, and only then the direct development begins, in our initial conditions, it would have greatly influenced our fast release cycle, which is an unaffordable luxury for banking services. In addition, the installation, configuration and support of all the necessary software for BDD-automation to all participants of the development stages, not to mention the training with the BDD framework, would slow us down further in the development of the methodology. We found the solution in the “roots” of the BDD methodology. Let me remind you that the methodology is based on the use of DSL-language (Gherkin), a human-readable language for describing the behavior of a system that describes the functional with the help of business situations (behavior scenarios, user stories). Due to the simple rules of scripting, you can quickly attract a large number of employees to the formulation of tasks and their subsequent detailing in the format of the Gherkin language. The whole process is also carried out within the framework of the project management and bugtracking software you use, in our case this is Atlassian Jira. Managers and analysts are moving away from the old format of problem statements and starting to set tasks in BDD format. Designers and developers get a description of the tasks in the BDD-format, but they can themselves add to the list of user stories, based on the features of the interface or the libraries used during development. As a result, when a task gets into testing, the tester sees a list of user stories, analyzing which, supplements them with his own user scenarios, based on the position of the tester. Each story is a complete script for testing. This is a live product documentation. And most importantly, all the scripts from the task description are quickly transferred by the testers to the BDD framework, becoming virtually ready-made autotests.
As a result, it is practically the transition of the BDD-methodology to full-fledged acceptance testing (ATDD), when all development stages are involved in the development of product acceptance criteria.
This is a description of the automation of the BDD methodology in practice "from a distance." And what happens at a lower level, how is the process of replenishing the autoregress? To answer this question, you need to return to the basis of the BDD framework. The mechanism of user interaction with the BDD framework is built on instructions. The list of available instructions in the BDD framework and the tree of the described pages and elements make it possible to automate most user scripts. It is logical to assume that the list of instructions and the tree of the described pages and elements do not always allow to automate this or that scenario. Especially when automating scenarios of a new, not yet automated functionality, despite the fact that the instructions are constructed at the most atomic, with a minimum level of abstraction, at the level of interaction with elements. Plus, there may be new pages and their new elements that are not yet described in our hierarchy of elements and pages. Thus, there are BDD-cases, for the automation of which there are not enough instructions, pages and their elements. The tester introduced this situation into a stupor from the category of “what should I do if the necessary instruction is not yet available?”. There are two options for how to build a process. The first is not to write a new BDD case, notify automatists that not all instructions are enough or not all elements are described. Immediately assess whether enough to automate the case, it is quite difficult. Or a second, more rational option is to give the user to write his case the way he wants, despite the lack of instructions, pages and elements. We gave testers the freedom to write instructions , gave them the opportunity to write cases the way they were comfortable and built the process in such a way that such cases having a lack of instructions or element descriptions gave rise to tasks in Jira that would be “input” to automation tools. Automators implement the missing instructions, describe the missing elements and tasks, and the self-test of the tester enters the autoregress work branch.
To ensure the quality of new BDD cases, a mandatory code review process has been set up. Specific rules are formulated that describe how the main blocks of BDD cases should look, how they should be designed. It is by these rules that the written cases are verified by colleagues from functional testing. The rules affect the suitability of the location of the case and its semantic content, the presence of the necessary labels and other important points. Without these rules, the autorepress replenishment process will be chaotic and unpredictable. But about this in the following parts of the article.
This part is a logical continuation of the previous part.
Any methodology, any approach to development, production, any organization of labor is greatly hampered if the participants in these processes do not know their place in this process. A person must know his role, must know the final list of responsibilities that this role implies. Moreover, the more detailed these responsibilities are, the clearer and more transparent the process will be for the participant of the process himself, and for everyone else. These theses do not bypass even such a complicated and time-consuming methodology as BDD.
The methodology is quite flexible, and often many people stop at attracting only automation engineers and testers to the methodology. The full use of the methodology involves the involvement of all the participants in the development, and that is what we will consider. So who are the main “actors” of the BDD methodology? Of course, these are functional testers, which we denote as “FT”. FT are divided into “just testers,” writing and automating BDD cases, and those who, in addition, conduct the code review of new BDD cases, which we denote as “KR”. This automatics, which we call the "AT". Analysts, managers and technologists of all other participants in the process, we combine the designation "AN". We will call the “God-like” role the “project manager”, or “RP” for short. These are all highlighted methodology roles:
- “FT” (functional testers);
- “KR” (code-reviewers);
- "AT" (automatics);
- "AN" (analysts, managers, technologists etc);
- “RP” (project manager).
Now let's look at the main possible actions in the methodology. It:
- writing BDD-cases in free format in the Gherkin language;
- automation of BDD-cases in the BDD-framework;
- code review of automated BDD cases, checking for compliance with the selected rules for drafting new BDD cases;
- automation of missing instructions, their support; description of new pages and elements, their support;
- carrying out a code-review of automation of missing instructions, their support, the description of new pages and elements;
We create the final matrix of the roles of the methodology and possible actions:
Thus, each in the process performs its role and functions:
- the new functionality is described by managers, analysts and technologists in a convenient format for them in the Gherkin language;
- Functional testers are busy writing case-quality tests, automate them and carry out code review of case studies themselves;
- Automators are busy only with pure automation and related automation work, and not writing cases, as is often the case.
Despite the fact that BDD-cases are written in a human-like language, and automation is carried out in a specially created framework, you will have to take time for minimal training of all participants in the methodology. Let's start with testers, without which it is very difficult to imagine the BDD methodology.
The role of testers in the process is to create high-quality BDD-cases and automate them in the BDD framework, or, as is often the case, automate “right on the spot”, directly in the framework. Testers should understand how to properly compose a BDD case, what it consists of, and how to work with it in the future. It requires knowledge of the language Gherkin. Information can be conveyed to the children either at a meeting with a presentation, or using self-study materials in your corporate wiki, or better, both.
The skills of testers with the framework is best to get with practice. Our experience has shown that the most effective "learning task". These are typical - testers will most likely encounter them during automation - tasks for a project in which BDD is being implemented. Each task is given a certain deadline for execution, after which collective meetings are held with a discussion of the results of the tasks and the problems that arise.
Training managers, analysts, technologists and others involved only in writing BDD-cases, is reduced to learning the Gherkin language, which they will use in compiling a description of tasks for developing a functional or its revision / change. Training differs little from training testers: a good presentation with an interactive help in the form of a discussion of the issues raised and a corporate wiki with all the necessary information will help.
Automators, as a rule, do not need training: the main thing is to convey to them the main points of the automation process, and to make it clear to them that it is impossible to limit the flight of fancy of testers, even if the cases they compiled look strange and incomprehensible to them. Testers have their own “kitchen”, they have their own unique view of the world.
It is worth noting that in the approach we have developed to the methodology, evolution from one role to another is possible. Thus, a functional tester, having gained the necessary experience in compiling BDD cases and in their automation, can become a code-reviewer. A code reviewer, wishing not only to write cases, but also a full-fledged code inside a BDD framework, can become an automator when he learns all the necessary skills. There were no precedents yet, but we, the automation specialists, started the automation training course among our BDD testers and hope that we will soon “grow” our automation tools. The minimum requirements are skills in any OOP language and a great desire to learn something new.
From all incoming “new” testing employees, from time to time, form BDD automation training groups, select mentors from trained people who will not only help the new ones to understand the methodology well, but will also play an important role in introducing new ones into your team. This also applies to all other participants in the methodology.
Remember that the results of the implementation of the methodology depend on a competent systematized approach to employee training.
The results of the autorepress replenishment are discussed every week, when we sum up the results of the so-called “sprint” - the weekly segment, during which we record how the automation of our products develops and discuss the problems that arise. The meeting summarizes the past week, the quantitative indicators of automation are announced. For the BDD automation team, this is a great motivational tool that makes it clear to them that their work does not go unnoticed. , «», , , , .
, BDD- «BDD-», . , , , . , BDD- , , BDD-. , , , , .
— , «» , BDD-. : , « ».
, . ; , , .
— , . . , , . , , - , , , . , , . : , , , . , . . , , .
. , , , - . , : . «» .
— . , . — , . , . : « — — ».
, , - .
, , , . open-source . , Selenium open-source «» , . . CI-: - , - , - . : — . , , .
BDD- BDD-. , — , «BDD-», BDD- . , 100% — BDD- (-, , ), BDD- , . :
, BDD-, , , , . , , . , / , , . , , BDD- . BDD- , : , BDD- .
BDD- BDD-, , . BDD-. BDD-, , . . , , . . , , . , .
BDD-, , «», . , N . , . TeamCity Selenium Grid.
, , . . BDD- . BDD- . .
BDD . , , , BDD BDD , , BDD «» .
, BDD : — , . BDD , , , , , .
, , , , , , . , , , .
The BDD methodology is increasingly gaining the attention of the IT industry as a logical step in the development of traditional approaches to project testing, including approaches to test automation. The current era of information technology dictates its own rules, and in this technology race, the one who knows how to react to any changes quickly and efficiently wins. This is especially true of companies associated with banking activities, such as our bank, where each delayed hour before a release can affect the overall picture of the quality of services compiled by our customers. When used correctly, the BDD methodology allows us to reduce the time spent on testing manufactured products, improve the quality of testing and make the process itself transparent and understandable to all, which prompted us to use it. At the moment, the BDD methodology is being implemented on our two web-products, is actively developing and is already bearing fruit. I would like to share our experience in the implementation of BDD by testing automation and talk about the basic principles that will allow you to implement this methodology without serious consequences, quickly and, most importantly, to make its use effective.
1. Build a good and convenient framework
The framework on which BDD cases will later be written is the heart of the whole methodology, its main element. That is why the framework should be given increased attention. This is your starting point . Our solution is written in Java and is based on the jBehave library. Since BDD is used on web projects, Selenium WebDriver is nowhere. When using the PageObject-pattern, a large role is given to working with elements and pages. Serenity was chosen as a report generator. Let us name three obvious main principles that your framework must adhere to. First, it must be reliable . Take the time and write a good core of the project. Unspent 30 minutes of work can result in hours of downtime for your BDD team. Secondly, the framework should be as convenient as possible to use . The main users of the framework are not you, and not the automation team! The framework should be friendly, pleasant and understandable to those who are going to write their BDD-cases on it. Third, the framework must be flexible . Returning to responding to rapid changes, you understand — it is not known how the project under test will change tomorrow. Initially, build your BDD framework so that you always have the opportunity to rebuild the basic mechanisms of the framework.
A good BDD automation framework is associated with the following concepts:
')
- the mechanism of writing BDD-cases;
- BDD case triggering mechanism;
- work with test data.
All three of the above items should be as user friendly as possible for the framework. Remember that users are not developers , and things that are obvious to you, developers, may be far from obvious to them.
The mechanism for writing BDD cases is quite simple. IDEA Intellij CE is installed to each user of the framework, a plug-in is installed to it, which gives the user the ability to autocomplete existing BDD instructions. Almost every instruction has its own aliases, allowing users to write their case in the way they are comfortable.
Most instructions have parameters: names of pages, elements, previously saved values, functions. All of them allow us to make cases flexible and efficient.
Surely, many wondered how to make work with elements, their locators, collections of elements, pages with elements more convenient and transparent. We stopped at the description of the full hierarchy of pages and their elements. Such an approach should be familiar to everyone who used the PageObject pattern in their projects. The entire portal is divided into pages within which all the elements on them are described. Repeating blocks of pages are rendered in “widgets” or “sub-pages”, for example, such as “Menu” or “Right Board”. At the moment, we store more than 2,200 locators of elements of the main portal of the bank and more than 1000 elements of the portal for legal entities, and these figures are growing restlessly with the release of all new and new functionality. Automators and functional testers involved in the methodology are wondering - how to work with so many elements? The solution was found: write your own plugin for IDEA Intellij, parsing the data file into a tree structure and allowing you to add and edit new pages and elements.
The instructions are structured in such a way that the user can work with all elements of the loaded page in advance by “loading” a certain page. Thanks to this, users of our framework avoid confusion with the choice of the necessary element, page or widget.
To increase the capabilities of BDD instructions, we implemented the following functionality:
- the ability to write your own small auxiliary scripts (for example, to obtain a random value, calculate mathematical operations, etc.) both in the body of the instruction and in the variables of the “Meta” block;
- the ability to save the necessary values ​​into variables and further use these variables.
<!-- > Meta: @profile #{AutoPayments} @region . @code 916 @phone #{random{1000000,9999999}} @sum #{random{100,200}} @sum2 #{random{100,200}} @level 150 @level2 600 @limit #{random{650,999}} @limit2 #{random{650,999}} @dateStamp #{now(ddMMyy HHmmss)} @preName EditMAP <!-- > When "name" "#{script{ String temp = "#{preName}" + " " + "#{dateStamp}"; RESULT = temp}}" <!-- > When "" "" When : |field|value| | |#{code} #{phone}| | |#{name}| ||#{sum}| | |#{level}| | |#{limit}| The launch of BDD-cases is implemented using Apache Maven and a modified standard StoryRunner`a, allowing you to run BDD-cases in multi-threaded mode. Functional workers can run the necessary tests directly from their working machines and view the progress of the tests. All autotests are run through TeamCity, manually - quite rarely.
Testing and self-testing are closely related to test data (TD). It is one of the fundamental pillars of any quality assurance process. High-quality test data and their competent use will allow to bring testing to a new level. Returning to the BDD instructions: it is through them that the user of the BDD framework will transfer test data. "Hardcoding" test data directly in the instructions - moveton, condemning the difficulty in updating the TD. As a result, we faced a task:
- take storage of TD in a single place of storage;
- to realize the possibility of convenient work with stored TDs in BDD-instructions;
- to realize the possibility of convenient updating of TD and control over them;
How we solved it:
- all test data are entered into the database;
- implemented an internal test API that allows working with data in the database;
- implemented the mechanism for working with the test API within the BDD framework in the format of instructions;
- created a web portal that gives users a framework interface for working directly on updating, updating and editing test data;
The web portal for working with test data is written in Skalovsky Play, divided into our web projects that use BDD automation. There is also an internal division by test data categories: fields, values, their parameters for correct / incorrect validation, test users.
Test users are replenished due to the implemented mechanism for generating test data , the details of which may be a separate article. The mechanism allows testing on clients with any set of products and with any necessary parameters.
This is, in our opinion, a brief description of the necessary technical elements of a convenient and effective BDD framework. Briefly summarizing: do not forget about who you are writing the BDD framework for, how you will work with page elements, how the user will interact with instructions and test data.
Perhaps we will tell you more about the framework as a separate article.
2. Customize the environment for employees
After the framework is written or is in the final stages of writing, it's time to think about how users of your BDD framework will work with it. Issuing the necessary rights to install software and network accesses can turn into an epic with a hazy future in many large companies. Fortunately, the bureaucratic delays in our company are minimal, and we were able to quickly install all the necessary software for a selected group of future BDD-automators. Of course, sometimes there were specific problems, but they were quickly resolved. For colleagues from outstaff / outsource, a simple and effective solution was found in the form of pre-prepared virtual machines with the full necessary “combat pack”, which also quickly “multiplied” if necessary. If you have any problems, you can always go to the virtual machine and see what's wrong.
The moral of this brief, but very important chapter: make sure in advance how future users of your BDD framework will work with it, whether it is full installation of the necessary software on local machines or pre-configuration and subsequent copying of virtual machines with the framework. The faster your colleagues start using the framework, the faster you will receive from them the long-awaited feedback that will be especially useful in the beginning of the development of BDD automation.
3. Build workflows. Methodology Use Options
Without aligned workflows, automation using the BDD methodology is unlikely to be beneficial. It is worth talking about how you can embed the use of this methodology in your projects. The options are quite flexible and depend on how much you want to involve all participants in the development of the desired product. Perhaps this will be the initial incomplete version involving only testers and automation engineers. In this case, you will gain time by removing the “layer” between your automators and functional testers in the form of test cases for automation.
Product managers with analysts can indirectly participate in this process, indicating which functionality you need to pay particular attention to and automate primarily in terms of the business significance of the functionality.
The next use case is full. This option assumes full participation in the BDD methodology of all participants in the development process - from product managers to testers and automators. Each participant in a particular development stage participates in the formation of the final BDD-cases, which together provide a full-fledged automated regression of the product. Also, this option smoothly leads you to full acceptance testing (ATDD), when all the criteria for “acceptance” of a product are formed not only by testers, as is often the case, but by all participants in the development process. This approach is most useful and effective from the point of view of automation, it gives a quick and very high-quality replenishment of the final autoregressive.
We started with the first, incomplete version, involving only functional testers and automation engineers in the methodology, largely as an experiment that would show us in practice the strengths and weaknesses of the methodology. We realized that such an approach to testing is bearing fruit, and we began to think about how we would involve all the participants in the methodology. The classic {x} DD option, when all the tests are initially written, and only then the direct development begins, in our initial conditions, it would have greatly influenced our fast release cycle, which is an unaffordable luxury for banking services. In addition, the installation, configuration and support of all the necessary software for BDD-automation to all participants of the development stages, not to mention the training with the BDD framework, would slow us down further in the development of the methodology. We found the solution in the “roots” of the BDD methodology. Let me remind you that the methodology is based on the use of DSL-language (Gherkin), a human-readable language for describing the behavior of a system that describes the functional with the help of business situations (behavior scenarios, user stories). Due to the simple rules of scripting, you can quickly attract a large number of employees to the formulation of tasks and their subsequent detailing in the format of the Gherkin language. The whole process is also carried out within the framework of the project management and bugtracking software you use, in our case this is Atlassian Jira. Managers and analysts are moving away from the old format of problem statements and starting to set tasks in BDD format. Designers and developers get a description of the tasks in the BDD-format, but they can themselves add to the list of user stories, based on the features of the interface or the libraries used during development. As a result, when a task gets into testing, the tester sees a list of user stories, analyzing which, supplements them with his own user scenarios, based on the position of the tester. Each story is a complete script for testing. This is a live product documentation. And most importantly, all the scripts from the task description are quickly transferred by the testers to the BDD framework, becoming virtually ready-made autotests.
As a result, it is practically the transition of the BDD-methodology to full-fledged acceptance testing (ATDD), when all development stages are involved in the development of product acceptance criteria.
This is a description of the automation of the BDD methodology in practice "from a distance." And what happens at a lower level, how is the process of replenishing the autoregress? To answer this question, you need to return to the basis of the BDD framework. The mechanism of user interaction with the BDD framework is built on instructions. The list of available instructions in the BDD framework and the tree of the described pages and elements make it possible to automate most user scripts. It is logical to assume that the list of instructions and the tree of the described pages and elements do not always allow to automate this or that scenario. Especially when automating scenarios of a new, not yet automated functionality, despite the fact that the instructions are constructed at the most atomic, with a minimum level of abstraction, at the level of interaction with elements. Plus, there may be new pages and their new elements that are not yet described in our hierarchy of elements and pages. Thus, there are BDD-cases, for the automation of which there are not enough instructions, pages and their elements. The tester introduced this situation into a stupor from the category of “what should I do if the necessary instruction is not yet available?”. There are two options for how to build a process. The first is not to write a new BDD case, notify automatists that not all instructions are enough or not all elements are described. Immediately assess whether enough to automate the case, it is quite difficult. Or a second, more rational option is to give the user to write his case the way he wants, despite the lack of instructions, pages and elements. We gave testers the freedom to write instructions , gave them the opportunity to write cases the way they were comfortable and built the process in such a way that such cases having a lack of instructions or element descriptions gave rise to tasks in Jira that would be “input” to automation tools. Automators implement the missing instructions, describe the missing elements and tasks, and the self-test of the tester enters the autoregress work branch.
To ensure the quality of new BDD cases, a mandatory code review process has been set up. Specific rules are formulated that describe how the main blocks of BDD cases should look, how they should be designed. It is by these rules that the written cases are verified by colleagues from functional testing. The rules affect the suitability of the location of the case and its semantic content, the presence of the necessary labels and other important points. Without these rules, the autorepress replenishment process will be chaotic and unpredictable. But about this in the following parts of the article.
4. Share the roles
This part is a logical continuation of the previous part.
Any methodology, any approach to development, production, any organization of labor is greatly hampered if the participants in these processes do not know their place in this process. A person must know his role, must know the final list of responsibilities that this role implies. Moreover, the more detailed these responsibilities are, the clearer and more transparent the process will be for the participant of the process himself, and for everyone else. These theses do not bypass even such a complicated and time-consuming methodology as BDD.
The methodology is quite flexible, and often many people stop at attracting only automation engineers and testers to the methodology. The full use of the methodology involves the involvement of all the participants in the development, and that is what we will consider. So who are the main “actors” of the BDD methodology? Of course, these are functional testers, which we denote as “FT”. FT are divided into “just testers,” writing and automating BDD cases, and those who, in addition, conduct the code review of new BDD cases, which we denote as “KR”. This automatics, which we call the "AT". Analysts, managers and technologists of all other participants in the process, we combine the designation "AN". We will call the “God-like” role the “project manager”, or “RP” for short. These are all highlighted methodology roles:
- “FT” (functional testers);
- “KR” (code-reviewers);
- "AT" (automatics);
- "AN" (analysts, managers, technologists etc);
- “RP” (project manager).
Now let's look at the main possible actions in the methodology. It:
- writing BDD-cases in free format in the Gherkin language;
- automation of BDD-cases in the BDD-framework;
- code review of automated BDD cases, checking for compliance with the selected rules for drafting new BDD cases;
- automation of missing instructions, their support; description of new pages and elements, their support;
- carrying out a code-review of automation of missing instructions, their support, the description of new pages and elements;
We create the final matrix of the roles of the methodology and possible actions:
| Action / Role | FT | KR | AT | An | RP |
| Writing BDD Cases in Gherkin | X | X | X | X | X |
| Automating BDD Cases in a BDD Framework | X | X | - | - | X |
| Code Review of Automated BDD Cases | - | X | - | - | X |
| Automation of instructions, description of elements and pages, etc. | - | - | X | - | X |
| Conducting automation code review | - | - | X | - | X |
Thus, each in the process performs its role and functions:
- the new functionality is described by managers, analysts and technologists in a convenient format for them in the Gherkin language;
- Functional testers are busy writing case-quality tests, automate them and carry out code review of case studies themselves;
- Automators are busy only with pure automation and related automation work, and not writing cases, as is often the case.
5. Educate employees
Despite the fact that BDD-cases are written in a human-like language, and automation is carried out in a specially created framework, you will have to take time for minimal training of all participants in the methodology. Let's start with testers, without which it is very difficult to imagine the BDD methodology.
The role of testers in the process is to create high-quality BDD-cases and automate them in the BDD framework, or, as is often the case, automate “right on the spot”, directly in the framework. Testers should understand how to properly compose a BDD case, what it consists of, and how to work with it in the future. It requires knowledge of the language Gherkin. Information can be conveyed to the children either at a meeting with a presentation, or using self-study materials in your corporate wiki, or better, both.
The skills of testers with the framework is best to get with practice. Our experience has shown that the most effective "learning task". These are typical - testers will most likely encounter them during automation - tasks for a project in which BDD is being implemented. Each task is given a certain deadline for execution, after which collective meetings are held with a discussion of the results of the tasks and the problems that arise.
Training managers, analysts, technologists and others involved only in writing BDD-cases, is reduced to learning the Gherkin language, which they will use in compiling a description of tasks for developing a functional or its revision / change. Training differs little from training testers: a good presentation with an interactive help in the form of a discussion of the issues raised and a corporate wiki with all the necessary information will help.
Automators, as a rule, do not need training: the main thing is to convey to them the main points of the automation process, and to make it clear to them that it is impossible to limit the flight of fancy of testers, even if the cases they compiled look strange and incomprehensible to them. Testers have their own “kitchen”, they have their own unique view of the world.
It is worth noting that in the approach we have developed to the methodology, evolution from one role to another is possible. Thus, a functional tester, having gained the necessary experience in compiling BDD cases and in their automation, can become a code-reviewer. A code reviewer, wishing not only to write cases, but also a full-fledged code inside a BDD framework, can become an automator when he learns all the necessary skills. There were no precedents yet, but we, the automation specialists, started the automation training course among our BDD testers and hope that we will soon “grow” our automation tools. The minimum requirements are skills in any OOP language and a great desire to learn something new.
From all incoming “new” testing employees, from time to time, form BDD automation training groups, select mentors from trained people who will not only help the new ones to understand the methodology well, but will also play an important role in introducing new ones into your team. This also applies to all other participants in the methodology.
Remember that the results of the implementation of the methodology depend on a competent systematized approach to employee training.
6. Discuss the results and problems.
The results of the autorepress replenishment are discussed every week, when we sum up the results of the so-called “sprint” - the weekly segment, during which we record how the automation of our products develops and discuss the problems that arise. The meeting summarizes the past week, the quantitative indicators of automation are announced. For the BDD automation team, this is a great motivational tool that makes it clear to them that their work does not go unnoticed. , «», , , , .
, BDD- «BDD-», . , , , . , BDD- , , BDD-. , , , , .
— , «» , BDD-. : , « ».
, . ; , , .
— , . . , , . , , - , , , . , , . : , , , . , . . , , .
. , , , - . , : . «» .
— . , . — , . , . : « — — ».
, , - .
, , , . open-source . , Selenium open-source «» , . . CI-: - , - , - . : — . , , .
7.
BDD- BDD-. , — , «BDD-», BDD- . , 100% — BDD- (-, , ), BDD- , . :
, BDD-, , , , . , , . , / , , . , , BDD- . BDD- , : , BDD- .
BDD- BDD-, , . BDD-. BDD-, , . . , , . . , , . , .
BDD-, , «», . , N . , . TeamCity Selenium Grid.
, , . . BDD- . BDD- . .
Conclusion
BDD . , , , BDD BDD , , BDD «» .
, BDD : — , . BDD , , , , , .
, , , , , , . , , , .
Source: https://habr.com/ru/post/322688/
All Articles