Executable Specification: SpecFlow A to Z

This article is a continuation of the first part and reveals the technical details of working with the “executable specification” using SpecFlow .

To get started, you will need a plug-in to Visual Studio (downloaded from the official site) and the SpecFlow package (installed from nuget ).

So, our Product Owner asked the team to develop a calculator ...

@calculator Feature: Sum As a math idiot I want to be told the sum of two numbers So that I can avoid silly mistakes @positive @sprint1 Scenario: Add two numbers Given I have entered 50 into the calculator And I have entered 70 into the calculator When I press add Then the result should be 120 on the screen 

It is worth paying attention to @ attributes. They have several important properties. First, if you are using NUnit, SpecFlow add the attribute [NUnit.Framework.CategoryAttribute ("calculator")] . This is very convenient for making test plans. Category divisions are supported by R #, the native NUnit Runner and Team City.
')
Let's automate this script. By this time, the developers have already prepared the calculator interface:

 public interface ICalculator { decimal Sum(params decimal[] values); decimal Minus(decimal a, decimal b); decimal Sin(decimal a); decimal Multiply(params decimal[] values); decimal Divide(decimal a, decimal b); } 

Add the service testing context:

 public class CalculationContext { private readonly List<decimal> _values = new List<decimal>(); public ICalculator Calculator { get; private set; } public decimal Result { get; set; } public Exception Exception { get; set; } public List<decimal> Values { get { return _values; } } public CalculationContext() { Calculator = new Calculator(); } } 

To automate steps, SpecFlow uses special attributes:

 [Binding] public class Sum : CalcStepsBase { public CalculationContext Context {get;set;} public Sum(CalculationContext context) { Context = CalculationContext(); } [Given("I have entered (.*) into the calculator")] public void Enter(int digit) { Context.Values.Add(digit); } [When("I press (.*)")] public void Press(string action) { switch (action.ToLower()) { case "add": case "plus": Context.Result = Context.Calculator.Sum(Context.Values.ToArray()); break; default: throw new InconclusiveException(string.Format("Action \"{0}\" is not implemented", action)); } } [Then("the result should be (.*) on the screen")] public void Result(decimal expected) { Assert.AreEqual(expected, Context.Result); } } 

There are several advantages to this approach:

1. Each step needs to be automated only once.
2. You avoid problems with complex inheritance chains, the code looks much clearer
3. Attributes use regular expressions, so one attribute can “catch” several steps. The When attribute, in this case, will work for the phrase “add” and “plus”

All, all the steps are implemented, the test can be run directly from the .feature file.

Alternative recording

 @positive Scenario: Paste numbers Given I have entered two numbers | a | b | | 1 | 2 | When I press add Then the result should be 3 on the screen [Given("I have entered two numbers")] public void Paste(Table values) { var calcRow = values.CreateInstance<CalcTable>(); Context.Values.Add(calcRow.A); Context.Values.Add(calcRow.B); } public class CalcTable { public decimal A { get; set; } public decimal B { get; set; } } 

Such a record option can be handy when you need to fill a large object. For example, user account data.

Are we going to test only one data set?

Of course, one test is not enough; writing dozens of scenarios for different numbers is a dubious pleasure. Scenario Outline comes to the rescue

 @calculator Feature: Calculations As a math idiot I want to be told the calculation result of two numbers So that I can avoid silly mistakes @positive @b12 @tc34 Scenario Outline: Add two numbers Given I have entered <firstValue> into the calculator And I have entered <secondValue> into the calculator When I press <action> Then the <result> should be on the screen Examples: | firstValue | secondValue | action | result | | 1 | 2 | plus | 3 | | 2 | 3 | minus | -1 | | 2 | 2 | multiply | 4 | 

SpecFlow will substitute the values ​​from the table into the placeholder . Already not bad, but you still need to add automation:

 [When("I press (.*)")] public void Press(string action) { switch (action.ToLower()) { case "add": case "plus": Context.Result = Context.Calculator.Sum(Context.Values.ToArray()); break; case "minus": Context.Result = Context.Calculator.Minus(Context.Values[0], Context.Values[1]); break; case "multiply": Context.Result = Context.Calculator.Multiply(Context.Values.ToArray()); break; case "sin": Context.Result = Context.Calculator.Sin(Context.Values[0]); break; default: throw new InconclusiveException(string.Format("Action \"{0}\" is not implemented", action)); } } [Then("the result should be (.*) on the screen")] [Then("the (.*) should be on the screen")] public void Result(decimal expected) { Assert.AreEqual(expected, Context.Result); } 

We changed the second line for better readability. Therefore, on the Result method you need to hang the second attribute. At the exit you will receive 3 tests with reports of the form:

 Given I have entered 1 into the calculator -> done: Sum.Enter(1) (0,0s) And I have entered 2 into the calculator -> done: Sum.Enter(2) (0,0s) When I press plus -> done: Sum.Press("plus") (0,0s) Then the result should be 3 on the screen -> done: Sum.Result(3) (0,0s) 

And what about the negative tests?

Add a check of division by 0:

 @calculator Feature: Devision As a math idiot I want to be told the devision of two numbers So that I can avoid silly mistakes @negative @exception Scenario: Zero division Given I have entered 10 into the calculator And I have entered 0 into the calculator When I press divide Then exception must occur 

In this case, I do not want to interfere with flies with meatballs and for division I would prefer to have a separate file. The question arises. What to do with the context? He stayed in Sum class. SpecFlow supports injection into the constructor. Select the base class:

 public class CalcStepsBase { protected CalculationContext Context; public CalcStepsBase(CalculationContext context) { Context = context; } } 

We will inherit a new class with steps to divide from it:

 [Binding] public class Division : CalcStepsBase { public Division(CalculationContext context) : base(context) { } [When("I press divide"), Scope(Scenario = "Zero division")] public void ZeroDivision() { try { Context.Calculator.Divide(Context.Values[0], Context.Values[1]); } catch (DivideByZeroException ex) { Context.Exception = ex; } } [Then("exception must occur")] public void Exception() { Assert.That(Context.Exception, Is.TypeOf<DivideByZeroException>()); } } 

To ensure that the boarding does not conflict, we divide them into positive and negative

 [When("I press (.*)"), Scope(Tag = "positive")] [When("I press divide"), Scope(Scenario = "Zero division")] 

We can filter the scope by script and tags.

Data Driven Tests

Four examples of addition, subtraction and multiplication are not enough. There are many systems, with impressive volumes of input and output values. In this case, the sheet in the DSL will not look very clear. In order to write tests on the one hand it was convenient, and on the other, in order to save the GWT format you can go two ways:

1. Use BDDfy
2. finish SpecFlow to fit your needs

I stopped at the second version, so as not to support two formats:

 [TestFixture] [Feature( "Sum Excel", As = "Math idiot", IWant = "to be told sum of two numbers", SoThat = "I can avoid silly mistakes")] public class ExcelSumTests : GherkinGenerationTestsBase { [TestCaseSource("Excel")] [Scenario("Sum Excel", "excel", "positive", "calculator")] public void AddTwoNumbers_TheResultShouldBeOnTheScreen(string firstValue, string secondValue, string action, string result) { Given(string.Format("I have entered {0} into the calculator", firstValue)); Given(string.Format("I have entered {0} into the calculator", secondValue), "And "); When(string.Format("I press {0}", action)); Then(string.Format("the result should be {0} on the screen", result)); } public static IEnumerable<object[]> Excel() { return ExcelTestCaseDataReader.FromFile("Sum.xlsx").GetArguments(); } } 

Background and preconditions

Sometimes for a whole batch of scripts you need to specify a set of preconditions. Copying them into each script is obviously not convenient. Two approaches come to the rescue.

Background

 Background: Given Calculator is initialized @positive Scenario: Add two numbers Given I have entered 1 into the calculator When I press sin Then the result should be 0.841470984807896 on the screen 

In the Background section you can put all the big preconditions for scenarios.

Use of tags

Preconditions can also be implemented using tags:

 [Binding] public class Sum : CalcStepsBase { public Sum(CalculationContext context) : base(context) { } [BeforeScenario("calculator")] [Given("Calculator is initialized")] public void InitCalculator() { Context.Init(); } } 

The method marked with the BeforeScenario attribute will be executed before running the script. Attributes are passed to the constructor to limit the scope. We tagged scripts as a calculator tag. Now, before each launch of such a script, the InitCalculator method will be executed.

Reports

In order to build a report, you will need the utility specflow.exe and nunit. Below is the msbuild script that first runs nunit and then builds a specflow report.

 <?xml version="1.0" encoding="utf-8" ?> <Project ToolsVersion="4.0" DefaultTarget="Compile" xmlns="http://schemas.microsoft.com/developer/msbuild/2003"> <ItemGroup> <NUnitAddinFiles Include="$(teamcity_dotnet_nunitaddin)-2.6.2.*" /> </ItemGroup> <PropertyGroup> <teamcity_build_checkoutDir Condition=" '$(teamcity_build_checkoutDir)' == '' ">.</teamcity_build_checkoutDir> <NUnitHome>C:/Program Files (x86)/NUnit 2.6.2</NUnitHome> <NUnitConsole>"$(NUnitHome)\bin\nunit-console.exe"</NUnitConsole> <testResultsTxt>"$(teamcity_build_checkoutDir)\TestResult.txt"</testResultsTxt> <testResultsXml>"$(teamcity_build_checkoutDir)\TestResult.xml"</testResultsXml> <projectFile>"$(teamcity_build_checkoutDir)\Etna.QA.SpecFlow.Examples\Etna.QA.SpecFlow.Examples.csproj"</projectFile> <SpecflowExe>"C:\Program Files (x86)\TechTalk\SpecFlow\specflow.exe"</SpecflowExe> </PropertyGroup> <Target Name="RunTests"> <MakeDir Directories="$(NUnitHome)/bin/addins" /> <Copy SourceFiles="@(NUnitAddinFiles)" DestinationFolder="$(NUnitHome)/bin/addins" /> <Exec Command="$(NUnitConsole) /domain:multiple /labels /out=$(testResultsTxt) /xml=$(testResultsXml) $(projectFile)" ContinueOnError="true"/> </Target> <Target Name="SpecflowReports"> <Exec Command="$(SpecflowExe) nunitexecutionreport $(projectFile) /xmlTestResult:$(testResultsXml) /testOutput:$(testResultsTxt) /out:"$(teamcity_build_checkoutDir)/SpecFlowExecutionReport.html""/> <Exec Command="$(SpecflowExe) stepdefinitionreport $(projectFile) /out:"$(teamcity_build_checkoutDir)/SpecFlowStepDefinitionReport.html""/> </Target> </Project> 

It is worth paying attention to the flag / domain: multiple . It tells NUnit to start the assembly from the folder in which it is located. Otherwise, there may be problems with configs.

As a result, we get this report

Run on schedule in Team City

In the build setup, you need to specify a new artifact: our progress report:


Instead of the step with the launch of NUnit, we will use the msbuild script that we wrote earlier:

A new report tab will appear in Team City. We have it looks like this:

Non-automated scripts are shown in purple, green - successfully, passed, red - tests with errors.
Everything, it is necessary to put only the trigger on start of tests according to the schedule.

Synchronization with task tracker

TechTalk offers its commercial SpecLog product for requirements management and communication with task trackers. He did not suit us for several reasons. Now I am working on transparent communication of SpecFlow test scenarios with test cases in TFS. With Update 2 in TFS tags appeared. An interesting idea seems to be to use agreements in the abstract to communicate with the task tracker, for example: @ b8924 , tc345 . As soon as a working decision appears, I will write about it.

Technical aspects of working with Selenium WebDriver and SpecFlow

Described in this article .