SpecFlow and alternative testing approach

Testing with SpecFlow firmly entered my life, the list of necessary technologies for a "good project". Moreover, despite the fact that SpecFlow is focused on behavior tests, I came to the conclusion that integration and even unit tests can take advantage of this approach. Of course, people from BA and QA will no longer be involved in writing such tests, but only the developers themselves. Of course, for small tests this introduces a considerable overhead. But how much more pleasant to read the human description of the test, rather than the bare code.


As an example, I will give the test, revised from the usual type of tests in MSTest to the test in SpecFlow

Classic approach

The example above does not apply to the alternative approach, which I want to talk about in this article, it refers to the classical one. In this very classical approach, the “input” data for the test is specially created in the test itself. This phrase can already serve as a hint, what is the "alternative".
Another, slightly more complicated example of the classic creation of data for a test, with which you can then compare the alternative:

 Given that I have a insurance created in year 2006 And insurance has an assignment with type 'Dependent' and over 70 people covered 

These lines, which I will call steps, will correspond to the following lines with code:

 insurance = new Insurance { Created = new DateTime(2006, 1, 2), Assignments = new List<Assignment>() }; insurance.Assignments.Add(new Assignment { Type = assignmentType, HeadCount = headCount + 1 }); 

Also in the definitions of test steps there is a code that allows you to transfer these objects between steps, but for brevity this code has been omitted.
')

Alternative

At once I want to mention that this approach was found and applied not by me, but by my colleague. On the habr and on the githab, it is registered as a gerichhome . I took it to describe and publish. Well, maybe according to the Habra tradition, a comment will appear that is more useful than the article, it turns out that it was not for nothing that he wrote and wasted time.
In some cases, as in the case of our project, a considerable amount of data is needed to display the portal page. And to test some specific features only a small portion is needed. And, so that the rest of the page does not fall due to lack of data, you will have to write a considerable amount of code. And, even worse, you will most likely have to write some SpecFlow steps. Thus it turns out that you want, you do not want, but you have to test the whole page, as it were, and not the part that is necessary at the moment.
And in order to circumvent this, the data can not be created, but searched among the available ones. The data can be either in the test database or in the mock files collected and serialized on a slice of some API. Of course, this approach is more suitable for the case when we already have a lot of functionality, at least the part that will allow to manipulate this data. So that, if there is no necessary data set for the test, you could first go through the test scenario with your hands, make a data cast, and then automate it. It is convenient to use this approach when there is a desire and / or need to cover the existing code with tests, then refactor / rewrite / expand and not be afraid that the functionality will break.

As before, the test requires an Insurance object, created in 2006 and having an Assignment with the type of Dependent and the number of people covered by more than seventy. Any of the insurance stored in the database contains many other entities, in our project the model occupied more than 20 tables. As part of the demonstration, I did not use the full model, my simplified model includes only three entities.
In order to find the necessary insurance, you need to somehow determine the source in the form of IEnumerable <Insurance> and change the definition of the steps to the following:

 insurances = insurances.Where(x => x.Created.Year == year); insurances = insurances.Where(x => x.Assignments.Any(y => y.Type == assignmentType && y.HeadCount > headCount)); 

Next, in the next steps, you need to open the portal, giving it the ID of the first insurance found in the HTTP request line, and actually test the necessary functionality. The specifics of these actions, of course, far beyond the bounds of this topic,

Search algorithm

So, the insurance is found, we opened the portal, checked that the name of the insurance is displayed on the UI in the required section, now we need to check that the other section displays the required number of tenants. And then the question arises, how in this step to find out what kind of Assignment allowed our insurance to pass on this condition? Which of, for example, five take to compare its HeadCount with the number on the UI?
For this, we would have to repeat this condition in the “Then” step, and duplicating the code is obviously bad. In addition, the conditions will have to be duplicated in the steps of SpecFlow, which is completely unacceptable.
Lyrical digression - we already had something similar on one of the projects, there was an sql query (for simplicity, let it be coming from configs), which searches for people returning the SSN list. According to the scenario, these people should have had a child over 18 years old. And business people discussed for a long time, almost cursing, could not understand why we could not decompose this query in order for a particular person to find those children who came under the condition. Could not understand why we need a second request. And since there is a vision of a brighter future, in which BA will write the text of the test, it is much more difficult to explain why duplication in steps is more difficult than eliminating this duplication, and this is the first task that is solved by the search algorithm.
In addition, with the usual search in the previous paragraph, the second step cannot be divided into two steps by SpecFlow. This is the second problem solved by the algorithm. It will be discussed later about this algorithm and its implementation.
The algorithm is schematically represented in the following picture:

Search works quite simply. The initial collection of root entities, in this case insurance, is represented as IEnumerable <IResolutionContext <Insurance >>. Only those insurances that meet their own conditions and have a collection of subsidiary entities that satisfy the conditions get into it. In order to designate these child entities, it is necessary to register a so-called. provider with lambda type Func <T1, IEnumerable <T2 >>.
Child collections may also have conditions, the simplest of which is Exists. With this condition, the collection will be considered valid if it contains at least one element that satisfies its own conditions.
Own conditions, they are filters, are lambdas of the type Func <T1, bool> in the simple case.
The picture shows the case where two insurance found suitable for all conditions, the first one has a number of objects Assignment, of which fit under conditions 4, and also a number of objects Tax, of which two approached. Also for the second insurance there were 3 suitable Assignment and 4 Tax.
And, although this seems fairly obvious, it should be noted that, apart from insurance that did not fit its own conditions, those insurances that did not find suitable objects for Assignment or did not find suitable Tax objects did not appear on the list.
Red arrows indicate the tree of interactions of specific elements. A specific Assignment element, there is only one link up, it “knows” only about the specific Insurance element that gave rise to it, and “does not know” either about the Assignments collection in which it is located, nor, especially about the Insurances collection, for an element collections of parent elements may exist.

From here on, there are descriptions and examples of the use of the implementation, which I developed and published in NuGet (link at the end). The colleague also created his own implementation of the search algorithm, which differs mainly in that it pulls the registration network into a chain, that is, a tree with a single branch. Its implementation has some of its features, which I do not have, and my own shortcomings. Also, its implementation has some unnecessary dependencies, which slightly hinder the decision making in a separate module. In addition, for political reasons, roofing felts for personal reasons, he is not particularly keen to publish his decisions, which would make it somewhat difficult to use him in other projects. I had free time and desire to embody some interesting algorithm. With this I approached the writing of this article.

Registration of entities and filters

To achieve maximum granularity, the steps are broken down as follows.

 Given insurance A is taken from insurancesSource #1 And insurance A is created in year 2007 #2 And for insurance A exists an assignment A #3 And assignment A has type 'Dependent' #4 And assignment A has over 70 people covered #5 

Such granularity may of course look somewhat redundant, but it does allow for achieving a very high level of reuse. When writing tests for the search algorithm, having 5-6 written tests, all the following tests for a good three-quarters consisted of reused steps.

To register such sources and filters, use the following syntax:

  context.Register() .Items(key, () => InsurancesSource.Insurances); #1 context.Register() .For<Insurance>(key) .IsTrue(insurance => insurance.Created.Year == year); #2 context.Register() .For<Insurance>(insuranceKey) .Exists(assignmentKey, insurance => insurance.Assignments); #3 context.Register() .For<Assignment>(key) .IsTrue(assignment => assignment.Type == type); #4 context.Register() .For<Assignment>(key) .IsTrue(assignment => assignment.HeadCount >= headCount); #5 

The context used here is (TestingContext context) injected into the classes containing the definitions. All the lines in the SpecFlow test are marked with numbers only to indicate the correspondence with the definitions, the order of arrangement of these lines can be any. This can be useful when using the “Background” feature. Such freedom of registration is achieved due to the fact that the tree of providers is built not during the actual registration, but at the first receipt of the result.

Retrieving Search Results

 var insurance = context.Value<Insurance>(insuranceKey); var insurances = context.All<Insurance>(insuranceKey); var assignments = context.All<Assignment>(assignmentKey); 

The first line returns the first policy that fits under all conditions, i.e. was established in 2007, and has at least one Dependent type Assignment in which there are 70 people.
The second line returns all policies that meet these conditions.
The third line returns all matching Assignment objects from matching policies. That is, the result does not contain suitable Covers from unsuitable policies.
The “All” method returns IEnumerable <IResolutionContext <Insurance >>, not IEnumerable <Insurance>. To obtain the latter, select the Value field using Select. The IResolutionContext interface allows you to get a list of matching child entities for the current parent. Example:

 var insurances = context.All<Insurance>(insuranceKey); var firstPolicyCoverages = insurances.First().Get<Assignment>(assignmentKey); 

It is important to mention here that, for any two pairs T1-key1 and T2-key2, the following condition is true - If from the context T1-key1, let's say it will be a variable

  IResolutionContext<T1> c1 

get a collection of contexts T2-key2

 IEnumerable<IResolutionContext<T2>> cs2 = c1.Get<T2>(key2) 

then you can call back to T1-key1 from any of the elements of this collection and the resulting collection will contain the original element.

 cs2.All(x => x.Get<T1>(key1).Contains(c1)) == true 

In addition, between c1 and cs2 elements, all the conditions indicated in the combined filters will be fulfilled.

Combined filters

In some cases, it is necessary to compare the fields of two entities. An example of such a filter:

  And assignment A covers less people than maximum dependents specified in insurance A 

The condition, of course, is unrealistic, like some others. I hope it does not confuse anyone, an example is an example.
The definition of such a step:

 context .For<Assignment>(assignmentKey) .For<Insurance>(insuranceKey) .IsTrue((assignment, insurance) => assignment.HeadCount < insurance.MaximumDependents); 

The filter is assigned to the entity farthest from the root of the tree, in this case it is an Assignment, and in the process of execution it searches for insurance, passing along the red arrow (in the first figure) upwards.
If two entities of the same type with different keys are involved in the filter, then an inequality filter is automatically applied between them.
that is, in the case of .For <Assignment> ("a") .For <Assignment> ("b"), the same entity will never fall into the lambda (a1, a2) => in both arguments.

I limited myself to the fact that two entities can be used in the filter, since most likely the condition using 3 entities can be broken into pieces. There is no technical limit, I can add a “triple” filter at will.

Collection filters

Several filters on the collection have already been laid, and one of them - Exists, has already been used previously. There are also filters DoesNotExist and Each.
They literally mean the following - the parent entity, in this case insurance, is considered to be suitable for the condition, if there is at least one child entity - the Assignment is suitable for the condition. This is for Exists. For DoesNotExist - if there is not one Assignment suitable for the conditions, and Each - if all the Assignment of this insurance are suitable for the conditions.
In addition, you can set your own filters for collections. For example:

 context.ForAll<Assignment>(key) .IsTrue(assignments => assignments.Sum(x => x.HeadCount) > 0); 

The filter for the collections, of course, includes only suitable Assignment, that is, those that first went through their own filters.

The second example involves comparing two collections.
SpecFlow text for example:

  And average payment per person in assignments B, specified in taxes B is over 10$ 

and the corresponding definition:

 context .ForAll<Assignment>(assignmentKey) .ForAll<Tax>(taxKey) .IsTrue((assignments, taxes) => taxes.Sum(x => x.Amount) / assignments.Sum(x => x.HeadCount) > average); 

Another method of testing

The trick is to first prepare a fully populated object, check that the page (meaning that we are testing the web application) works out the happy-path script successfully, and then using the same object in each subsequent test, one breaks something and check that the page issues a warning to the user. For example, a user entering under happy-path must have a password, mail, address, access rights, etc. And for bad tests, the same user is taken and the password is broken for him, mail for the next test, and so on.
The same technique can be used when searching for data:

 Background: Given insurance B is taken from insurancesSource And for insurance B exists an assignment B And for insurance B exists a tax B Scenario: No assignment with needed count and type Given there is no suitable assignment B Scenario: No tax with needed amount and type Given there is no suitable tax B 

The text I gave is not complete, only significant lines. In the example in the Background section, all the registrations required for the happy-path are specified, and in specific “bad” scenarios one of the filters is inverted. In the happy-path test, an insurance will be found in which there is a suitable Assignment and a suitable Tax. For the first "bad" test, insurance will be found in which there is no suitable Assignment, for the second, respectively, insurance, in which there is no suitable Tax. This inversion can be included as follows:

 context.InvertCollectionValidity<Assignment>(key); 

In addition, you can assign a key to any filter, then invert this filter using this key.

Logging failed search

In the case when many filters are specified, it is not immediately possible to understand why the search returns nothing. Of course, it is quite easy to fight this with the dichotomy method, that is, to comment out half of the filters and see what changes, then half from half, etc. However, for convenience, and to reduce time costs, it was decided to give the opportunity to print to the log that filter that invalidated the last available entity. This means that if the first of the three filters eliminated half of the entities, the second eliminated the second half, and the case did not even reach the third filter, then the second specified filter will be printed in this case. To do this, subscribe to the OnSearchFailure event.

That's all. The project is available on github. Github.com/repinvv/TestingContext
You can get the finished build from NuGet www.nuget.org/packages/TestingContext