Data-Driven MS-Test Tests for Unit and Acceptance Testing

I would immediately like to emphasize the fact that the world of unit tests and the world of acceptance tests through the user interface are very different worlds: with their own laws, different capabilities and limitations. And if the world of unit tests works to cover each part of the application separately and in isolation, tests through the user interface is an emulation of the user's work with the system, mostly by pressing buttons and typing, which eventually merge into larger business scenarios .



It rarely happens that even if the tool provides very good opportunities for unit tests, then these possibilities turn out to be practically inapplicable for UI tests.



It happened in my practice when I decided to use data-driven tests in the Ms-Test framework. In this article I will describe in more detail the problem and my solution, for which I still do not understand - I need to be proud or ashamed.

Data-driven approach in the world of unit tests

Suppose we have an int method ConvertToNumber (string input), which accepts a string with a number and returns the same number, only in an integer type.

')

Let us write one positive test, ( which brings joy ), for example, for input = “3” and check that the result will also be 3.



What else? Of course, the boundary values ​​and equivalence classes: “-1”, “0”, “MaxInt”, “- (MaxInt)”.

Well, what if the input gets a decimal? And if the line starts with a number, followed by the characters?



What should be the right decision in such different situations?

And the right decision is not.



Please note that different programming languages ​​handle such situations differently. For example, JavaScript as a result of the operation {“10” + 1} returns the string “101”, and as a result {“10” - 1} will be the number 9. And Perl will say: 11 and 9, respectively. But the absence of a “true” answer does not at all prevent us from coming up with the right behaviors and saying that from now on, this is the truth, and punching this truth with unit tests in stone.



But, have you really gathered to accumulate a dozen unit tests for one unfortunate method with a single parameter?

And why not create a table of input and expected values, and pass each row of such a table as a parameter into one single test.

At the beginning, I myself was very surprised that Ms-Test provides quite rich possibilities for this.

Example data-driven unit test for Ms-Test

For clarity of the example, in the implementation of ConvertToNumber, I will not use Convert.ToInt32 () - it would be too simple. Instead, I will offer my implementation:

static int ConvertToNumber(string input) { int result = 0; while (input.Length > 0) result = (input[0] >= '0' || input[0] <= '9') ? (result * 10) + input[0] - '0' + (((input = input.Remove(0, 1)).Length > 0) ? 0 : 0) : 0; return result; } 

Unfortunately, in Ms-Test, it is not yet possible to set the incoming and expected values ​​directly in the code, using attributes. But there is an alternative - to use data sources. And as such a source, you will not believe, we will use an Excel file.



Although, in fact, you can use any data source that can be accessed from .NET. Including CSV, XML and others.

But working with Excel is great! For example, our table might look like this:







The main thing - do not forget to select the desired fragment and format it as a "table with the title." Otherwise, magic will not work.



Then the unit test with the implementation of the connection with the Excel-table and the reading of the data will look like this:

 const string dataDriver = "System.Data.OleDb"; const string connectionStr = "Provider=Microsoft.ACE.OLEDB.12.0;Data Source=C:\\matrix.xlsx;Extended Properties=\"Excel 12.0 Xml;HDR=YES\";"; [TestMethod] [DataSource(dataDriver, connectionStr, "Shit1$", DataAccessMethod.Sequential)] public void TestMe() { string rowInput = TestContext.DataRow["Input"].ToString(); string rowExpected = TestContext.DataRow["Expected Result"].ToString(); string rowException = TestContext.DataRow["Exception"].ToString(); string rowComment = TestContext.DataRow["Comment"].ToString(); int actualResult = ConvertToNumber(rowInput); Assert.AreEqual(rowExpected, actualResult.ToString()); } 

Yes, I had to spend a little effort on setting up the connection and transforming the input data, but look at what visual results we got.







And to add a new test - just add a new line in Excel. And you don’t need to go into the code to see what exactly is covered with tests and what isn’t.



In addition, you can go to any of the tests to see the details of the error or the log.



Please note that even on my low-performing machine, all 14 tests passed in less than 1 second.



I don’t know about you, but I was just impressed with such features and visual results.

Note: In order for it to work, do not forget to copy the matrix.xlsx file to the root of the C: \ drive.

Data-driven approach in the world of testing through the user interface

To my great regret, after I took up the UI tests using Selenium WebDriver - my admiration for the mega-cool feature of Ms-Test came to an end.



The fact is that UI tests in any implementation are, by their nature, very slow compared to modular ones. It is impossible to simply take and call a function from the core of the system, easily juggling context and parameters. No ... in order to implement a complex user script - you need to click a lot of buttons.



Therefore, with all possible optimizations, one UI test can go from 30 seconds to tens of minutes. It all depends on the complexity of the script.



For example, if we assume that 1 out of 14 data-driven tests takes one minute, then the entire set will pass in 14 minutes.

Now imagine that one test will fall for some reason, for example, because the button on the page did not have time to appear ...

It does not matter, you say, because you can fix and get rid of only fallen tests.

Not. Ms-Test considers a pack of data-driven tests as one test. Therefore, in order to overtake 1 test - you have to run all 14 again. And not the fact that those tests that took place earlier - suddenly will not fall. And in the case of debugging, you will not only have to set a breakpoint, but set up a conditional breakpoint: Visual Studio allows this ... but it takes extra effort and time.



It was necessary to find a way that would allow only the fallen test to override. And I found such a way.

Problem solving: "cycle through inheritance"

To solve the problem, you need to add 2 files to the project: TestBase.cs and TestRows.cs .



Then in the namespace “MsTestRows.Rows. *”, The classes TestRows_ 01 ... TestRows_ 100 will appear, which contain from one to one hundred generated methods.



Inherit your TestClass from TestRows_NN with the required number of methods (NN).



Further, Visual Studio will ask you to implement two methods:

• GetNextDataRow () - which will return data for the next test
• TestMethod () - which will be called from the test with data obtained from GetNextDataRow ().

As a result, using such a perverted maneuver, I managed to achieve an opportunity to restart only the fallen test.

Source Code and Links

• Github project repository
• Demo code

Other materials on the topic:

• How to simulate RowTest with MS Test
• Enabling parameterized tests in MSTest using PostSharp
• MSTest data driven tests - Using Excel 2007 as a data source
• NUnit vs. MSTest: Parameterized Tests

---

Note 1: The TestRows.cs file contains 30,000 lines of code. If your salary bonus depends on the number of written lines of code, and the system will automatically charge you a bonus of $ 1 million, I think it will be fair if you send me at least 5% of this amount.



Note 2: Try to implement ConvertToNumber so that it passes all the tests.

Attention: stick to the “author's style”.