Profiling of .NET Framework 4 NUnit tests

Everyone has probably come across application profiling, but how often have you had to profile tests?

As my personal experience has shown, in order to successfully accomplish this task for an assembly compiled under .NET Framework 4 , it is necessary to perform a series of actions that I had to spend on searching for a certain time. Therefore, I decided to summarize the experience gained into a single compilation and make it available so that others could avoid those rakes that I had to step on.

The task was quite real - it was necessary to measure the memory consumption when raising a large number of tests - in order to remove memory leaks.

In the working draft, NUnit 2.4 was used as a test environment, and .NET Memory Profiler 4.0 was selected as a profiler.
')

Test data

Make an assembly containing an example test.

To do this, in Visual Studio 2010 we will create a new WPF User Control Library project, add nunit.framework.dll to the references and create the TestClasses file for the tests.


Fig.1 References and project composition


Fig.2 Project options window

Let's write one single test, the code of which is given below:

using System; using NUnit.Framework; using System.Windows.Controls; using System.Windows; namespace WpfClassLibrary { [TestFixture] public class MyTests { [Test] public void TestForProfiling() { TextBox textBlock = new TextBox(); textBlock.Text = "UIElement Test"; Assert.IsTrue(textBlock.Text.Length > 0); MessageBox.Show("Ready for Collect Snapshot..."); } } } 

In this example, I added a code to the test to issue a message. This will more clearly show that the test code was actually called. For taking snapshots at different stages of passing a selected test, the placement of such messages may also be very useful - this will allow you to make the necessary pause when performing the test.

Profiler Setup

How to start profiling test?

Run the profiler and specify the console version of NUnit as the application to be launched. To do this, specify the file nunut-console.exe (in my case it was located in the following path: C: \ Program Files (x86) \ TestDriven.NET 3 \ NUnit \ 2.4 \ ).

It now remains to set the arguments. Just run nunut-console.exe with the "/?" to get us a list of interest. The file launch format looks like this:

NUNIT-CONSOLE [inputfiles] [options]

Ie, you must specify the path to the assembly with the tests, specifying the options. We will be interested in only one:

/ run = STR Name of the test to run

I want to note that the full name of the test must be specified ! The path to the test method, including the namespace.

We arrive at the fact that we write the following line as arguments for the profiler:

C: \ Sandbox \ WpfClassLibrary \ WpfClassLibrary \ bin \ Debug \ WpfClassLibrary.dll /run=WpfClassLibrary.MyTests.TestForProfiling



Fig.3 Profiler settings window for running the test

Everything seems to be ready, but the test is not performed under the given conditions. It's time to figure out why this is so ...

---

How to let NUnit run assemblies assembled in VS2010?

Let's try to run our test using nunit-console.exe, not a profiler.

As a result, we get the error System.BadImageFormatException :
Unhandled Exception:
System.BadImageFormatException: Could not load the file or assembly 'C: \ Sandbox \ WpfClassLibrary \ WpfClassLibrary \ bin \ Debug \ WpfClassLibrary.dll' or one of its dependencies. This assembly is built by a runtime. Newer than the currently loaded runtime and cannot be loaded.
File name: 'C: \ Sandbox \ WpfClassLibrary \ WpfClassLibrary \ bin \ Debug \ WpfClassLibrary.dll'


Because that our build is built under Framework 4, it cannot be loaded to run the test.

Find the file nunit-console.exe.config , located next to the console version of NUnit. On my machine it is located here:
"C: \ Program Files (x86) \ TestDriven.NET 3 \ NUnit \ 2.4 \ nunit-console.exe.config"

We are looking for the startup section, in which the description tells us:

<!--
The startup section may be used to specify the runtime versions
supported in the order that they will be used if more than one
is present.
-->

* This source code was highlighted with Source Code Highlighter .

This section contains information about support for the .NET Framework versions 1 and 2, but not 4.
Create your startup section (or modify the existing one) and add a line there to support launching the 4th framework build. The line to add looks like this:

< startup >
< supportedRuntime version ="v4.0.30319" />
</ startup >

* This source code was highlighted with Source Code Highlighter .

The first problem is solved - NUnit can now run tests compiled under .NET Framework 4.

---

Configuring the launch of the assembly with tests in the STA

Let's return to our test. It runs perfectly and runs if you run it directly from Visual Studio 2010 using the integrated TDD tools. If you try to run a test from nunit.exe or its console version (for the console version, we will not forget to specify the test assembly as a parameter), then the following error will occur:
MyTestLib.MyTests.TestForProfiling:
System.InvalidOperationException: The calling thread must be STA, because many UI components require this.



Fig.4 Error running test in NUnit

From the message and call stack, it becomes obvious that it is necessary to run tests in a streaming STA apartment. Those interested can read the details here or here .

Running tests for WinForms or ASP.NET may differ from the WPF example under consideration, but, nevertheless, there are a number of limitations (such as OLE drag-n-drop, working with a clipboard, etc.) that will require running tests in the STA .

In the newer version of NUnit 2.5, there is even a special RequiresSTAAttribute attribute for this purpose.

For those who are limited to using NUnit version 2.4, you can use configuration files .

For the assembly under test, create such a file indicating the need to run tests in the STA. I want to note that it must have a name that matches the name of the assembly and the postfix config , and must be located next to one another.

Create a file named WpfClassLibrary.dll.config with the following contents:

<? xml version ="1.0" encoding ="utf-8" ? >
< configuration >
< configSections >
< sectionGroup name ="NUnit" >
< section name ="TestRunner" type ="System.Configuration.NameValueSectionHandler" />
</ sectionGroup >
</ configSections >
< NUnit >
< TestRunner >
< add key ="ApartmentState" value ="STA" />
</ TestRunner >
</ NUnit >
</ configuration >

* This source code was highlighted with Source Code Highlighter .

The second problem is solved - the test build file now has a configuration file that determines the correct way to run from NUnit.

---

Profile test

It's time to check the result. Run the profiler and verify that the test was called.


Fig.5 Profiling dough

The problem is solved - the assembly with the test is correctly loaded and the test is started.

In addition, I want to say that there is another way to start the profiler. The essence of the method is that you can tell the system to start a specific process (for TDD, this is ProcessInvocation86.exe) under the debugger (profiler). This is done by writing the Image File Execution Options key in the registry.
But this is a topic for a separate article ...

That's all! I hope that this information will be useful and will save time on performing such a task.

Have a good testing and profiling!