Automate testing of Windows applications using .Net

Test automation.

Testing is an activity performed to evaluate and improve software quality. This activity, in general, is based on the detection of defects and problems in software systems.
Testing of software systems consists of dynamic verification of program behavior on a finite (limited) set of tests, selected appropriately from the usually performed actions of the application area and ensuring compliance with the expected system behavior.
The main approach when testing software is black box testing. With this approach, the tester does not know the internal structure of the program. The tester interacts with the program: enters data, presses buttons, manipulates other visual components and evaluates the results.

Since complete testing is usually impossible or difficult to implement, the task of the tester is to select data and a sequence of actions where the errors of the program under test are most likely to occur.
When developing software, testers conduct a large number of identical tests when conducting acceptance and regression testing. In this regard, test automation can save a lot of man-hours. When conducting automated testing, the task of the tester is to develop a test suite. With direct testing, testers can do more important and intelligent tasks. In addition, large sets of test items can be run at night, which will allow to intensify the development process and reduce the cost of the project. In the morning, testers analyze the results, refine or recheck some tests and issue the required reports for transmission to the project manager or developers.

.Net platform mechanisms

When developing Windows applications on the .Net platform, you can use the platform to create a lightweight and flexible tool that will run the developed tests. It is not necessary to create a complete solution, sometimes it makes sense to quickly create a set of automated tests that will be specified directly in the code. To quickly modify and add tests, it is necessary to create and use an assembly with various methods that will perform the necessary actions for the tester with the user interface of the program.
With automated testing of the user interface of Windows applications, two solutions are possible. The first of these is based on the Reflection mechanism.
The reflection mechanism allows you to get objects that describe assemblies, modules and types. Reflection can be used to dynamically create an instance of a type, bind a type to an existing object, as well as get the type from an existing object and call its methods or access its fields and properties.
The second approach is based on the low-level functions of the Win32 API libraries: FindWindow, FindWindowEx, SendMessage, PostMessage, and the P / Invoke mechanism (invoking unmanaged code).
Calling unmanaged code is a service that allows managed code to call unmanaged functions implemented in dynamic-link libraries (DLLs). Calling unmanaged code detects and calls the exported function.
The second approach works not only for .Net applications, but for any Windows applications with a user interface.

Test application

The application under test adds two integers and, after pressing the button, displays the result. If the addition failed for any reason, the word “Error” is displayed instead of the result.




')
Access to the code when testing the "black box" is not required. We can not use our code and get all the necessary information from the assembly and the running application. But when accessing the code, some procedures are simplified.
The application is located at D: \ visual studio 2010 \ Projects \ WindowsFormsApplication1 \ WindowsFormsApplication1 \ bin \ Debug \ AUT.exe
The name AUT is the common abbreviation for Application Under Testing.

Testing based on the mechanism of reflection.

When using the reflection mechanism, you must load the assembly, get the form type, create an instance, and start the application with this form in the new stream.
If the tester does not have access to the source code, then you can use Red Gate's Reflector utility. With its help you can see the names of the form, controls and methods.



This way you can see that the form class is called Form1. The text fields are named textBox1 and textBox2, the button is button1, the result is output in a Label with the name label1. The method of processing a button click is called button1_Click and accepts object and EventArgs parameters as input.
To create an automated test data knowledge of the internal structure of the application is enough.
To increase code reuse, we will create a series of classes and methods.
The test application is launched using the StartApplication method.

static Form StartApplication( string path, string formName)
{
Form result = null ;
Assembly a = Assembly .LoadFrom(path);
Type t = a.GetType(formName);
result = (Form)a.CreateInstance(t.FullName);
ApplicationState aps = new ApplicationState(result);
ThreadStart ts = new ThreadStart(aps.RunApp);
Thread thread = new Thread(ts);
thread.Start();
return result;
}

* This source code was highlighted with Source Code Highlighter .

At the entrance passed the path to the assembly and the name of the form. The specified assembly is loaded, the form type is determined, an instance of the form is created and with its use the application is launched in the new stream: the delegate and the method for launching in the new stream are specified.
Class ApplicationState.

class ApplicationState
{
public readonly Form formToRun;

public ApplicationState(Form f)
{
this .formToRun = f;
}

public void RunApp()
{
Application.Run(formToRun);
}
}

* This source code was highlighted with Source Code Highlighter .

For reuse it is necessary to define a number of methods, fields and delegates.

static BindingFlags flags = BindingFlags.Public |
BindingFlags.NonPublic |
BindingFlags.Static |
BindingFlags.Instance;

delegate void SetControlPropertyValueHandler(Form f, string controlName, string propertyName, object newValue);

static void SetControlPropertyValue(Form f, string controlName, string propertyName, object newValue)
{
if (f.InvokeRequired)
{
f.Invoke(
new SetControlPropertyValueHandler(SetControlPropertyValue),
new object [] { f, controlName, propertyName, newValue }
);
}
else
{
Type t1 = f.GetType();
FieldInfo fi = t1.GetField(controlName, flags);
object ctrl = fi.GetValue(f);
Type t2 = ctrl.GetType();
PropertyInfo pi = t2.GetProperty(propertyName);
pi.SetValue(ctrl, newValue, null );
}
}
static AutoResetEvent are = new AutoResetEvent( false );
delegate void InvokeMethodHandler(Form f, string methodName, params object [] parms);
static void InvokeMethod(Form f, string methodName, params object [] parms)
{
if (f.InvokeRequired)
{
f.Invoke(
new InvokeMethodHandler(InvokeMethod),
new object [] { f, methodName, parms }
);
}
else
{
Type t = f.GetType();
MethodInfo mi = t.GetMethod(methodName, flags);
mi.Invoke(f, parms);
are.Set();
}
}
delegate object GetControlPropertyValueHandler(Form f, string controlName, string propertyName);
static object GetControlPropertyValue(Form f, string controlName, string propertyName)
{
if (f.InvokeRequired)
{
object iResult = f.Invoke(
new GetControlPropertyValueHandler(GetControlPropertyValue),
new object [] { f, controlName, propertyName }
);
return iResult;
}
else
{
Type t1 = f.GetType();
FieldInfo fi = t1.GetField(controlName, flags);
object ctrl = fi.GetValue(f);
Type t2 = ctrl.GetType();
PropertyInfo pi = t2.GetProperty(propertyName);
object gResult = pi.GetValue(ctrl, null );
return gResult;
}
}
delegate object GetControlHandler(Form f, string controlName);
static object GetControl(Form f, string controlName)
{
if (f.InvokeRequired)
{
object iCtrl = f.Invoke(
new GetControlHandler(GetControl),
new object [] { f, controlName }
);
return iCtrl;
}
else
{
Type t1 = f.GetType();
FieldInfo fi = t1.GetField(controlName, flags);
object gCtrl = fi.GetValue(f);
return gCtrl;
}
}
delegate object GetFormPropertyValueHandler(Form f, string propertyName);
static object GetFormPropertyValue(Form f, string propertyName)
{
if (f.InvokeRequired)
{
object iResult = f.Invoke(
new GetFormPropertyValueHandler(GetFormPropertyValue),
new object [] { f, propertyName }
);
return iResult;
}
else
{
Type t = f.GetType();
PropertyInfo pi = t.GetProperty(propertyName);
object gResult = pi.GetValue(f, null );
return gResult;
}
}
delegate void SetFormPropertyValueHandler(Form f, string propertyName, object newValue);
static void SetFormPropertyValue(Form f, string propertyName, object newValue)
{
if (f.InvokeRequired)
{
f.Invoke(
new SetFormPropertyValueHandler(SetFormPropertyValue),
new object [] { f, propertyName, newValue }
);
}
else
{
Type t = f.GetType();
PropertyInfo pi = t.GetProperty(propertyName);
pi.SetValue(f, newValue, null );
}
}

* This source code was highlighted with Source Code Highlighter .

The InvokeRequired property gets a value indicating whether the calling operator should access the invoke method during method calls from the control, because the calling operator is not in the thread in which the control was created [4].
InvokeMethod - calls the method specified in the parameters with the specified parameters.
SetControlPropertyValue - sets the property of the specified control.
GetControlPropertyValue - gets the value of the specified property of the control.
SetFormPropertyValue - sets the specified form property.
GetFormPropertyValue - gets the value of the form property.
GetControl - gets the specified control.
Using these methods, you can create a testing application.

static void Main( string [] args)
{
string path = @"D:\visual studio 2010\Projects\WindowsFormsApplication1\WindowsFormsApplication1\bin\Debug\aut.exe" ;
string nameForm = "AUT.Form1" ;
Form myForm = StartApplication(path, nameForm);
string f1 = "521" ;
string f2 = "367" ;
SetControlPropertyValue(myForm, "textBox1" , "Text" , f1);
SetControlPropertyValue(myForm, "textBox2" , "Text" , f2);
object ctrl = GetControl(myForm, "button1" );
InvokeMethod(myForm, "button1_Click" , ctrl, EventArgs .Empty);
string res = GetControlPropertyValue(myForm, "label1" , "Text" ).ToString();
string resTest = "FAIL" ;
if (res == "888" ) resTest = "PASS" ;
Console .WriteLine( "{0} + {1} = {2}. Test {3}" , f1, f2, res, resTest);
}

* This source code was highlighted with Source Code Highlighter .

The required values ​​are set in the text fields, then the button1_Click method is called with the required parameters. After that, the value of the Text property of the label1 control is obtained and it is compared with the required one, after which information about the test is displayed on the screen.
Only a special case of using the reflection mechanism is given, the tester can complement the test suite.

Testing based on calling unmanaged code.

For testing, a number of functions from the user32.dll library are used.
To import functions, use the DllImport attribute.

[DllImport( "user32.dll" , EntryPoint = "FindWindow" , CharSet = CharSet.Auto)]
static extern IntPtr FindWindow( string lpClassName, string lpWindowName);

[DllImport( "user32.dll" , EntryPoint = "FindWindowEx" , CharSet = CharSet.Auto)]
static extern IntPtr FindWindowEx( IntPtr hwndParent, IntPtr hwndChildAfter, string lpszClass, string lpszWindow);

[DllImport( "user32.dll" , EntryPoint = "SendMessage" , CharSet = CharSet.Auto)]
static extern void SendMessage1( IntPtr hWnd, uint Msg, int wParam, int lParam);

[DllImport( "user32.dll" , EntryPoint = "PostMessage" , CharSet = CharSet.Auto)]
static extern bool PostMessage1( IntPtr hWnd, uint Msg, int wParam, int lParam);

[DllImport( "user32.dll" , EntryPoint = "SendMessage" , CharSet = CharSet.Auto)]
static extern int SendMessage3( IntPtr hWndControl, uint Msg, int wParam, byte [] lParam);

* This source code was highlighted with Source Code Highlighter .

These functions will allow you to find the main form handler, find the required controls, enter text, click the buttons and get the value of the control.
For reuse, we define a series of methods.

static void ClickKey( IntPtr hControl, VKeys key)
{
PostMessage1(hControl, ( int )WMessages.WM_KEYDOWN, ( int )key, 0);
PostMessage1(hControl, ( int )WMessages.WM_KEYUP, ( int )key, 0);
}
static void ClickOn( IntPtr hControl)
{
PostMessage1(hControl, ( int )WMessages.WM_LBUTTONDOWN, 0, 0); // button down
PostMessage1(hControl, ( int )WMessages.WM_LBUTTONUP, 0, 0); // button up
}
static void SendChar( IntPtr hControl, char c)
{
uint WM_CHAR = 0x0102;
SendMessage1(hControl, WM_CHAR, c, 0);
}
static void SendChars( IntPtr hControl, string s)
{
foreach ( char c in s)
{
SendChar(hControl, c);
}
}
static IntPtr FindWindowByIndex( IntPtr hwndParent, int index)
{
if (index == 0)
return hwndParent;
else
{
int ct = 0;
IntPtr result = IntPtr .Zero;
do
{
result = FindWindowEx(hwndParent, result, null , null );
if (result != IntPtr .Zero)
++ct;
} while (ct < index && result != IntPtr .Zero);
return result;
}
}
static List < IntPtr > GetAllControls( IntPtr hwndParent)
{
IntPtr ctrl = IntPtr .Zero;
List < IntPtr > res = new List < IntPtr >();
ctrl = FindWindowEx(hwndParent, ctrl, null , null );
while (ctrl != IntPtr .Zero)
{
res.Add(ctrl);
ctrl = FindWindowEx(hwndParent, ctrl, null , null );
}
return res;
}
static IntPtr FindMainWindowHandle( string caption)
{
IntPtr mwh = IntPtr .Zero;
bool formFound = false ;
int attempts = 0;
do
{
mwh = FindWindow( null , caption);
if (mwh == IntPtr .Zero)
{
Thread.Sleep(100);
++attempts;
}
else
{
formFound = true ;
}
} while (!formFound && attempts < 25);
if (mwh != IntPtr .Zero)
return mwh;
else
throw new Exception( "Could not find Main Window" );
}

* This source code was highlighted with Source Code Highlighter .

The main task is to determine the order of controls. You can get a pointer to a control if its name / title / caption property is known. If this property is unknown, or it is not unique, you must use the FindWindowByIndex method. The zero control is the window itself. Next, the controls follow in the order of addition:

this .Controls.Add( this .label1);
this .Controls.Add( this .button1);
this .Controls.Add( this .textBox2);
this .Controls.Add( this .textBox1);

* This source code was highlighted with Source Code Highlighter .

But since it is assumed that the source code is unknown, it is necessary to use the SPY ++ utility. It shows the order of controls.



The first number is the Label, the second button, the third and fourth text fields.
Based on this information, code is generated to test the application.

static void Main( string [] args)
{
string path = @"D:\visual studio 2010\Projects\WindowsFormsApplication1\WindowsFormsApplication1\bin\Debug\aut.exe" ;
string nameForm = "AUT" ;
Process p = Process.Start(path);
//
IntPtr mwh = FindMainWindowHandle(nameForm);
//
IntPtr tb1 = FindWindowByIndex(mwh, 3);
IntPtr tb2 = FindWindowByIndex(mwh, 4);
//
string f1 = "521" ;
SendChars(tb1, f1);
string f2 = "367" ;
SendChars(tb2, f2);
//
IntPtr btn = FindWindowByIndex(mwh, 2);
ClickOn(btn);
//
Thread.Sleep(150);
IntPtr lbl = FindWindowByIndex(mwh, 1);
uint WM_GETTEXT = 0x000D;
byte [] buffer = new byte [256];
string res = null ;
int numFetched = SendMessage3(lbl, WM_GETTEXT, 256, buffer);
res = System.Text. Encoding .Unicode.GetString(buffer);
// Label
string resTest = "FAIL" ;
if (res == "888" ) resTest = "PASS" ;
Console .WriteLine( "{0} + {1} = {2}. Test {3}" , f1, f2, res, resTest);
Thread.Sleep(3000);
p.CloseMainWindow();
p.Close();
}

* This source code was highlighted with Source Code Highlighter .

Conclusion

The development of a complete tool for automated testing was not the purpose of this article. This article only showed the mechanisms of the .Net platform that are used for automated testing. Outside of this article are questions related to storing test suites and saving results. The questions of choosing the purpose of testing, data, problems of the oracle were not considered.

List of sources

1. swebok.sorlik.ru/4_software_testing.html IEEE Guide to Software Engineering Body of Knowledge, SWEBOK, 2004. Basics of Software Engineering. Translation Sergey Orlik.
2. Sam Kaner, Jack Folk, Eng Kek Nguyen. Software testing. Fundamental concepts of business application management. DiaSoft 2001.
3. msdn.microsoft.com/ru-ru/library/ms173183.aspx - Reflection (C # and Visual Basic).
4. msdn.microsoft.com/en-ru/library/system.windows.forms.control.invokerequired.aspx - .NET Framework Class Library. Property Control.InvokeRequired.
5. msdn.microsoft.com/en-ru/library/26thfadc.aspx - Using unmanaged DLL functions.
6. James D. McCaffrey. Net test automation recipes: a problem-solution approach. books.google.com/books?id=3vN9zsMLvxkC
7. www.automatedtestingstitute.com/home/index.php?option=com_content&task=view&id=1312&option=com_content&Itemid=1000 UI Automation Beneath the Presentation Layer. Bj rollison. Link to the magazine with the full version of the article. www.automatedtestinginstitute.com/home/ASTMagazine/2010/AutomatedSoftwareTestingMagazine_June2010.pdf
8. msdn.microsoft.com/en-us/magazine/cc163864.aspx Lightweight UI Test Automation with .NET. James McCaffrey.
9. habrahabr.ru/blogs/net/58582 NET in an unmanaged environment: platform invoke or what LPTSTR is.
10. PInvoke.net

Project source code