Writing automated tests to test the desktop user interface

In recent years, the topic of unit tests, regression testing, continuous integration, TDD, BDD, etc has become increasingly popular and more and more developers are beginning to actively use these techniques in their projects. In this case, a separate issue is the problem of automatic testing of the user interface in desktop applications. In this article I will try to consider already existing solutions, as well as to give the version of my bike writing tests for UI on .net.

Formulation of the problem

In the beginning, you need to make a reservation that the following techniques are designed for use by programmers who want to automate the verification process of the UI they created, so for the QA team, which does not even have basic programming skills, I'm afraid the article will be of little use. I also note that tests for UI can in no way be considered unit tests, should not be included in the TDD code writing cycle, and preferably should be run on a separate server during the build of the build (ideally, of course, after each commit) . Why not locally? Because it will be very slow, it will start to annoy and after a while the developer will just score on their launch.

We will have a simple task for example - there is an application with two buttons. By pressing the first text field, a certain text will appear (let it be “Habrahabr”). By clicking on the second, the current time and date will also be displayed there.

Accordingly, you need at least three tests for the following cases:

1. The original text in the text box when the application starts.
2. Text after pressing the first button.
3. Text after clicking on the second button.

Review of existing solutions

1. .Net UI Automation

The framework itself appeared quite a long time ago, along with the release of WPF, but I did not find proper coverage in blogs. UI Automation is a library of virtualization of the control tree of an arbitrary Win32, Windows Forms or WPF application, with the possibility of subsequent access to the properties of these controls for reading and writing. There is also support for input event emulation. If one of the readers worked with the Microsoft Active Accessibility library in due time, I would note that UI Automation is almost its direct heir.
In this library, each control is represented as an object of type AutomationElement , which provides us with methods for generating events, getting properties, and finding child elements. The very first AutomationElement object for our application window can be obtained using AutomationElement.FromHandle methods (process.MainWindowHandle) , where process is a link to the process of the application under test, or via the desktop:

AutomationElement.RootElement.FindFirst(TreeScope.Children, new PropertyCondition(AutomationElement.NameProperty, " " ));
* This source code was highlighted with Source Code Highlighter .

For specific controls, UI Automation provides a set of additional wrappers called AutomationPatterns , for example, ExpandCollapsePattern , SelectionItemPattern , etc., which allows you to use the functionality specific to these controls, for example, the ability to expand / collapse the expander.
')

Virtues

• Free
• Support from Microsoft.
• It is part of the .net framework.
• Ability to test Win32, Windows Forms and WPF applications.

disadvantages

• The need to specify the path to the executable file or the link to the window title is bad, because the path may change, and the title may not be unique.
• Limited set of available properties - in fact, we can work only with those types of controls and properties that Microsoft has virtualized. Any customization (for example, the use of complex compound controls) will complicate the process of writing tests.
• A possible disadvantage is that the application is launched as a third-party process, so if you suddenly need to change or change the behavior of an object in your system, the corresponding dances on the ears will begin. “But you don’t need to replace them, the tests should check the system in the same way that it will work for the user.” - you say and you’re partly right. But when it is necessary to check the behavior of the system under critical conditions (refusal to access the file at a certain moment), make sure that there are no memory spaces or at least test the output of the current date, it will refuse to do this without having access to the "guts" of the system not comfortable, long, and sometimes not at all possible.

Examples of tests for the task

The original text in the text box when the application starts

1. [TestMethod]
2. public void TestStartup ()
3. {
4. var appPath = Path.Combine (Path.GetDirectoryName ( Assembly .GetExecutingAssembly (). Location), @ ".. \ .. \ .. \ TestUI \ bin \ Debug \ TestUI.exe" );
5. var process = Process.Start (appPath);
6. try
7. {
8. Thread.Sleep (5000);
9. var mainWindow = AutomationElement.FromHandle (process.MainWindowHandle);
10. var buttonControl = mainWindow.FindFirst (TreeScope.Children, new PropertyCondition (AutomationElement.ControlTypeProperty, ControlType.Button));
12. var textBoxControl = mainWindow.FindFirst (TreeScope.Children, new PropertyCondition (AutomationElement.ControlTypeProperty, ControlType.Edit));
13. var textBox = (ValuePattern) textBoxControl.GetCurrentPattern (ValuePattern.Pattern);
15. Assert.AreEqual ( "123123123" , textBox.Current.Value);
16. }
17. finally
18. {
19. process.Kill ();
20. }
21. }

* This source code was highlighted with Source Code Highlighter .

Text after pressing the first button.

1. [TestMethod]
2. public void TestMethodUIAutomation ()
3. {
4. var appPath = Path.Combine (Path.GetDirectoryName ( Assembly .GetExecutingAssembly (). Location), @ ".. \ .. \ .. \ TestUI \ bin \ Debug \ TestUI.exe" );
5. var process = Process.Start (appPath);
6. try
7. {
8. Thread.Sleep (5000);
9. var mainWindow = AutomationElement.FromHandle (process.MainWindowHandle);
10. var buttonControl = mainWindow.FindFirst (TreeScope.Children, new PropertyCondition (AutomationElement.ControlTypeProperty, ControlType.Button));
12. var textBoxControl = mainWindow.FindFirst (TreeScope.Children, new PropertyCondition (AutomationElement.ControlTypeProperty, ControlType.Edit));
13. var textBox = (ValuePattern) textBoxControl.GetCurrentPattern (ValuePattern.Pattern);
15. Assert.AreEqual ( "123123123" , textBox.Current.Value);
17. var button = (InvokePattern) buttonControl.GetCurrentPattern (InvokePattern.Pattern);
18. button.Invoke ();
20. Assert.AreEqual ( "Habrahabr" , textBox.Current.Value);
21. }
22. finally
23. {
24. process.Kill ();
25. }
26. }

* This source code was highlighted with Source Code Highlighter .

After pressing the second button, the text cannot be realized, since the current time is always different, and we cannot lock something.

2. White project

Free framework from codeplex based on UI Automation . The advantages and disadvantages are the same; it differs only in a more convenient and expanded api for working with the control tree.

Examples of tests for the task

The original text in the text box when the application starts

1. [TestMethod]
2. public void TestStartup ()
3. {
4. var appPath = Path.Combine (Path.GetDirectoryName ( Assembly .GetExecutingAssembly (). Location), @ ".. \ .. \ .. \ TestUI \ bin \ Debug \ TestUI.exe" );
5. var application = White.Core.Application.Launch (appPath);
7. Assert.IsNotNull (application);
9. var window = application.GetWindow ( "MainWindow" );
10. var textBox = window.Get <White.Core.UIItems.TextBox> ();
12. Assert.IsNotNull (textBox);
13. Assert.AreEqual ( "123123123" , textBox.Text);
14. }

* This source code was highlighted with Source Code Highlighter .

Text after pressing the first button

1. [TestMethod]
2. public void TestWithWhite ()
3. {
4. var appPath = Path.Combine (Path.GetDirectoryName ( Assembly .GetExecutingAssembly (). Location), @ ".. \ .. \ .. \ TestUI \ bin \ Debug \ TestUI.exe" );
5. var application = White.Core.Application.Launch (appPath);
7. Assert.IsNotNull (application);
9. var window = application.GetWindow ( "MainWindow" );
10. var textBox = window.Get <White.Core.UIItems.TextBox> ();
12. var button = window.Get <White.Core.UIItems.Button> (SearchCriteria.ByText ( "Click for test" ));
13. button.Click ();
15. Assert.AreEqual ( "Habrahabr" , textBox.Text);
16. }

* This source code was highlighted with Source Code Highlighter .

3. Visual Studio 2010 Coded UI Test

Coded UI is a solution from Microsoft, which appeared in 2010 studios and has been repeatedly described, including on the habr, for example, here and here .

Virtues

• Presence of a recorder recording user actions for autogeneration of tests.
• The “out of the box” delivery for Visual Studio.
• Support from Microsoft, integration into TFS.
• Ability to work at the tree level controls, without reference to the coordinates of the screen.

disadvantages

In general, the set of flaws is the same as that of UI Automation . Separately, it is only necessary to emphasize that the opportunity to work is only in certain versions of the 2010 studio (Ultimate, Premium, Professional). And if the launch of tests is possible in all three, then the creation of the appropriate item type in the project and the launch of the recorder is possible only in Ultimate and Premium versions. And if you can buy the Ultimate version for your home projects and torrents , then for a commercial project, where we are talking about licenses for dozens of developers, such a step may come across misunderstanding on the part of higher-level bosses and accounting.

I will not give the test code, since it is autogenerated and therefore it is of no particular interest.

4. Test Complete and others like him

Systems like Test Complete can be summarized into one group. I will not describe them in detail, because This topic is a separate article, I will highlight only a few points. Their main advantage is a wide range of applications, no need for programming skills and the presence of a separate, not requiring Visual Studio, system for creating, storing and supporting tests. The disadvantages repeat the previous solutions - paid, related to the system under test as a “black box”, without the possibility of mocking, plus Test Complete has its own environment for running tests, so using normal mstest will not work.

Let's make something of our own

If you write the UI of your application on WPF, then to test it you can use the VisualTreeHelper class. The algorithm is quite simple - we run our application in a test method in a separate thread, we get the necessary control through VisualTreeHelper , we emulate events and read the values ​​for asserts.

For more convenient test creation, for myself I made a small utility class, which simplifies the execution of routine actions:

Application launch

var application = UI.Run(() => new App { MainWindow = new MainWindow() });

* This source code was highlighted with Source Code Highlighter .

Getting the property control. I will explain, because Only the thread in which they were created can work with controls; you have to make such a feint with your ears.

var window = application.Get(x => x.MainWindow);

* This source code was highlighted with Source Code Highlighter .

Well, search in the tree

var textBox = _mainWindow.FindChild((TextBox el) => el.Name == "SomeText" );

* This source code was highlighted with Source Code Highlighter .

It may also be necessary to check lightning, colors and other compositional things. Then you can get an image of the control in the old-fashioned way for comparison via RenderTargetBitmap .

1. private void AssertRender ( string expectImageName, FrameworkElement elementForTest)
2. {
3. var image = elementForTest.Render ();
4. var expectPath = Path.Combine (AppDomain.CurrentDomain.BaseDirectory, expectImageName);
5. if (! ( File .Exists (expectPath) && File .ReadAllBytes (expectPath) .SequenceEqual (image)))
6. {
7. File .WriteAllBytes (Path.Combine (AppDomain.CurrentDomain.BaseDirectory, "fail_" + expectImageName), image);
8. throw new AssertFailedException ( string .Format ( "Element {0} not equal to image '{1}'" , elementForTest.Get (x => x.Name), expectImageName));
9. }
10. }
12. AssertRender ( "button.png" , button);

* This source code was highlighted with Source Code Highlighter .

Virtues

• Free
• No need to install additional libraries.
• Allows you to mock parts of the system.

disadvantages

• No recorder for tests. We write everything with our hands.
• Support for tests falls entirely on your shoulders.

Examples of tests for the task

The original text in the text box when the application starts

1. [TestMethod]
2. public void TestStartup ()
3. {
4. var application = UI.Run (() => new App {MainWindow = new MainWindow ()});
5. var mainWindow = application.Get (x => x.MainWindow);
7. var textBox = mainWindow.FindChild ((TextBox el) => el.Name == "SomeTexBox" );
9. Assert.IsNotNull (textBox);
11. Assert.AreEqual ( "123123123" , textBox.Get (x => x.Text));
13. application.Invoke (x => x.Shutdown ());
14. }

* This source code was highlighted with Source Code Highlighter .

Text after pressing the first button

1. [TestMethod]
2. public void TestFirstButtonClick ()
3. {
4. var application = UI.Run (() => new App {MainWindow = new MainWindow ()});
5. var mainWindow = application.Get (x => x.MainWindow);
7. var textBox = mainWindow.FindChild ((TextBox el) => el.Name == "SomeTexBox" );
8. var button = mainWindow.FindChild ((Button el) => el.Content.Equals ( "Click for test" ));
9. button.Raise (ButtonBase.ClickEvent);
11. Assert.AreEqual ( "Habrahabr" , textBox.Get (x => x.Text));
13. application.Invoke (x => x.Shutdown ());
14. }

* This source code was highlighted with Source Code Highlighter .

Text after clicking on the second button.

1. [TestMethod]
2. [HostType ( "Moles" )]
3. public void TestSecondButton ()
4. {
5. var application = UI.Run (() => new App {MainWindow = new MainWindow ()});
6. var mainWindow = application.Get (x => x.MainWindow);
8. var dateTimeExpect = new DateTime (2011, 12, 08, 12, 30, 25);
9. MDateTime.NowGet = () => dateTimeExpect;
11. var button = mainWindow.FindChild ((Button el) => el.Content.Equals ( "Click for test 2" ));
12. button.Raise (ButtonBase.ClickEvent);
14. var textBox = mainWindow.FindChilds <TextBox> (). First ();
15. Assert.AreEqual (dateTimeExpect.ToString (), textBox.Get (x => x.Text));
17. application.Invoke (x => x.Shutdown ());
18. }

* This source code was highlighted with Source Code Highlighter .

Here I just locked the call to DateTime.Now using the Moles framework, an overview of which can be viewed, for example, here .

Creating an application and getting windows out of it can, of course, be added to the static constructor of the test class in order to speed up the test suite passage time and simplify their code.

Conclusion

Which way to choose to automate testing of UI should be decided on the basis of specific realities of the current project and own experience. I tried to highlight the main methods of writing tests for the UI, which I happened to test and show their strengths and weaknesses. I hope that this article will be useful for you.

Links

.Net UI Automation
White project
Visual Studio 2010 Coded UI Test
Test complete
Moles
Sources of the application under test + sample tests using VisualTreeHelper (to run the test with the date, you will have to install Moles)