Automated testing of ASP.NET applications via CUITe - features

Immediately make a reservation - I am not a professional test automation engineer. However, since at the moment it turned out that this is my main occupation in work, I will be especially glad to receive comments from people with a specialization in this area.

The article is a brief description of CUITe for those who have not encountered, the use of this framework for testing an application with a front end on ASP.NET and the problems that have been encountered.

What is CUITe

As the description of the project on the codeplex says , this is a thin add-on above the UI Testing framework from Microsoft. In the description there are many advantages, but for me they come down to two: instead of UIMap, Object repository (more beautifully, definitions of UI objects separately from the rest of the code), and various syntactic sugar (everything is visual - we take control in the UI object and call his method).
The installation is trivial - we run the installer, Next-> Next-> Finish, connect to the project CUITe.dll - everything. Elements for interaction are found with the help of the company's CUITe Object Recorder ™ or manually (I prefer the latter). I will not give the basics of the record here - the article is not about that, there is a lot of information on the basics, which cannot be said about the problems described below (there will be interest - I will write a separate post on the basics).
')
So, not everything is so rosy.

Problems

At once I will say that the application on ASP.NET 3.5, all the tests took place on the Internet Explorer 8. Some problems may be associated with this. Colleagues say that the project on ASP.NET MVC (and the more modern technology stack in general) did not have similar problems.

Control misses

For each control element (control) there are methods for interacting with it, for example, in the drop-down list (Combo-box) you can select a value. Unfortunately, the engine sometimes misses. It happens by chance, somewhere in 10% of cases - it gives a lot of joy when it happens at the end of a 40-minute test (about the length of the tests a little later), and the whole test must be repeated anew.
The solution is to increase the value of the Playback.PlaybackSettings.DelayBetweenActions parameter. This is the time that the engine waits before doing the next action. By default it is 100 ms. I raised to 120, which significantly reduces the likelihood of a miss.

Tests fall, not having time to find an element, while the page loads

Can occur in brake applications. Throws UITestControlNotFoundException ().
Solution - use Playback.Wait (n), where n is the time in ms. If the user of the application agrees to wait for a few seconds for some action - the same test should wait.

Error screenshots always start page

By default, CUITe makes an application screen during an error - when it cannot find any element or fails Assert. Usefulness is difficult to overestimate. But suddenly all the screenshots were not with the place of the error, but with the start page of the application. In google I found a single topic on msdn where I complained about a similar problem. As you can see, the “solution” is “it should be so, it just runs over you and applies the wrong screenshot”. I doubt that this was a solution to the problem in my case, since in the folder where all the screenshots (LastRun) are stored, the necessary screenshots were not found anyway (but there were even more screenshots of the first page).
The solution - thank the gods, there is an event (event) Playback.PlaybackError, to which we are broadcasting a handler with the UITestControl.Desktop.CaptureImage(); method UITestControl.Desktop.CaptureImage(); . As a result, we get something like:

 string path = @"C:\ErrorScreens"; Image pic = UITestControl.Desktop.CaptureImage(); pic.Save(path + "\\" + "myerror__" + DateTime.Now.ToString("HH-mm-ss_ddd") + ".bmp"); 

As a result, we get the ErrorScreens folder with all the screenshots and with a custom name, instead of the standard packaged in folders with random names, in which there is another folder with the name of the agent. Plus, in the standard implementation, screenshots are transferred to folders at one time, so sorting by time does not work (you need to go into LastRun).

Problems with finding elements / Lack of necessary methods for elements

There are problems with finding drop-down lists (Combo-box). For simple lists ( CUITe_HtmlList ), there is no method for selecting an item at all.
The solution is to use elements from Microsoft.VisualStudio.TestTools.UITesting.HtmlControls, all HTML elements with the prefix CUITe_ are the superstructure above. For example, set the value in the drop-down list:

 public void SetMyComboBoxValue(string value) { HtmlComboBox myComboBox = new HtmlComboBox(this); myComboBox.SearchProperties[HtmlComboBox.PropertyNames.Id]= "my-element-id"; myComboBox.SelectedItem = value; } 

Heavy debugging of long test cases

Debugging the test, which falls on the 39th minute, without the ability to immediately go to the problem area - still that occupation.
Solution - If testers are used to checking some parts of the application only after 40-minute data entry on different pages - try to find a faster way. Have test examples ready for use. Automatic tests, like unit tests, should be short.

Engine instability

CUITe in conjunction with IE 8 is not the most stable option. Periodically there are falls with “Window handle is invalid” or crashes of the eighth donkey. The latter breaks down all subsequent tests.
No solution found at this time.

Conclusions and general recommendations

Coded UI and CUITe in particular is not a silver bullet. Although the advantages of Coded UI testing are obvious, you will not be able to completely replace good manual testing. Of course, this is really cool - set up automatic testing of nightly builds to know exactly what we broke yesterday. But if your environment is unstable - take the automation specialist in the team, or prepare for the fact that bringing tests to mind is not a trivial task.

Sources:
1. http://blogs.msdn.com/b/mathew_aniyan/archive/2009/08/10/configuring-playback-in-vstt-2010.aspx - List of Playback.PlaybackSettings tinctures.