Winium: now for Windows Phone

To automate testing under Windows Phone and Windows, there are no convenient and open tools that can be easily adapted to your needs. Those that are closed, limited and offer their own approach, different from the generally accepted standards like Selenium WebDriver.

My colleague skyline-gleb recently wrote on Habré , as we developed our selenium-like tool to automate the functional testing of desktop applications under Windows. In parallel, we have developed a similar tool, only for mobile platforms from Microsoft.

About why we started writing it, what benefits we received from using a single platform to automate testing for all mobile platforms and how to implement it on your project, read this article.

A bit of history

• October 2010. Windows Phone 7 came out. A year later, Expensify puts in open source WindowsPhoneTestFramework - BDD testing tool for native applications;
• October 2012, Windows Phone 8 is coming out. Microsoft has not yet released a tool for testing via UI;
• February - March 2014, we post the first prototype of WinphoneDrvier, the first open-source implementation of Selenium for native Silverlight applications under Windows Phone;
• In April 2014, Microsoft releases Windows Phone 8.1 and finally, almost 4 years later, they release the official toolkit for testing applications for Windows Phone via UI - CodedUI. But, of course, this is not a Selenium-compatible tool and it is available only in the most expensive Visual Studio subscriptions;
• In May 2014, Salesforce.com posted in open source windowsphonedriver , an implementation of Selenium for testing web applications under Windows Phone. At about the same time, we updated our driver to support Windows 8.1;
• In February 2015, we uploaded Winium.StoreApps, an updated version of winphonedriver, into the open source, which implements a significant part of the protocol commands and supports native StoreApps applications under Windows Phone 8.1. This is the driver we use in our processes.

Literally, we introduced our toolkit to Codefest 2015 , where the idea of ​​Winium CodedUi, the implementation of a CodedUI based Selenium driver that supports native and hybrid applications and, most importantly, device testing, was born from a discussion on the sidelines of Badoo from Sathish Gogineni.
')
At the start of the project, there was one open tool - Expensify / WindowsPhoneTestFramework, which did not work for us because it was incompatible with Selenium and had a non-standard API. Moreover, he was sharpened by BDD. During the development of the project, Microsoft managed to release its version of the toolkit - CodedUI, which again had its non-standard API, was sharpened by writing tests in Visual Studio in C #, was closed and paid (which does not scale well).

So, we remembered how events developed. Let's go back to Winium. Since the above tools did not suit us, we decided to write our own. This is how the Winium project was born, which began as a test automation tool for Windows Phone Silverlight applications and soon turned into a whole set of test automation tools for the Windows platform:

• Windows Phone Driver - driver for native Windows Phone Silverlight applications;
• Winium for Store Apps - driver for native Windows Phone Store Apps apps;
• Winium.StoreApps.CodedUi - CodedUI based driver for Windows Phone XAML-based applications;
• Winium for Desktop - driver for native Windows Desktop applications;
• Winium.Cruciatus is a library for automating Windows Desktop applications.

About Winium.Desktop and Winium.Cruciatus we already told on habr. And today we will talk about Winium for Store Apps (the successor to Windows Phone Driver) and Winium.StoreApps.CodedUi.

Winium.StoreApps

Main features and driver restrictions

Winium.StoreApps - the main implementation of the driver for mobile devices. We use it and develop it. The source code is open and available on Github .

Main features:

• implements the Selenium protocol for testing native StoreApps applications under the Windows Phone platform;
• Works with Json Wire Protocol. You can use selenium- or appium-bindings and write tests in the same way as for iOS or Android;
• supports the installation and launch of the application under test, as well as the loading of files into the local storage of the application;
• supports single-touch gestures;
• Provides a base inspector to view the UI of the application under test;
• supports Selenium Grid, which allows you to parallelize the execution of tests.

Limitations:

• only emulators are supported (although with minor changes the driver can install the application on the device and work with it, but for now we do not know how to fully simulate gestures and input on the devices);
• It is necessary to build the automation server into your application (i.e., connect the nuget package to the application and add one line of code to start the server on a separate thread. Yes, this violates the first Appium rule, but so far the option is better, except for CodedUI, it was not possible );
• only one session is supported, but the driver can be connected to the Grid to distribute tests and run them in parallel.

Winium.StoreApps supports all basic Selenium commands and can be embedded into the existing testing infrastructure built on the basis of Selenium / Appium. In general, it can already be actively used in a continuous process, which we are doing.

How it all works

In essence, Winium.StoreApps.Driver.exe is an HTTP server running via the REST protocol JsonWire / WebDriver. If necessary, the incoming driver commands will be proxied to the InnerServer test server embedded in the application under test.


The structure of the interaction between the tests, the driver and the application under test.

How to prepare the application and write tests

To run tests against our application, you must perform three simple steps:

• prepare the application;
• write tests (Ok, this is not so easy.);
• run tests.

Preparing the application

Everything is simple here: we connect the Winium.StoreApps.InnerServer nuget-package and initialize the automation server on the main thread after the UI is created. For example, in the MainPageOnLoaded method. The server is initialized on the main thread only to get the correct dispatcher. The server will operate on a different thread, except for directly accessing the UI.

After that, it would be good to ensure the testability of the application by writing names and identifiers for those elements that you plan to use in the tests, but this is not necessary.

That's all, it remains only to build the appx-package with your application.

We write tests

Tests are written the same way as for the web or mobile devices using selenium- or appium-bindings.

The first thing we need to do is create a new session. When creating a session, we can specify various desired properties. Here are some basic driver-supported (full list available on wiki ).

dc = { 'deviceName': 'Emulator 8.1 WVGA 4 inch 512MB', 'app': r'C:\YorAppUnderTest.appx', 'files': { { 'C:\AppFiles\file1.png': 'download\file1.png', 'C:\AppFiles\file2.png': 'download\file2.png' } }, 'debugConnectToRunningApp': False } 

• deviceName is the partial name of the device on which we want to run our tests. If empty, the first emulator from the list will be selected.
• app - the full path to the appx-package with the application under test, in which you have embedded the automation server.
• files - a dictionary of files that will be downloaded from the local disk to the local storage of the application.
• debugConnectToRunningApp - allows you to skip all the installation steps and download files to the application and connect to an already running application. This is very convenient when you run an application from Visual Studio, set breakpoint points there and want to debug an error that occurs in the application when you run one of the tests.

So, the session was created, the application was launched. Now we need to somehow find the elements if we want to interact with them. The driver supports the following locators:

• id - AutomationProperties.AutomationId;
• name - AutomationProperties.Name;
• class name - the full name of the class (Windows.UI.Xaml.Controls.TextBlock);
• tag name is the same as class name;
• xname - x: Name, Legacy Locator, is usually not supported by default bindings and requires modification of bindings for use, but allows you to search by name, which is usually prescribed for the element to access from the code.

To make it easier to find locators and a general understanding of the UI structure of the application under test, we made an inspector who can connect to the application and display the current state of the UI, as the driver sees it.


The main inspector window

The inspector is still able to do a little, but you can already see the screen shot, the UI tree, how the driver sees it, find out the element locators and their main properties: position, visibility, text.

So, we found the element, now we can do anything with it.

 #     element.text #       element.click() #    -   element.send_keys('Hello!'+Keys.ENTER) #       element.get_attribute('Width') #     element.get_attribute('DesiredSize.Width') #     ,      JSON element.get_attribute('DesiredSize') # '{"Width":300.0,"Height":114.0,"IsEmpty":false}' 

In the world of mobile phones, the usual click can not do, here we need gestures. We support the old and proven API from JSWP (but we will soon add support for the new Mobile WebDriver API). Already you can make flick and scrolls.

 TouchActions(driver).flick_element(element, 0, 500, 100).perform() TouchActions(driver).scroll(200,200).perform() #       ActionChains(driver) \ .click_and_hold() \ .move_by_offset(100, 100) \ .release().perform() 

And since we are introducing an automation server into the application, we can also do more interesting things. For example, call MS Acessability API commands:

 # direct use of Windows Phone automation APIs app_bar_button = driver.find_element_by_id('GoAppBarButton') driver.execute_script('automation: invoke', app_bar_button) list_box = driver.find_element_by_id('MyListBox') si = {'v': 'smallIncrement', 'count': 10} driver.execute_script('automation: scroll', list_box, si) 

This allows you to use the exact means of scrolling items instead of simulating gestures if necessary.
And you can also set the value of the public property, although we do not recommend doing this in tests:

 text_box = driver.find_element_by_id('MyTextBox') driver.execute_script('attribute: set', text_box, 'Width', 10) driver.execute_script('attribute: set', text_box, 'Background.Opacity', 0.3) 

But there are situations when it is justified. For example, in our tests - in order to quickly and accurately move the map to the desired position, we just use this API instead of moving the map with gestures.

This is not a complete list of commands supported by the driver. For a more detailed list and notes on the commands, read the wiki .

So, let's build a simple test based on all these commands:


In this example, we are creating a new session. Hide the on-screen keyboard (for demonstration and just not to interfere).

Switch to the second tab in the pivot element. We find there an input field with prompts and begin to enter text. After that, select one of the prompts in the list and check that the value in the input field matches the prompt value. And finally, close the session.

Running tests

It remains the simplest. Launch Winium.StoreApps.Driver.exe (which can be downloaded from GitHub ), run the tests with your favorite tester and enjoy the magic.

Demo .

Winium CodedUI

The main features and limitations of the driver. After Codefest 2015, we had an idea to create a prototype of a selenium-driver wrapping CodedUI. The idea was brought to life and is now available on Github .

Main features:

• does not require modification of the application under test. You can even test applications pre-installed or downloaded from the store;
• works on emulators and devices;
• compatible with Json Wire Protocol;
• supports native applications;
• already has limited support for hybrid apps.

Limitations:

• early prototype, therefore some stability problems are possible;
• Requires license for Visual Studio Premium or higher (for 2013, for 2015 - Business);
• one session (but we port multisession from StoreApps whenever possible).

How it all works

It works on the same principles as in StoreApps, but now instead of implementing the server in the application, we start the server as a separate background process via vs.test.console and CodedUI. This test server has access to the phone and the UI of running applications directly through the Accessibility API (for example, to search for items) and through the CodedUI API (for example, for gestures).


The structure of the interaction between the tests, the driver and the application under test.

How to prepare the application and write tests

Since this approach does not need to change the application being tested, it is possible to test both release versions of applications and pre-installed applications. Those. This driver version is as close as possible to Appium’s philosophy . This is a plus and a minus - because it imposes restrictions on access to some of the internals of the application.
No special preparation of the application is required. Tests are written and run the same way as for Winium.StoreApps.

Watch the demo video in which we automate the creation of an event in the standard pre-installed Calendar application.

What Winium gave us

What did the creation of this tool give us instead of using platform-specific?

Now all our mobile teams use uniform tools, which allows us to easily share experiences and build a unified toolkit and infrastructure to run our tests and not disperse attention across platforms. In particular, one experienced engineer was able to easily transfer from iOS automation to Windows Phone, share experience and train testing engineers, which significantly raised the level of the project.

From an infrastructure point of view, this allowed us to focus on creating a single tool (vmmaster) to provide a repeatable environment for on-demand testing, but this is a topic for a separate article.

But the most important thing is that it allowed us to start integrating our tests for different platforms into one project ( demo ).

In short, now we have ...

• fewer bicycles;
• less code duplication;
• support is simpler;
• better code quality;
• “Sharing” of knowledge;
• systems approach;
• faster development.

And of course, all this opensource, so you can also use this tool to automate the testing of your Windows Phone applications. Moreover, Winium.StoreApps can be used completely free of charge by downloading the latest release and installing emulators or the Visual Studio Community with a mobile SDK. But for Winium.CodedUi you need a paid version of Visual Studio Premium or higher.

Once again the link to the repository and other opensource 2GIS products .