UI Autotests for Xamarin
Testing is an essential element of any software development. A lot of time is allocated for it in the total amount of labor costs, but testing itself is a monotonous and monotonous thing, so it lends itself well to automation. In our today's article, we will look at how to use UI tests in your Xamarin projects, including large business applications.
I give the word to the author.
')
While the software used few third-party modules and had a primitive user interface, for most of the tasks, traditional unit-tests were enough that tested the performance of individual methods and classes. In mobile applications, unit-tests also got their use, but they are not so easy to cover modules and classes tied to the user interface or platform functionality. This will help us to special UI-tests that will simulate the action of a real user: data entry, clicking on the screen, swipe and other gestures.
If you have not yet had the opportunity to use Xamarin UI Tests in your projects, then briefly recall that with the help of this framework you can write automated tests in C # and run them both locally and in App Center Test Cloud. Currently supported by iOS / Android, on Windows UWP this is not yet working, but you can use Coded UI Tests or another framework instead of Xamarin UI Tests.
For a deeper acquaintance with automatic testing of mobile applications and Xamarin UI Tests, we recommend these articles on Habré:
These articles very much describe the first steps in mastering autotests, but we will move on and consider how best to use autotests on large projects and integrate them with the App Center to perform in the cloud on a wide fleet of devices.
Since UI-tests are not widely used in all teams on a regular basis, let's define what and how can be tested using autotests. To do this, we will go from those classes of errors and problems that are specific to mobile applications:
The curvature of some programmers, leading to bugs, is not a feature of mobile development, so we leave it outside our article.
Based on the frequent problem areas, autotests are reasonably divided into 3 main types:
We will discuss how to correctly organize the QA process using UI tests in our next articles, and today we will focus on practice - how to organize your code base with tests on Xamarin UI Tests and how to run tests in App Center.
The structure of the UI tests just added to the project is as follows:
The default structure is not very suitable for large projects where there can be a lot of tests. Therefore, we divide the logic into the description of the pages and the very logic of the tests that will operate on the pages. A similar structure is recommended by Xamarin in its examples:
An example of a project structure with a large number of tests:
For convenience, tests can be grouped by functionality, which they check. For example: MapTests, AutorizationTests, etc.
All visual elements (buttons, labels, lists, and other controls) in Xamarin.Forms applications with which interaction is possible, you need to set the AutomationId property through which the tests from them are accessed. If you are writing on the classic Xamarin iOS / Android, then you need to set the AccessibilityIdentifier properties in iOS and ContentDescription in Android. This should be done in the application code, and the tester himself can handle it.
It is necessary for more convenient work with UI-tests. Separating the logic of the behavior of pages (Pages) and the logic of the most tests (Tests) increases the readability of the code and allows you to further refine them further as the project progresses.
After creating the structure of the UI tests, you can start writing them. The tests are run from the unit test panel. Tests for Android can be run directly into Windows from Visual Studio, but for iOS, at least for debugging, they will need to be performed on a Mac using Visual Studio for Mac, and then launched in App Center.
The main tool for writing tests is the Repl method. It launches the console, in which we will enter the methods necessary for working with Xamarin UI Test.
The main methods include :
For more information on how to manually write tests using Xamarin UI Tests, you can find out in the official documentation:
Currently, Xamarin UI Tests cannot perform the following actions:
You will also need some additional actions when interacting with system dialogs, for example, when you receive Permissions.
In addition to manually writing tests, you can use the Xamarin Test Recorder tool, which will record your actions on the device (data entry, tapes, gestures) and automatically create UI test code in C #. As conceived by Xamarin, it should reduce the time spent on creating tests, but it is worth remembering that this tool is still in the Preview stage. Often does not see AutomationId, and tests recorded for iOS do not work on Android.
In real Xamarin Test Recorder projects, it’s better not to count for the time being, so we recommend getting comfortable with writing tests in C #.
So, with the writing of tests we figured out. Before embedding UI tests into your CI / CD pipeline for automatic start, you need to run and debug them locally. To do this, you can perform UI tests through the tool for working with unit tests in Visual Studio on both Windows (Android only) and Mac (Android + iOS).
In order to get to practice, let's consider a simple example of UI tests for local testing.
There is a simple application consisting of 4 screens. Let's write a test for it.
First, we compose the structure and describe BasePage, BaseTest and AppInitializer:
Then we describe each page (Page):
And only now we write the test itself:
If you did everything correctly, the test will succeed.
If you need to perform different actions on iOS and Android in one test, then OniOS and OnAndroid properties from BasePage will be useful. This allows you to write common scripts for both platforms:
The main advantage of UI tests is revealed when they are embedded in your daily product development process. You can make your farm devices for autotests, but it is expensive and very labor intensive. It is better to use ready-made, now there are different cloud farms, including those from Google and Amazon (see the 7 best device farms for testing mobile applications ).
Recall that, most recently, Microsoft combined the Hockey App services (feedback collection and distribution), Xamarin Test Cloud (automated UI testing on real devices) with developments from Visual Studio Team Services (build) and introduced a service called App Center.
If you have already used Xamarin Test Cloud, then the interface will seem familiar to you:
Information about a specific test includes step-by-step results with screenshots:
Start an acquaintance with the App Center is better for official documentation .
To run the tests themselves in the console, you must write the following commands:
You can see an example of launching on the AppCeter.ms website in the Test section. For our example application, it looked like this:
The minimum time to perform one test on one device takes about 1 minute, but it takes time to prepare the device and deploy your application to it. Thus, the minimum time to launch your project in the App Center will be about 5 minutes. The tests themselves will also take from 2 to 15 minutes, depending on the number of steps and the required time to wait for data from the server.
Often in practice there is a need to run various types of tests, such as Smoke, Acceptance and others. To do this, you must specify the category name (Category) for each set of tests (or an individual test) and run only the desired set of scripts in App Center.
And this is how the test launch line of the required category will look like:
This will allow you to run only those tests that are currently needed.
Autotest allows you to remove the itch of manual testing, and with the help of Xamarin UI Tests you can write and debug custom C # scripts directly in Visual Studio. Xamarin UI Test allows you to cover most user scenarios for iOS / Android, and then also run your tests on hundreds of real devices in the App Center.
Repository with an example .
Stay in touch and ask your questions in the comments!
Vyacheslav Chernikov - head of development at Binwell . In the past, he was one of the Nokia Champion and Qt Certified Specialists, currently he is the Xamarin and Azure platform specialist. He came to the sphere of mobile in 2005, since 2008 he has been developing mobile applications: he started with Symbian, Maemo, Meego, Windows Mobile, then switched to iOS, Android and Windows Phone.
Articles Vyacheslav you can also read the blog on Medium .
Pavel Grushevsky , Automated Testing Specialist , Binwell .
I give the word to the author.
')
While the software used few third-party modules and had a primitive user interface, for most of the tasks, traditional unit-tests were enough that tested the performance of individual methods and classes. In mobile applications, unit-tests also got their use, but they are not so easy to cover modules and classes tied to the user interface or platform functionality. This will help us to special UI-tests that will simulate the action of a real user: data entry, clicking on the screen, swipe and other gestures.
Xamarin UI Tests
If you have not yet had the opportunity to use Xamarin UI Tests in your projects, then briefly recall that with the help of this framework you can write automated tests in C # and run them both locally and in App Center Test Cloud. Currently supported by iOS / Android, on Windows UWP this is not yet working, but you can use Coded UI Tests or another framework instead of Xamarin UI Tests.
For a deeper acquaintance with automatic testing of mobile applications and Xamarin UI Tests, we recommend these articles on Habré:
These articles very much describe the first steps in mastering autotests, but we will move on and consider how best to use autotests on large projects and integrate them with the App Center to perform in the cloud on a wide fleet of devices.
Types of UI tests and Mobile problems
Since UI-tests are not widely used in all teams on a regular basis, let's define what and how can be tested using autotests. To do this, we will go from those classes of errors and problems that are specific to mobile applications:
- Various operating systems and their versions. The application should work correctly on a wide range of very different operating systems, each of which has its own characteristics and limitations. Frequent errors: the application does not start or crashes in any version of the OS.
- Different variants of the CPU architecture. Iron smartphones and tablets are constantly improving, but do not forget about the "old men" five years ago, which may be in the hands of your real users. Frequent errors: the application is not installed on too old, too new or rare (for example, Android x86) devices.
- Different screen resolutions devices. Regardless of the number of pixels or the aspect ratio, the interface of the mobile application should correctly “drag” and “carry”. Frequent errors: UI elements are cropped or clumsily displayed on the smallest or largest screens.
The curvature of some programmers, leading to bugs, is not a feature of mobile development, so we leave it outside our article.
Based on the frequent problem areas, autotests are reasonably divided into 3 main types:
- Smoke tests to run the assembled application on the “most” extreme iron options: the oldest and the newest. This will make sure that the update of a dependent library or a tick in the project settings is not broken.
- Acceptance tests to verify key user scripts. This will make sure that the last edits did not break at least those scenarios that real users will follow in most situations.
- Advanced tests to check the layout of all screens at various resolutions, as well as tests for any specific functionality. This also includes regression autotests.
We will discuss how to correctly organize the QA process using UI tests in our next articles, and today we will focus on practice - how to organize your code base with tests on Xamarin UI Tests and how to run tests in App Center.
Project structure with UI tests
The structure of the UI tests just added to the project is as follows:
The default structure is not very suitable for large projects where there can be a lot of tests. Therefore, we divide the logic into the description of the pages and the very logic of the tests that will operate on the pages. A similar structure is recommended by Xamarin in its examples:
- github.com/Microsoft/SmartHotel360-mobile-desktop-apps/tree/master/src/SmartHotel.Clients.UITests
- github.com/xamarinhq/app-evolve/tree/master/src/XamarinEvolve.UITests
- github.com/xamarin/test-cloud-samples
An example of a project structure with a large number of tests:
- Pages contain methods that describe the logic of the individual pages.
- Tests contains tests that use ready-made methods from Pages
For convenience, tests can be grouped by functionality, which they check. For example: MapTests, AutorizationTests, etc.
All visual elements (buttons, labels, lists, and other controls) in Xamarin.Forms applications with which interaction is possible, you need to set the AutomationId property through which the tests from them are accessed. If you are writing on the classic Xamarin iOS / Android, then you need to set the AccessibilityIdentifier properties in iOS and ContentDescription in Android. This should be done in the application code, and the tester himself can handle it.
It is necessary for more convenient work with UI-tests. Separating the logic of the behavior of pages (Pages) and the logic of the most tests (Tests) increases the readability of the code and allows you to further refine them further as the project progresses.
Run UI tests and basic methods
After creating the structure of the UI tests, you can start writing them. The tests are run from the unit test panel. Tests for Android can be run directly into Windows from Visual Studio, but for iOS, at least for debugging, they will need to be performed on a Mac using Visual Studio for Mac, and then launched in App Center.
The main tool for writing tests is the Repl method. It launches the console, in which we will enter the methods necessary for working with Xamarin UI Test.
The main methods include :
- tree - gives all the information on the elements on the screen;
- app.Tap - tap on the selected item;
- app.WaitForElement - waiting for an element to appear on the page;
- app.Scroll Down \ Up - scroll imitation;
- app.EnterText - text input;
- app.Query - gives all the information on the selected element on the screen;
- app.Screenshot - makes a screenshot of the screen.
For more information on how to manually write tests using Xamarin UI Tests, you can find out in the official documentation:
Currently, Xamarin UI Tests cannot perform the following actions:
- Internet connection and network management;
- Integration with other applications on the device is not available;
- Bluetooth / WiFi control functions, physical rotation are not available; device and simulate a variety of battery conditions;
- No interaction with system notifications;
- Limited work with the camera.
You will also need some additional actions when interacting with system dialogs, for example, when you receive Permissions.
Xamarin Test Recorder
In addition to manually writing tests, you can use the Xamarin Test Recorder tool, which will record your actions on the device (data entry, tapes, gestures) and automatically create UI test code in C #. As conceived by Xamarin, it should reduce the time spent on creating tests, but it is worth remembering that this tool is still in the Preview stage. Often does not see AutomationId, and tests recorded for iOS do not work on Android.
In real Xamarin Test Recorder projects, it’s better not to count for the time being, so we recommend getting comfortable with writing tests in C #.
Local testing
So, with the writing of tests we figured out. Before embedding UI tests into your CI / CD pipeline for automatic start, you need to run and debug them locally. To do this, you can perform UI tests through the tool for working with unit tests in Visual Studio on both Windows (Android only) and Mac (Android + iOS).
In order to get to practice, let's consider a simple example of UI tests for local testing.
There is a simple application consisting of 4 screens. Let's write a test for it.
First, we compose the structure and describe BasePage, BaseTest and AppInitializer:
Then we describe each page (Page):
And only now we write the test itself:
If you did everything correctly, the test will succeed.
If you need to perform different actions on iOS and Android in one test, then OniOS and OnAndroid properties from BasePage will be useful. This allows you to write common scripts for both platforms:
App Center
The main advantage of UI tests is revealed when they are embedded in your daily product development process. You can make your farm devices for autotests, but it is expensive and very labor intensive. It is better to use ready-made, now there are different cloud farms, including those from Google and Amazon (see the 7 best device farms for testing mobile applications ).
Recall that, most recently, Microsoft combined the Hockey App services (feedback collection and distribution), Xamarin Test Cloud (automated UI testing on real devices) with developments from Visual Studio Team Services (build) and introduced a service called App Center.
If you have already used Xamarin Test Cloud, then the interface will seem familiar to you:
Information about a specific test includes step-by-step results with screenshots:
Start an acquaintance with the App Center is better for official documentation .
To run the tests themselves in the console, you must write the following commands:
appcenter test run uitest --app “_” --devices "_DEVICE_SET" --app-path “__APK/IPA” --test-series "_TEST_SERIES" --locale "ru_RU" --build-dir “____UI_” --uitest-tools-dir “___test-cloud.exe”
You can see an example of launching on the AppCeter.ms website in the Test section. For our example application, it looked like this:
The minimum time to perform one test on one device takes about 1 minute, but it takes time to prepare the device and deploy your application to it. Thus, the minimum time to launch your project in the App Center will be about 5 minutes. The tests themselves will also take from 2 to 15 minutes, depending on the number of steps and the required time to wait for data from the server.
Several types of tests in one project
Often in practice there is a need to run various types of tests, such as Smoke, Acceptance and others. To do this, you must specify the category name (Category) for each set of tests (or an individual test) and run only the desired set of scripts in App Center.
And this is how the test launch line of the required category will look like:
This will allow you to run only those tests that are currently needed.
Total
Autotest allows you to remove the itch of manual testing, and with the help of Xamarin UI Tests you can write and debug custom C # scripts directly in Visual Studio. Xamarin UI Test allows you to cover most user scenarios for iOS / Android, and then also run your tests on hundreds of real devices in the App Center.
Repository with an example .
Stay in touch and ask your questions in the comments!
The authors
Vyacheslav Chernikov - head of development at Binwell . In the past, he was one of the Nokia Champion and Qt Certified Specialists, currently he is the Xamarin and Azure platform specialist. He came to the sphere of mobile in 2005, since 2008 he has been developing mobile applications: he started with Symbian, Maemo, Meego, Windows Mobile, then switched to iOS, Android and Windows Phone.
Articles Vyacheslav you can also read the blog on Medium .
Minute advertising. We invite you on April 13 to the Mobile DevOps in Practice master class, where you can talk with Vyacheslav Chernikov and learn how to organize the DevOps process in the mobile developers team.
Pavel Grushevsky , Automated Testing Specialist , Binwell .Source: https://habr.com/ru/post/351536/
All Articles