Automate mobile apps with SeeTest Automation
The article is published on behalf of Bateeva Ekaterina, Neifmetus
The world does not stand still: ten years ago, users went to the Internet using a computer, and today, according to statistics, most of the visits fall on the mobile version of the site or mobile applications. Therefore, the requirements for stability and usability of these applications are becoming higher.
')
In our bank, we also observe this trend: the mobile application uses an order of magnitude more users compared to the Internet bank. Therefore, there was an urgent need to automate the testing of a mobile application. Despite the widespread use of mobile applications, tools for automated testing are far from ideal. In addition, we place high demands on them. For example, the most popular Appium framework - an open-source solution that supports Android and iOS platforms - did not suit us. Our developers used many fashionable libraries, and sometimes they had to interact with the application at a lower level. UI Automator and UI Automation proved to be more difficult to deploy, each application used its own language to write tests, which caused problems in the redistribution between platforms in the auto-test team.
At the start of the automation process, our team had two people for two projects and 100 test scenarios for each platform and rather tight deadlines. We were looking for a solution, which would take us a minimum of time to develop and finalize. And we also hoped that the new employees would quickly learn the solution.
Our attention was attracted by a little-known solution from the company “Experitest” - SeeTest Automation. This is a cross-platform test solution. The system integrates with WebDriver (Selenium), HP UFT / QTP, JUnit, Microsoft Visual Studio and others. SeeTest Automation supports iOS, Android and Windows mobile platforms, both for smartphones and tablets.
To connect devices using a simple and intuitive interface.
The interface has a list of added devices. For each device, its status (online / offline), the name and version of the operating system are displayed. Applications for testing are installed through this interface, and for each device you can see the installed test applications and their versions.
With connected phones, you can open the screen and see everything that happens on the device. In developer mode, the device responds to mouse and keyboard controls. Screen display is updated automatically in real time. Also on the screen of the device is available inspector (Object Spy), where you can see the tree of elements.
Here you can see both the basic and added attributes of documents. For example, for elements that are difficult or impossible to identify by parameters such as text, label or id, we ask developers to specify additional tags with a unique value.
The inspector allows you to generate a unique xpath element using both the text of the element and ignoring the text. The generated xpath is stable in 95% of cases. Object Spy also provides the ability to enter your xpath and displays the items found.
Another useful feature of the inspector that allows calling the public methods of an object is RunNativeAPICall. With its help, you can create and run scripts based on the Native API public methods of the class of this element.
Like many other tools, SeeTest allows you to test mobile applications in “Record and Replay” mode. I am not in favor of this approach for autotests: the simplicity of writing such tests is illusory. Fortunately, SeeTest has an alternative option - using the library to interact with devices. Supported languages ​​are Java, Python, Perl, Ruby. But, if the user is weak in development and it is difficult to write tests, SeeTest has code generation for supported languages ​​and test frameworks such as TestNG or JUnit. We chose Java language, deployed SeeTest, connected devices and began to automate our mobile bank.
The process of writing tests turned out to be quite simple and similar to automated testing web'a. As a runner for tests, we chose TestNG.
Each test is created as follows:
As a result of the run, a report is generated in which all the running tests are grouped. The Summary Report page contains statistics on launches, the name, duration and result of each test, and if the test has fallen, then the time of the fall and a description of the error. In each test, you can go and see the detailed information. By default, all the logs within one test fall in succession in one list. But, if the test is a little more than the usual authorization on the form, the Client object has the functionality to combine the tests into groups. Groups can be given a clear description, they can be opened and collapsed, which makes it easier to work with long reports.
Connecting a large number of devices is limited. To develop tests, you need developer licenses for a machine or for each specific user. To run tests, agents are required (Executor Add-on). The agent is not tied strictly to the device, rather thread to run tests. If you need to use more than a couple of devices, Experitest suggests using their device farm or deploying SeeTestCloud locally. Cloud makes it very convenient to monitor and operate the connected devices. They are displayed in the SeeTestAutomation interface: you can keep track of which are connected, which are busy, are available at the moment, you can add and delete new devices. For devices connected to the Cloud, it is also possible to watch online actions on the phone online.
Parallelization of tests is implemented, but is fraught with some difficulties. Since when you start the test, you create a connection to the device, the Client object has access to a specific device. The Executor Add-on allows you to run different tests on different devices at the same time, but to perform the same test on different devices you need to parallelize yourself.
Support favorably distinguishes paid solutions. In SeeTest, support is a weighty argument in favor of choosing this particular tool for testing. If you have any problems with setting up, deploying, updating the system, you can call the call-center, and experts will explain and show everything. If the testing revealed a framework bug, notify the developers about it, and they will create a task to fix it, informing them of the approximate timing of the task.
When our team was looking for a means to automate, I confess, SeeTest was a bit scary because it’s pretty little known about it, and in Russia it’s not very common. But, starting to use it, we found that it is a good boxed solution with a flexible configuration for “its” needs and great functionality. SeeTest allows you to perform both standard and rather exotic manipulations. Although we still continue to explore open-source solutions, as long as they do not fully meet our requirements.
The world does not stand still: ten years ago, users went to the Internet using a computer, and today, according to statistics, most of the visits fall on the mobile version of the site or mobile applications. Therefore, the requirements for stability and usability of these applications are becoming higher.
')
In our bank, we also observe this trend: the mobile application uses an order of magnitude more users compared to the Internet bank. Therefore, there was an urgent need to automate the testing of a mobile application. Despite the widespread use of mobile applications, tools for automated testing are far from ideal. In addition, we place high demands on them. For example, the most popular Appium framework - an open-source solution that supports Android and iOS platforms - did not suit us. Our developers used many fashionable libraries, and sometimes they had to interact with the application at a lower level. UI Automator and UI Automation proved to be more difficult to deploy, each application used its own language to write tests, which caused problems in the redistribution between platforms in the auto-test team.
At the start of the automation process, our team had two people for two projects and 100 test scenarios for each platform and rather tight deadlines. We were looking for a solution, which would take us a minimum of time to develop and finalize. And we also hoped that the new employees would quickly learn the solution.
What did we choose?
Our attention was attracted by a little-known solution from the company “Experitest” - SeeTest Automation. This is a cross-platform test solution. The system integrates with WebDriver (Selenium), HP UFT / QTP, JUnit, Microsoft Visual Studio and others. SeeTest Automation supports iOS, Android and Windows mobile platforms, both for smartphones and tablets.
To connect devices using a simple and intuitive interface.
The interface has a list of added devices. For each device, its status (online / offline), the name and version of the operating system are displayed. Applications for testing are installed through this interface, and for each device you can see the installed test applications and their versions.
With connected phones, you can open the screen and see everything that happens on the device. In developer mode, the device responds to mouse and keyboard controls. Screen display is updated automatically in real time. Also on the screen of the device is available inspector (Object Spy), where you can see the tree of elements.
Here you can see both the basic and added attributes of documents. For example, for elements that are difficult or impossible to identify by parameters such as text, label or id, we ask developers to specify additional tags with a unique value.
The inspector allows you to generate a unique xpath element using both the text of the element and ignoring the text. The generated xpath is stable in 95% of cases. Object Spy also provides the ability to enter your xpath and displays the items found.
Another useful feature of the inspector that allows calling the public methods of an object is RunNativeAPICall. With its help, you can create and run scripts based on the Native API public methods of the class of this element.
Like many other tools, SeeTest allows you to test mobile applications in “Record and Replay” mode. I am not in favor of this approach for autotests: the simplicity of writing such tests is illusory. Fortunately, SeeTest has an alternative option - using the library to interact with devices. Supported languages ​​are Java, Python, Perl, Ruby. But, if the user is weak in development and it is difficult to write tests, SeeTest has code generation for supported languages ​​and test frameworks such as TestNG or JUnit. We chose Java language, deployed SeeTest, connected devices and began to automate our mobile bank.
The process of writing tests turned out to be quite simple and similar to automated testing web'a. As a runner for tests, we chose TestNG.
From theory to practice
Each test is created as follows:
- create a Client object, which we prescribe data for connecting to the device. This object will be a peculiar variation of the selenium-driver;
- on the device, we launch a test application;
- manipulate the application using the methods of the client object. Client allows you to click on the elements, enter text, read the text of objects. He also knows how to do such familiar mobile actions as swipe, long tap and others. The latest version of SeeTestAutomation has a useful synchronize function that allows you to wait until the screen is fully loaded without resorting to unstable custom solutions;
- on the device, we launch a test application;
- we log all the method calls of the client object, each method makes a screenshot of the screen, so it is clear which element was manipulated;
- add additional information if necessary. For example, you can display test data or make an additional screen;
As a result of the run, a report is generated in which all the running tests are grouped. The Summary Report page contains statistics on launches, the name, duration and result of each test, and if the test has fallen, then the time of the fall and a description of the error. In each test, you can go and see the detailed information. By default, all the logs within one test fall in succession in one list. But, if the test is a little more than the usual authorization on the form, the Client object has the functionality to combine the tests into groups. Groups can be given a clear description, they can be opened and collapsed, which makes it easier to work with long reports.
Licensing and Scalability
Connecting a large number of devices is limited. To develop tests, you need developer licenses for a machine or for each specific user. To run tests, agents are required (Executor Add-on). The agent is not tied strictly to the device, rather thread to run tests. If you need to use more than a couple of devices, Experitest suggests using their device farm or deploying SeeTestCloud locally. Cloud makes it very convenient to monitor and operate the connected devices. They are displayed in the SeeTestAutomation interface: you can keep track of which are connected, which are busy, are available at the moment, you can add and delete new devices. For devices connected to the Cloud, it is also possible to watch online actions on the phone online.
Parallelization of tests is implemented, but is fraught with some difficulties. Since when you start the test, you create a connection to the device, the Client object has access to a specific device. The Executor Add-on allows you to run different tests on different devices at the same time, but to perform the same test on different devices you need to parallelize yourself.
Support
Support favorably distinguishes paid solutions. In SeeTest, support is a weighty argument in favor of choosing this particular tool for testing. If you have any problems with setting up, deploying, updating the system, you can call the call-center, and experts will explain and show everything. If the testing revealed a framework bug, notify the developers about it, and they will create a task to fix it, informing them of the approximate timing of the task.
Conclusion
When our team was looking for a means to automate, I confess, SeeTest was a bit scary because it’s pretty little known about it, and in Russia it’s not very common. But, starting to use it, we found that it is a good boxed solution with a flexible configuration for “its” needs and great functionality. SeeTest allows you to perform both standard and rather exotic manipulations. Although we still continue to explore open-source solutions, as long as they do not fully meet our requirements.
Source: https://habr.com/ru/post/324766/
All Articles