How I chose Android for testing

Task:

The need for production raised the following two questions: how to determine the effectiveness of an existing set of devices for testing an Android application and how to correctly select the next device for testing.

Theory:

After some thought, I began to dig the Internet in search of a solution to this problem, but I did not find a ready-made solution (this was about six months ago). But there was enough theory and description of experience in this direction.

The three most useful links with descriptions of strategies for choosing devices for testing are located at the end of the article.

All the strategies described there can be reduced to a general theory, concluded in the following three points:

Use existing statistics (it is important to use current devices).
Take into account the parameters and characteristics of devices.
Parameters that can be taken into account:
- Screen size
- PPI (pixels per inch)
- screen resolution
- Android version
- speed (CPU, RAM)
- features of the design, such as the type of data entry (keyboard, touch, trackball, etc.), the presence of cameras ...
- browser
- manufacturer
- supported services (geolocation)
- price
Consider budget

')
Further, depending on the project, everyone chooses for himself the determining factors or their combinations.

The parameters that almost all bloggers take into account are Android version, PPI and screen resolution. For web-projects, the browser is important along with the listed parameters. For heavy native applications, speed is equally important ... and so on ...

The most logical and simplest strategy seemed to me the theory of popular and cheap (popular and cheap).

Practice. Stage 1:

Armed with knowledge, I proceeded to action.
The project, for which the obtained knowledge was applied, is used by approximately 1000 models of mobile devices every day more than 2 million times. And there were statistics on the number of visits for each device.

The stage of collecting data on devices turned out to be the most time-consuming. I collected information on the Internet about 123 popular devices, complementing the document with Android, CPU, screen sizes, resolutions and PPI for each device. This work took about a week.

It turned out that 123 out of 1000 devices cover 80% of visits, and I decided to stop there.

The stage of data analysis was the most interesting. Having noted the already existing devices (there were 7 of them) for testing in the table, I proceeded with the groupings on various parameters.

The grouping by major.minor of the Android version showed that all groups from 4.2 to 2.2 (76% of visits) are covered for testing, and only devices with Android 2.1 (2.95% of visits) and 1.x (0.28% of visits) are not covered. In general, the result is not bad.

A resolution analysis showed that the second most popular resolution was 480x320, with 18.61% of visits never checked. At the same time, the most popular group with a resolution of 800x480 (27.56%) is covered with 3 devices.

Analysis of Android versions and screen resolutions gave more interesting results. The three most popular groups are 2.3 and 800x480 (15.30%), 2.3 and 480x320 (11.83%), 4.1 and 1280x720 (10.75%) are not covered at all. In this case, the total coverage of a set of devices for testing was only 8% of visits.

As a result, a number of proposals were put forward that could significantly increase the coverage of visits on the basic parameters of devices.

On this my research in this area stopped.

Six months later ...

I recently returned to this issue while working on a new project. To be honest, the prospect of collecting information about devices did not please me.

One colleague saved me a lot of time by sharing the link choosedevice.com . This turned out to be a complete automated solution. In this project, information has already been collected on more than 200 devices, but only on 4 parameters: api level (os version), screen size, PPI and screen resolution. It is clear that this project is still young and is currently in beta testing.

Actions. Stage 2:

This is the experience gained when working with this service:

I went to the site using my google account.
Noted devices used for new project.
Made the export of mobile devices from google analytics.
Loaded this statistics in choosedevice.

All this took 10 minutes, and I received general statistics on mobile devices.

Next, I used the search bar to group devices.

The system calculated the coverage of user visits for the selected parameter and issued the recommended devices.

You can also group by two parameters. Below is the result of the ppi / resolution service.