Mobile applications and their testers: all you need to know

Hello to you, Habr! My name is Maxim and I work in Trinity Digital's QA department. In the field of quality assurance, I have more than two years, I love mobile applications, their complexity and dynamism. In this article, I tried to make a relatively small list of tools, sources of information and skills that a mobile application tester should always have with us in our 2K17 year.



If the article is divided into parts, it will look like this:

• Sources of information for the most successful testing
• Tools to simplify the life of the tester
• Hints
• Application Delivery and Analysis
• Where to grow further, if zen comprehended

// Alarma - below the paragraph for managers and judges of statistics

')

In this article, I will not tell you what iOS and Android are, but one can not say what an important role mobile platforms play in our lives. If we look at the statistics on sales of PCs and smartphones , then we can see that the number of mobile phones is growing every year, but the PC is less and less in demand. However, it is not necessary to breed the controversy about the death of any of the platforms. As was well said in the article by Paul Adams - every business should find its own perfect balance between mobile and stationary type of work with information. In the meantime, managers ran away to solve business issues, I will continue.

// Paragraph for managers ended

Sources of information for testing

In a sense, the tester must be a small specialist in each separate field, like a penknife. To always be in the know, you need to enlist certain sources of information.

• These are pages of strict guidelines from Apple and Google for publishing in app stores. In general, the rules are in many ways overlapping and quite trivial, however, as they say, the devil is in the details. It is these details that you always keep track of. After all, if your application works with user data, performs cash transactions, or simply provides content for children, then on the day of release, get ready to see the deviation due to the small legal details that you, as a tester, should have thought about with the manager and the developer.

• Also it is necessary to follow the design guidelines from the same Apple and Google . Certainly, a design review is a designer’s task, however, the knowledge gained in these guidelines will allow you to better understand what the end user expects from you, allow you to better understand the platform’s philosophy, and also, subsequently, enable you to conduct high-quality UI and UX tests.
• Be sure to look through all the technical conferences of Apple and Google, because it is on them that you can find out where each platform is moving, what new tools it provides to both users and developers. You should be interested in any tools, because each of them will either create a final product, or it will itself be the goal of your testing.
• Just need to read as many books about testing and software development, various blogs on Habré and other sources [here could be your advertisement]

Tools to simplify the life of the tester

The life of a tester is diverse and multifaceted. In order not to drown in the flow of information and to be as effective as possible, there are many techniques and tools.

• Keeping test cases and storage of tickets. One of the final exhausts of any testing is tickets in special programs. Each bug not only wants you to find it, but also documented it. For this, a huge number of tools of various kinds of complexity and payment were created. Among the most popular are Jira , RedMine , TestTarantula , TestLink , TestCollab , QACoverage . The main problem of these services is that they are very similar to the stars in the sky - there are an infinite number of them and they are all similar from the outside, but you should take a closer look, and their difference leaves no doubt. We also use Rapid Reporter for research testing. With it, you can quickly record all the "roughness" that you noticed when traveling through the application.

• In order not to forget anything in the process of testing it is worth using one of the smart cards and heuristic cheat sheets . It is important to note that both of them only help you not to miss something important. If any of the points seems unnecessary to you for a particular project, then you can always create your own heuristics or maps.
• A bit of math and logic - if you are faced with the task of testing 4 different input fields + their different combinations between each other, then the PairWise 'inga method comes to the rescue . With this tool you can always create a set of unique and self-sufficient test cases.
• Be sure to equip yourself with logging tools, screen recording and screenshots. Suitable for OS tools and software from third-party developers.
• A little automation of the routine, namely the generation of random information , the creation of mail , a basic check on the correctness of the naming of your ticket.
• A little more automation with the help of “monkeys” - an easy-to-use mechanism for poking the entire application. There are versions for both ios and for android . However, it is worth noting that the monkey gives a small exhaust, it is impossible to catch some fundamental pains with it. Relatively successful, such monkeys help solve problems related to the load on the application (how it will behave if you press one button 1000 times).
• Make friends with Android Studio and Xcode, because in them you will be able not only to perform white-chewing testing, but also to fully work with the debugger and breakpoint.
• Manipulations with networks - most modern applications are tightly tied to work with the network, so it would be reasonable to use tools like
  
  
  • Network Link Conditioner is a handy and extremely simple tool for setting up the desired connection. Need imitation of a bad connection on a real device? You are welcome. I would like to test how the application will behave when part of data packets are lost? No problem.
  • For android everything is more complicated. You cannot make such settings on the device itself, so you either need to use the device emulator with a special setting or use the Network Link Conditioner (see paragraph above) on a real iOS device + from it to set Hotspot mode. Enough crutch method, but guaranteed to check the behavior of a real android device without foil and Faraday cells .
  • It is also worth arming with the Fiddler or Charles data proxying services - both are beautiful in their own way. If you need to look at the application data sent / received via the network, then the best tool is not found. Each of them will perform the function man-in-the-middle, which will allow you not only to view the entire contents of data packets, but also give you the opportunity to feel like a young hacker.
    
    PS also heard about WireShark , but leaving it to you as a homework;)
• Simulators and Emulators are testers best friends with a small farm of devices. If you encounter a bug that appears only on some exotic kind of resolution (see Chinese smartphones), then AVD and Genymotion for Android can help you, as well as Xcode Simulator for iPhone. It is important to note that simulators and emulators will not be able to replace you with real devices, in any case do not assume that successful tests on simulators = successful tests on real devices.

• The latter type of tool is absolutely indispensable in case you need to test the API. Since many mobile applications are a large software layer from requests to the server and processing the received responses, it would be reasonable for us, as testers, to check what the server sends from its API methods. Products such as Paw , Insomnia , SoapUI and, my personal favorite, Postman can come to the rescue. At first, all you need from each of them is a form of a different request to the server (Get, Put, Post, Delete) and the resulting response. If you receive a valid response to the request, then you can breathe easy and go to drink tea to continue testing further.

Hints

In any kind of testing, there are historically established bottle necks and knowing that you can find most of the errors in the shortest possible time. Also here I tried to describe some of the working points that will simplify the search for errors.

• Always check the application in portrait and landscape modes. Sometimes developers forget to turn off landscape support, or just skip the layout of any of the necessary elements. Not very nice to know when your favorite program crashes at the slightest turn of the screen.
• Be sure to see how the application behaves after going to sleep, collapsing, returning to active mode and abruptly leaving active mode (ringing). For Android, there is even a special option in the settings - Do not keep activities ( DNKA ). Be sure to test your application with the DNKA turned on and off, this significantly reduces the time for checking activities with activities. Hint - getting tickets for bugs that were found using DNKA, do not be too lazy to indicate that you used this tool, so it will be easier for the developer to reproduce your error.
• A huge number of applications using the services of the OS. These include the camera, phone book, microphone, and so on. In Android, the user gives access to these services when the application is loaded, and in iOS (as well as in Android since version 6.0), permissions are given after the program starts. All this means that we need to check not only the text of requests for services (agree that the unreasonable request for access to the camera looks strange), so it is also necessary to check the operation of the system if permission has not been given. As a life example, I always recall how the Internet call application, codenamed “N”, fell if it was not given access to the microphone.
• Notifications / notifications - how much is in these words. When dealing with notifications, you always need to remember that they can be both local and server-based (i.e. tied to a network connection). In addition, notifications should ideally be displayed on the screen referred to in the notification. Unfortunately, the practice is extremely common when clicking on a notification like “Hey, you received a message from Vasya”, you get to the main screen of the application, and not to the much-desired message from Vasily.
• I already wrote above that there are special tools for tests related to the network, but I would like to remind once again and indicate that checking how an application reacts to different types of communication and transitions from one state to another is extremely important. Most users of mobile devices do not sit in a cozy office near the wifi router, but walk down the street or go to the subway. Keeping this in your head, you can always find the crash of your application where the lazy ones simply would not notice.
• It is extremely important to see how the application behaves if you quickly switch between two screens (recall the monkey test above). By experience, I will say that in Android an extremely huge number of errors are due to the fact that during a quick transition between two screens, the required fields do not have time to fill with data and, subsequently, lead to crash.
• I can not say about the certificates. Mobile application developers use different certificates to sign the product under development (store certificate and test). Most of the time you will see an application that was signed with a certificate for the test, but when it comes to checking the version with another certificate, do not be lazy to complete a full regression of the application, because more than once it turned out that a fully working application started to fall at startup due to certificate change.
• Change the language and time of the device (especially time). Never expect that all users will be synchronized with the network and automatically switch to the correct time zone. If your application somehow needs to change over time (for example, food delivery in 30 minutes), then you have endless scope for activity.
• Always communicate with managers and developers. A few minutes of informal communication about the new assembly will be able to open a lot of new things for you, and at the same time they will simplify life. Also, before starting the build verification, I recommend asking the developers to check if they are in your place, what they have already seen and why they are sure that this should work smoothly. The main thing - do not fall for stories about resiliency and maximum stability, nature has not yet produced such products.
• Try to correctly predict the time for testing, and, last but not least, know how to stick with it. Find your work pace, enter it and gradually improve. In the end - learn to stop and switch between tasks.

Application Delivery and Analysis

Delivering applications to test devices is a very important part of testing, because you can’t test the latest actual build until it falls into your hands. To solve this problem, various services come to the rescue. Below I will share those that we managed not only to try in business, but also to find the perfect place in the development process for each of them.

• TestFlight and Beta testing in the Play Market is a great tool for testing builds after rolling up on the old versions of the application that you previously released. These distribution methods imply that most of the application has already been tested, so using them in the early stages of a project is pointless.
• HockeyApp , AppBlade , Appaloosa , TestFairy - convenient tools for distributing builds for both iOS and Android. Each of them supports the ability to download past builds, and some even allow you to collect analytics on usage and crashes. These tools can be used both in the earliest stages and before release. Also, it should be noted that many enterprise customers stop distributing their applications through these services, so do not be surprised if some of your products never get to the App Store and Play Market, but remain within one of these services.
• Google Drive, Yandex Drive, [insert your own] - the services from the second point can be replaced by the usual cloud storage, but in this case, management and version control will fall on your shoulders. If you feel mad enough organized, you can try to build all the processes on cloud disks. We at Trinity Digital use these services when we need to make a backup of a specific version of the application, or when testing at very early stages of development.

An equally important tool is the analysis of applications, in the end, who is forewarned is armed. That is why before the release of the product (and better and before the start of development) it is worthwhile to decide on those services that will allow you to keep your eye on the pulse. Here I also continued the theme of the sources of information from the last chapter.

• CrashLytics , Bugsee , AppSee and FireBase give a very detailed analysis of the use of your applications, almost instantly crash reports, and, of course, are full of various graphs and diagrams. Each of them, of course, has its own unique pieces. For example, Bugsee allows the tester to create a ticket with recording errors directly from a mobile device with an open application, and FireBase was originally created as a means of quickly deploying a backend. Considering all these features and chips, you can choose a tool that is more suitable for you.
• Reviews in the App Store and Play Market - unlike our colleagues from the world of web, we can always look at the real opinion of our users. Will you write about positive things as much as negative ones? Hardly. Will negative feedback always be helpful? I doubt it. Will the time spent on reviews be worth it? Of course, if you want to make a really high-quality product that is interesting to users. However, it is worth pointing out that if the whole team is not set up to work with reviews, then your initiative alone will not be enough. Cultivate and instill a love for reviews and working with them - it will definitely pay off in the long run.

• Introduce feedback forms in each application - so the user can always contact your team without spoiling the rating in the app store + the user will find it easier to navigate the history of email correspondence, rather than faceless stor.
• Sooner or later you will definitely need to organize a farm of devices. I will not go into details, but, briefly, you will definitely need to take into account such factors as the popularity of devices in your region, the popularity of devices among the audience of your application, missing OS and screen sizes that cause problems most, fashion trends for the next two year (for Soup you do not currently take a month), and, of course, one should always keep in mind that Android smartphones have a huge number of producers, each of whom is committed to - implement a custom firmware with the maximum the number of conflicts for external libraries. Your analysts, as well as services such as
  
  
  
  
  • In the case of Android - AppBrain , Apteligent data , a bit outdated statistics from OpenSignal , statistics from AnTuTu (do not forget that they have a benchmark service, so their picture of the world will always have more productive devices), detailed statistics from Google itself
  • For iOS, statistics from Apple itself, from the previously mentioned Apteligent data , and modest statistics from David Smith

Where to grow further

We all start with manual testing, but rarely anyone remains in the same position for many years. So where should the tester go when all the tools from the list have already been studied and want something new? In fact, each tester eventually moves in one of the following ways - development, autotests, manager, DevOps. Whichever way you like, all of them are united by some qualities that must be present in every self-respecting professional.

• Systems approach - any self-respecting tester should have a systematic approach to problems. Refer to the mathematical analysis and statistics as often as possible, they will give more room for professional growth. This also includes work with risks, prioritization and analytics.
• Communication - the tester is not worth a penny if he cannot competently tell about the results of his test. Tighten both the native language and foreign. The higher your communication skills, the faster and better the bugs will close.
• Team management - sooner or later you will have to manage a certain number of people from your line of business. Begin to look at how teams in other companies are organized, why some approaches should be used, and some you can skip.
• Automation - do not think that automation is just writing a mountain of code and a complete replacement for manual labor. You can remain a manual tester, but know how to quickly fill in 50 different fields with the help of python, which you are so lazy to fill in yourself every time. Does it automate? Of course, though the base. Do not be afraid of automation, but master it in every way, albeit in small steps.
• Trends - be sure to look at the applications of competitors, winners of various nominations from Apple and Google , follow the trends in social networks. Maybe your primary goal is quality, but, knowing the problems of competitors, you can avoid problems in yourself.
• Programming - learn to program. This will not only help you with automation, but also allow you to speak the same language with the developers. I will not begin to describe how important it is to know programming languages, just keep in your head that this is an important skill, without which it will only be more difficult.

Conclusion

As I said at the very beginning of the article - I tried to make a small list of all the most necessary for a beginner mobile tester. Now, looking at the article with my gaze, I understand that the whole article perfectly reflects the testing itself as a phenomenon. At first it may seem that it is tiny, but with each new step it acquires new details.



As a final word, I want to wish you more devices on the farm, smaller final edits and maximum pleasure from the process.



PS and the afterword will say a farewell why when testing it is important to remain attentive not only to the screen of the smartphone: