Test analysis in mobile development

My name is Lena, I am the head of the Touch Instinct testing department.

Our company makes very different applications, so the quality requirements can vary greatly from project to project. So the set of test activities necessary to ensure the required level of quality can vary greatly. But it is impossible to effectively test the application without studying it.
')
I’ll tell you what analytical tasks arise before testing in Touch Instinct and how we solve them.

Documentation testing

The customer comes to us and tells us what product he would like to receive, what kind of functionality this product should have and what business goals to fulfill. So we get the business and user requirements for our product. Most often they are not documented and they are oral form. Based on these oral requirements, we think over the design and architecture of the IPA and mobile platforms. After this, the project manager compiles a list of tasks for development, and the analyst, if necessary, documents the functional requirements in a form that is optimal for the project.

At the exit we have the following artifacts:

• navshema, map application screens,
• API documentation
• the task list in the bugtracker,
• description of requirements.

All this is a set of functional requirements, based on which the developer will write the code, and the tester will check the product.

Obviously, there are risks - from the moment the customer voiced his requirements for the product until they were received by the developer, a lot of hands touched these requirements. In order not to get Uncle Fedor's letter, they need to be tested.

Testing requirements means checking for compliance with certain quality criteria . In this definition, it does not matter whether the requirements to be verified are documented and in what form. However, the testing process and quality criteria will vary depending on the type of requirements. Therefore, we test the documentation for a task for our department, and the entire team participates in testing the oral requirements throughout the project.

Verbal requirements can and should satisfy only two criteria - consistency and feasibility. But the quality criteria for documentation is much higher. From some of the criteria listed in the link above, we deliberately refuse in favor of saving resources. Thus, the guidelines and documentation of the API should satisfy the completeness criterion. A description of the requirements is optional, since such a document may be redundant in the presence of other documentation. Ideally, all documentation should be unambiguously interpretable. But it is often easier to leave a list of comments to the request in the form of question-answer than to force the analyst to endlessly rewrite the document until the wording is perfect.

Sometimes we may not even aim to test the requirements for compliance with any criteria. Not always a process built on a project implies such activity. However, the tester in any case should be familiar with all the documentation on the project before testing. In 90% of cases, awkward questions arise and there are inaccuracies in the requirements. Well, if this happens before the start of development. But most often the requirements go into development as soon as they appear. In testing, at about the same time, because the deadlines set by the customer do not allow to postpone development until the end of the documentation testing.

Testing documentation is the prevention of extra costs. Such activity can reduce the risks in the development.

Product decomposition

After reviewing the available documentation, we begin to design tests and study the product in more detail. First of all, you need to understand what components the product consists of. On what basis we will carry out the decomposition, depends on the goals set before testing on the project. For example, if the priority for the customer is the beauty and convenience of the application and great emphasis is placed on the UI, it makes sense to decompose the application screens. If the customer is more interested in working out certain scenarios, it is possible to decompose into use cases. If our goal is to conduct a full regression test, we can break the project into modules to further search for links and dependencies between them. To study logic and effective functional testing, we do a partitioning into objects (entities) and study possible actions on these objects.

Most often, when we start working on a project, some kind of decomposition is already done. For example, the scheme we get from designers, or the list of features from a project manager are examples of decomposition. Sometimes it is possible to get from the analyst any visual schemes that will help us understand the product. However, more often than not, this is not enough for effective work, both in the case of the need for test documentation on a project, and in the case of abandoning it.

Product decomposition is the first stage of test design, which consists in studying the components of the product and the connections between them. Such activity allows you to approach the testing system.

Regression Risk Analysis

This is not a stage as such, it is a process extended in time for all the work on the project. If there are resources, it can be documented to be 100% sure that nothing is missing anywhere. The point is to understand how editing in some part of the functionality can affect other functionality, assess the risk of breakdowns and test the functionality where these risks are.

By testing a black box for risk assessment, we can rely on the user logic of the product, personal experience, intuition and words of the developer. We can document known information about the links between functional blocks, create a table and note which functionality needs to be checked if there was an edit in a particular feature. However, most often we estimate the risks on the fly and do not document the links in order to save time.

Another thing is when an in-depth analysis of all the links between the modules inside the product is carried out in advance. This is no longer black box testing. Such activity requires both additional time and highly qualified specialists. The method is about the same - drawing up a table with marks of connections between functional blocks, only the analysis of the code serves as a basis. At the moment, we have not encountered the need for a complete risk analysis on any project, so far we have limited ourselves to the black box method.

When checking intermediate assemblies, we always discuss with developers what risks they see, and continually keep in mind the question “what could have broken this fix”.

In the final testing before the release, we strive to get rid of all the tests, because it is more accurate. However, time is never enough, and you have to prioritize the functional areas that we want to check. At the same time, we rely not only on business priorities, but also on our understanding of regression risks.

Risk analysis is an activity that enables high-quality regression testing under limited time resources.

This article covers only a small part of the quality assurance process in our company. We will tell about the remaining stages in the following articles.