One day in the life of a local positioning system tester

Hello!

My name is Denis nimpos Koivistunen and I am a tester at RTL-Service.



This is my first topic on Habré and in it I want to share the working life of the testing department and tell you how we have built the process of testing equipment and software.

Go

9:00

The beginning of our working day. First of all, we look at which letters came by e-mail and a list of current affairs for today. Usually, the list of tasks in the testing department is large, each of the tasks has different priorities - first of all we draw our attention to the tasks with the “Urgent” and “Immediately” status, which are set by the developers and project managers.



9:10

Responding to received emails. Some clients of our company work in other time zones, and the sooner they get answers to questions on the operation of our equipment, the better it will be for everyone. For example, there are customers of our company, with whom we have a time difference of five to six hours, as a result of this - when our working day begins, they have it coming to an end. As a result, the problem may remain unresolved until the next business day.

')

9:25

We review and analyze logs from devices left overnight for testing. To do this, we use standard linux commands to filter data (grep, cat, tail, more) and Python and Bash scripts written by our team.





Fig. № 1 - Analysis of logs received



10:30

We are starting to perform the tasks set for today: one of the priority tasks with the “Urgent” status is to test the new firmware version for the access point, in which the bug was previously fixed (arbitrary change of the device operation mode from the gateway to the repeater). We load the new firmware version into the device, check that it has loaded correctly. In the terminal, we configure filtering for a given mode of operation (we use grep), specify the output of the results in the file so that at the end of the experiment we can check it for errors if they occur in the mode. Data collection will take from several hours to several days, so you can proceed to the next task.



11:30

Next we have the queue for urgent tasks - checking the voice session through the repeater chains. As gateways and repeaters, we will use equipment of our production - RealTracTM Access Point Indoor (an access point in an office version, the task of which is to create a radio coverage zone and transfer data from / to mobile devices to provide local positioning and voice transmission (Fig. 2)).





Figure 2 # - RealTracTM Access Point Indoo



To do this, we are flashing our gateway (gateway-access point, SHDD), through which packets from repeaters (repeater-access point, RTD) will be received, all of our repeaters and communicator (wearable radio with voice communication function that provides data from / to fixed radio devices within radio coverage zones (Fig. № 3)) for firmware of one version.



Pic. № 3 - Communicator



We arrange our chain in the sequence of the CTD - RTD - RTD - RTD (Fig. № 4). One of the important conditions for correct data transmission is that the device must be in the line of sight of the next device in the chain.



Fig. № 4 - Equipment layout



12:00

All the devices that will be used in testing, placed, you can proceed to our test (Fig.№5).





Fig. № 5 - Equipment layout



And here, on our way, a problem arose - the test site is small, and the walls are not dense enough and do not shield the signals. As a result, the signal from the repeaters passes through the walls. This creates such an effect that several repeaters will hear the communicator, and this is unacceptable, since we check the correctness of signal transmission through the RTD chain. To solve this problem, we reduce the power of the repeaters and the communicator. To reduce the transmission power of the device, we use the utility developed by our company setparam.



12:15

Power on devices is reduced. Now we need to make sure that several RTDs will not hear the communicator. To check the audibility of access points from a specific place, we will use an analyzer (wearable device developed by our company specifically for measuring radio parameters of various device parameters, including RSSI, distances to all devices that are audible, and so on (Fig. No. 6)).





Fig. № 6 - Analyzer



We approach each of the links in the chain and check with the help of the Analyzer whether the neighboring repeaters are audible and whether other devices in the chain are audible.



12:30

After some permutations of repeaters across the territory, the desired effect was achieved. Now it's up to the small - one employee performs the role of a dispatcher and calls the communicator of another, which is at the end of the chain. If the call to the communicator has passed, then the chain of repeaters is working properly. Now we check the chain by calling the dispatcher from the communicator. When calling from the communicator, the dispatcher is notified of the incoming call.



Let us now try to break the chain by disconnecting one of the repeaters in the chain in order to verify that the transmission of the voice session will not go further along the chain after the repeater is off.



The repeater in the center of the chain is disabled, the employee from the communicator is at the last repeater in the chain, the dispatcher calls the switch - and the call does not go to the communicator. We try to make a call in the opposite direction - we call our dispatcher from the switch - in response, silence, this is exactly what we needed.



The test came to an end, it remains to check the server logs for errors. To do this, go to our directory with logs and execute the command to parse all the files for the presence of the lines “ ERROR ”, “ WARNING ”, “ EXCEPTION ”.



ERROR message detected







Files with logs have a considerable amount that will take some time to analyze them. In addition, we indicate the output of the errors found in a separate log file for further study.



13:03

Lunch time. For lunch, we will run the auto-tests of the web client (about using this method to speed up the automation of web-interface testing by using Python and Selenide, there is an article in our blog) (here - https://habrahabr.ru/company/rtl-service/ blog / 300860 / ), so that the system is not in vain.



2:00 pm

Of all the auto tests running at lunch, only one failed. When there is free time, it will be necessary to see what the problem is. To do this, save the task execution log to a separate file for further study.





Errors when performing autotests



We check logs filtered by errors in our experiment with calls to the communicator. Critical errors not detected. Go ahead.



14:30

A task has been received with the “Immediately” priority for execution, which means that all tasks are currently suspended. The essence of the task is that with the rotation of the substrate mapped in the web client, the load of RAM and the CPU increased to 80-90% and was not released at the end of the rotation. It has been suggested that such a load on the system is caused by the Java process affected by our system. To complete my task, I had to try to reproduce this error.



For this, substrates of several sizes were chosen (1000 x 1000, 3508 x 2480). We deploy our system, install our first substrate of the selected ones (1000 x 1000) and start rotating, watching the load on the CPU and RAM in a separate console increase (we use the top and htop console commands). When a substrate of this size could not load the system to the required more than 80%. Now we take a substrate of a larger size. We repeat our past actions already with a new substrate - we rotate it, at the same time we observe loading parameters in the console - here we already have the result, while rotating the substrate for about a minute we managed to create a load of 84%, this is enough to reproduce the task. We are waiting for 10 - 15 minutes to check whether the load on the processor and RAM will decrease.





Top command output



15:20

Since then, even more time was allocated than the planned 10-15 minutes, the load from the processor went down to 5-7%, and the load on the RAM remained in the region of 80%.



Based on this, it can be assumed that currently the rotation of the large-sized substrate causes a load on the RAM, and in the future it will not be released, which will affect the performance of the system. At the same time, the use of a substrate of such a large size clogs the memory slightly and any actions with it will not be able to incorrectly affect the performance of the system. But for the integrity of the experiment you will need to check it a couple more times.



15:50

Repeated experiment showed a similar result. We make out the results of the identified problem in the task.



16:20

Let's see if the device operation mode that we requested in the morning has changed. To do this, we look that the log filtering the values ​​of the change of device modes is empty. So, we make the assumption that for 5 hours of operation, a regime change was not observed. But for verification, we will leave the test for a day and tomorrow we will check the result using the saved logs.



16:40

We discuss the fixed bugs with colleagues, we make out the data obtained as a result of today's experiments, for each bug we start a ticket in Redmine.



5:30 pm

We check if there are any new letters, what tasks are set for tomorrow, their priorities and deadlines. We also remember that we collect logs from the gateway stitched in the morning. In this connection, before leaving work, you need not to forget that you should not turn off the power from the device, otherwise tomorrow we will be unpleasantly surprised by the lack of logs. It happens, of course, that the power is turned off in the whole building, but we can’t do anything except to repeat it.



18:00

We check that the logs are written to the file, the device is included in the network. You can go home. And tomorrow, again, to meet the new undiscovered bugs, interesting tasks and difficulties that are always encountered on the tester’s hard way.

What is the result

Summing up the past day, we can conclude that in order to correctly conduct tests and obtain reliable results, special attention should be paid to the preliminary planning stage of the test. It is necessary to foresee all the difficulties and possible problems that may be encountered during the test, it will minimize the time to eliminate (solve) problems.



For example, an experiment on voice transmission via a chain of repeaters would take much less time if we had previously (even at the planning stage of the experiment) considered that repeaters would hear each other through walls and would reduce the power of repeaters. And during the experiment would not be distracted by this problem.



Also, before starting equipment placement and testing, it is necessary to make sure several times that the parameters set on the equipment used correspond to the required values. Check that the equipment is determined in the system, light and sound indication, where it is, is in good condition and functions correctly. This will save time on replacing the faulty device and repeating the test (due to an incorrectly set parameter value).



Thank you for your attention, I am ready to answer questions in the comments.