How did radio frequency service measure communication quality?
In the middle of July, I happened to be among several journalists and TV bloggers in the RFTC of the Central Federal District to demonstrate the quality of communication. The organizer was either the radio frequency service itself, complaining about the low light of its important work in the media, or Roskomnadzor.
It turned out not without flaws and incidents, but interesting. And most importantly - inspired some optimism.
')
But I will start in order. First, the RFC CFA in the Central Federal District (namely, it is so funny after the reorganization is called the office where they were invited) is located not at all where Yandex leads us and some reference books are not on Dostoevskaya and not in Maryina Roscha, but in Butovo in the middle of the forest and next to the huge construction site "Etalon City".
The radio frequency service itself is located in a new three-storey building, lined with light tiles.
The conference hall gathered about 20 people, including the organizers of the event.
results
In a speech devoted to indicators of the quality of communication in the country, Dmitry Mitrofanov, the deputy leader of the Center, made a rather colorful and clear presentation. He spoke in general about the measurement methodology, approved by the order of the Ministry of Communications and the organization of work, which Roskomnadzor assumed.
Since the beginning of 2015, the RFTC of the Central Federal District regularly conducts measurements of the quality of communication in Russian cities. After that, the log files with the results are sent to the RKN and are laid out on the site of communication quality .
In general, measurements are carried out in four positions:
• Voice quality. Includes the proportion of unsuccessful attempts to establish a connection, the proportion of dangling calls, speech intelligibility;
• SMS delivery (the share of messages that did not reach the addressee, as well as the average delivery time);
• Data transfer, which contains the following parameters: Share of unsuccessful TCP / IP connection to the server (HTTP IP-Service Access Failure Ratio); Share of unsuccessful HTTP sessions (HTTP Session Failure Ratio); The average value of the data transfer rate to the subscriber (HTTP DL Mean User Data Rate) in KB / s; The duration of a successful session (HTTP Session Time) in ms.
• Total scan coverage, presence and signal level (measured in dBm).
Today, for 1.5 years, measurements were taken in 61 cities, and in Moscow and St. Petersburg - twice. A good result, given that the measurement time in one city is 2 months, during which about 7,300 test calls are made, not counting other operations.
Mr. Mitrofanov showed results in voice, where the average intelligibility of speech in the country was 3.9 (the coefficient calculated according to the method has the lowest indicator of 2.6 and the highest - 5) ... And the clear outsider was Crimea with a value of 3.35 (and this is above the low legibility threshold). However, according to Mitrofanov, the situation on the peninsula is changing rapidly and the next measurement should demonstrate a sharp improvement in the quality of communication in the region.
By sms, the situation is much better. Only every hundredth message (1.1%) does not reach the addressee.
According to the results of the measurements already made, the figures for the average data transfer rate of subscribers in the country were announced. It amounted to 5413 Kb / s. The speed is good, if you do not delve into the method of calculating the average. Indeed, in a small but densely populated Moscow, the speed is much higher than in the huge and half-empty Krasnoyarsk Territory. Whether this was calculated by the arithmetic average of the number of measured regions or by the population formally registered in it remained unknown.
In addition, the radio frequency service monitors the quality of communication at major events (SPIEF, 2016 IIHF World Championship, MAKS, Eastern Economic Forum, etc.). It is not surprising that during the period of such events in the territory of their conduct, the quality of communication is much better than the average for the city. Telecom operators are installing additional base stations, including mobile, working to eliminate interference, as confirmed by the representative of MegaFon who was present at the meeting.
Technique
The measurement method was demonstrated by a young man from the Central Federal District in the Urals Federal District who introduced himself as Eugene.
Control calls are made using the ASCOM TEMS Automatic complex to a fixed telephone number (call-generator in the territory of the RFC), which is also connected to the digital E1 lines of the fixed communication network (PSTN). At the same time, the complex includes 4 identical devices for calls, equipped with sims of four federal operators. Thus, calls are made synchronously at the same time, from the same place, which excludes the privileged conditions for any operator. I will add that speech quality is measured using the POLQA (Perceptual Objective Listening Quality Assessment) technology, approved by the International Telecommunication Union.
In the same way, incoming calls and incoming / outgoing messages are tested (with a cycle at 2:30). It's funny that the original test sims were blocked by operators, identifying them as spam. Currently, the problem, according to the organizers of the event, has been resolved.
Each voice call is evaluated for 3 minutes. This, as well as the restrictions of the Labor Code of the Russian Federation to work within the 8-hour working day, determine the slow speed of measurements in the city. As for the wearable complex, here the work is hampered by the impossibility of simultaneously running two scripts (for example, voice and sms).
Much more interesting with the measurements of the quality of data transmission, I mean the Internet connection. The procedure is performed for three standards: GSM (2G), UMTS (3G) and LTE (4G). And if it is the same for the first two, it is different for the most advanced generation of networks. For example, the file size for HTTP downloading for GSM and UMTS is 3 MB, and for LTE 100 MB.
Virtually all tests are performed using the TEMS environment:
You can not keep silent about the error of this technique, which immediately catches the eye. Tele2, having no frequencies for GSM networks, finds itself in a deliberately advantageous position. Where the Big Three is slipping into 2G and, accordingly, shows a weak result, Tele2 has no Internet at all. RFC staff retorted that such failures are also taken into account in the overall result. However, it is obvious that in the final schedule Tele2's leading positions are due precisely to this circumstance.
The journalists and representatives of telecom operators who were present at the meeting had questions to the leadership of Tele2 regarding data transfer. The operator, having no frequencies for GSM networks, is in a deliberately advantageous position. Where the Big Three is slipping into 2G and, accordingly, shows a weak result, Tele2 has no Internet at all. RFC staff retorted that the file in 2G and 3G weighs 3 MB, and for LTE 100 MB and the approximated result, this remark has no effect. Tele2 benefits from less network utilization than other operators. By the way, talking with guests in an informal setting after the meeting, I was convinced that the arguments of the staff of the RCSC did not convince the skeptics.
Equipment
After all the presentations and discussions in a beautiful but stuffy conference room, we were finally taken outside to demonstrate the equipment itself.
The carriage kit is a regular Renault Duster SUV with small rooftop antennas and equipment mounted in the trunk. According to regulations, the feeder is located at a height of no more than 1.5 meters from the ground.
The driver of the car, although trained in handling the complex in the event of an emergency, does not interfere with the work of the meters throughout the entire route. According to the standards, the car should move at a speed of no more than 40 km / h, i.e. The circle around South Butovo took about 1 hour in our presence.
At present, 10 units of similar complexes have been purchased - two each in the Central and Northwestern Federal Districts and one each for the rest. And if at first, as acknowledged by the employees of the RFC, the complexes were mounted into cars independently, now the car dealers supply the cars already prepared in accordance with the terms of reference.
The wearable complex is an ordinary backpack, inside of which there is a panel with four smartphones attached to it with the help of velcro, a tablet with the help of which a specialist via bluetooth controls the measurement and a TSMW scanner for analyzing the radio signal in the network. Outside, only two antennas of the scanner work out the manufacturability of the backpack.
An employee of the RFTC of the Central Federal District with us walked for about an hour in the nearby Viva shopping center, taking measurements.
Interestingly, if the coordinates for a portable complex are determined using GPS, then a specialist with a backpack manually links his location for measurement at a specific point.
For synchronous and objective measurement, RFC specialists ensure the possibility of testing at least 4 channels for portable equipment and at least 12 channels (8 for voice testing and 4 for testing data transmission) for portable equipment.
After the measurements are made, the data is sent to the RFC servers of the Federal District:
Here is the storage and post-processing of log files.
After unloading all the results, sometimes weighing very, very much (in our case, hourly measurements were unloaded for about an hour), the following documents are compiled:
• measurement protocol;
• Measurement report. Log files, which are then converted to a public format (txt or csv).
After subtracting the redundant data and building the tiles for visual display of the results, the log files are sent to the RFC Competence Center located in the Siberian Federal District, where the obtained data is checked for relevance and adequacy, and the measurements are compared with the methodology.
After this, the product of the work of specialists of the radio frequency service is laid out on the site of communication quality .
Finally
As a funny episode, I note that in the territory of the RFTC of the Central Federal District, both in the building and on the street, the Internet and the quality of data transmission leaves much to be desired. The organizers explained this to the huge Etalon City complex being built nearby and to the large distance of the base station on the other side. It seems to be true, but as they say, the shoemaker walks without shoes ...
Instead of an afterword
The cordial leaders of the RFTC of the Central Federal District knew no bounds and for sweetness we were shown mobile radio monitoring stations, which, in general, were not related to the subject of the event (measurement of the quality of communication, if anyone forgot).
Two German-made cars are equipped with high-tech equipment: one is domestic, another is overseas, produced by Rohde & Schwarz.
To be honest, this equipment is impressive where there are more complexes for measuring the quality of communication.
Quite the end
Recognizing some of the existing problems in the implementation of measurements, I want to say that the overall impression of what he saw is positive. Employees of the RFTC of the Central Federal District turned out to be young competent specialists (among whom the girl Irina stood out), and not technicians of retirement age, who were sitting there from the pre-Internet era.
It seems that there is no reason not to trust such measurements, although communication operators with not the best results, of course, will find something to argue ... Business, gentlemen. And for us, ordinary consumers, the site of the quality of communications. The Russian Federation will help in planning its access to the network and choosing a telecom operator.
It turned out not without flaws and incidents, but interesting. And most importantly - inspired some optimism.
')
But I will start in order. First, the RFC CFA in the Central Federal District (namely, it is so funny after the reorganization is called the office where they were invited) is located not at all where Yandex leads us and some reference books are not on Dostoevskaya and not in Maryina Roscha, but in Butovo in the middle of the forest and next to the huge construction site "Etalon City".
The radio frequency service itself is located in a new three-storey building, lined with light tiles.
The conference hall gathered about 20 people, including the organizers of the event.
results
In a speech devoted to indicators of the quality of communication in the country, Dmitry Mitrofanov, the deputy leader of the Center, made a rather colorful and clear presentation. He spoke in general about the measurement methodology, approved by the order of the Ministry of Communications and the organization of work, which Roskomnadzor assumed.
Since the beginning of 2015, the RFTC of the Central Federal District regularly conducts measurements of the quality of communication in Russian cities. After that, the log files with the results are sent to the RKN and are laid out on the site of communication quality .
In general, measurements are carried out in four positions:
• Voice quality. Includes the proportion of unsuccessful attempts to establish a connection, the proportion of dangling calls, speech intelligibility;
• SMS delivery (the share of messages that did not reach the addressee, as well as the average delivery time);
• Data transfer, which contains the following parameters: Share of unsuccessful TCP / IP connection to the server (HTTP IP-Service Access Failure Ratio); Share of unsuccessful HTTP sessions (HTTP Session Failure Ratio); The average value of the data transfer rate to the subscriber (HTTP DL Mean User Data Rate) in KB / s; The duration of a successful session (HTTP Session Time) in ms.
• Total scan coverage, presence and signal level (measured in dBm).
Today, for 1.5 years, measurements were taken in 61 cities, and in Moscow and St. Petersburg - twice. A good result, given that the measurement time in one city is 2 months, during which about 7,300 test calls are made, not counting other operations.
Mr. Mitrofanov showed results in voice, where the average intelligibility of speech in the country was 3.9 (the coefficient calculated according to the method has the lowest indicator of 2.6 and the highest - 5) ... And the clear outsider was Crimea with a value of 3.35 (and this is above the low legibility threshold). However, according to Mitrofanov, the situation on the peninsula is changing rapidly and the next measurement should demonstrate a sharp improvement in the quality of communication in the region.
By sms, the situation is much better. Only every hundredth message (1.1%) does not reach the addressee.
According to the results of the measurements already made, the figures for the average data transfer rate of subscribers in the country were announced. It amounted to 5413 Kb / s. The speed is good, if you do not delve into the method of calculating the average. Indeed, in a small but densely populated Moscow, the speed is much higher than in the huge and half-empty Krasnoyarsk Territory. Whether this was calculated by the arithmetic average of the number of measured regions or by the population formally registered in it remained unknown.
In addition, the radio frequency service monitors the quality of communication at major events (SPIEF, 2016 IIHF World Championship, MAKS, Eastern Economic Forum, etc.). It is not surprising that during the period of such events in the territory of their conduct, the quality of communication is much better than the average for the city. Telecom operators are installing additional base stations, including mobile, working to eliminate interference, as confirmed by the representative of MegaFon who was present at the meeting.
Technique
The measurement method was demonstrated by a young man from the Central Federal District in the Urals Federal District who introduced himself as Eugene.
Control calls are made using the ASCOM TEMS Automatic complex to a fixed telephone number (call-generator in the territory of the RFC), which is also connected to the digital E1 lines of the fixed communication network (PSTN). At the same time, the complex includes 4 identical devices for calls, equipped with sims of four federal operators. Thus, calls are made synchronously at the same time, from the same place, which excludes the privileged conditions for any operator. I will add that speech quality is measured using the POLQA (Perceptual Objective Listening Quality Assessment) technology, approved by the International Telecommunication Union.
In the same way, incoming calls and incoming / outgoing messages are tested (with a cycle at 2:30). It's funny that the original test sims were blocked by operators, identifying them as spam. Currently, the problem, according to the organizers of the event, has been resolved.
Each voice call is evaluated for 3 minutes. This, as well as the restrictions of the Labor Code of the Russian Federation to work within the 8-hour working day, determine the slow speed of measurements in the city. As for the wearable complex, here the work is hampered by the impossibility of simultaneously running two scripts (for example, voice and sms).
Much more interesting with the measurements of the quality of data transmission, I mean the Internet connection. The procedure is performed for three standards: GSM (2G), UMTS (3G) and LTE (4G). And if it is the same for the first two, it is different for the most advanced generation of networks. For example, the file size for HTTP downloading for GSM and UMTS is 3 MB, and for LTE 100 MB.
Virtually all tests are performed using the TEMS environment:
You can not keep silent about the error of this technique, which immediately catches the eye. Tele2, having no frequencies for GSM networks, finds itself in a deliberately advantageous position. Where the Big Three is slipping into 2G and, accordingly, shows a weak result, Tele2 has no Internet at all. RFC staff retorted that such failures are also taken into account in the overall result. However, it is obvious that in the final schedule Tele2's leading positions are due precisely to this circumstance.
The journalists and representatives of telecom operators who were present at the meeting had questions to the leadership of Tele2 regarding data transfer. The operator, having no frequencies for GSM networks, is in a deliberately advantageous position. Where the Big Three is slipping into 2G and, accordingly, shows a weak result, Tele2 has no Internet at all. RFC staff retorted that the file in 2G and 3G weighs 3 MB, and for LTE 100 MB and the approximated result, this remark has no effect. Tele2 benefits from less network utilization than other operators. By the way, talking with guests in an informal setting after the meeting, I was convinced that the arguments of the staff of the RCSC did not convince the skeptics.
Equipment
After all the presentations and discussions in a beautiful but stuffy conference room, we were finally taken outside to demonstrate the equipment itself.
The carriage kit is a regular Renault Duster SUV with small rooftop antennas and equipment mounted in the trunk. According to regulations, the feeder is located at a height of no more than 1.5 meters from the ground.
The driver of the car, although trained in handling the complex in the event of an emergency, does not interfere with the work of the meters throughout the entire route. According to the standards, the car should move at a speed of no more than 40 km / h, i.e. The circle around South Butovo took about 1 hour in our presence.
At present, 10 units of similar complexes have been purchased - two each in the Central and Northwestern Federal Districts and one each for the rest. And if at first, as acknowledged by the employees of the RFC, the complexes were mounted into cars independently, now the car dealers supply the cars already prepared in accordance with the terms of reference.
The wearable complex is an ordinary backpack, inside of which there is a panel with four smartphones attached to it with the help of velcro, a tablet with the help of which a specialist via bluetooth controls the measurement and a TSMW scanner for analyzing the radio signal in the network. Outside, only two antennas of the scanner work out the manufacturability of the backpack.
An employee of the RFTC of the Central Federal District with us walked for about an hour in the nearby Viva shopping center, taking measurements.
Interestingly, if the coordinates for a portable complex are determined using GPS, then a specialist with a backpack manually links his location for measurement at a specific point.
For synchronous and objective measurement, RFC specialists ensure the possibility of testing at least 4 channels for portable equipment and at least 12 channels (8 for voice testing and 4 for testing data transmission) for portable equipment.
After the measurements are made, the data is sent to the RFC servers of the Federal District:
Here is the storage and post-processing of log files.
After unloading all the results, sometimes weighing very, very much (in our case, hourly measurements were unloaded for about an hour), the following documents are compiled:
• measurement protocol;
• Measurement report. Log files, which are then converted to a public format (txt or csv).
After subtracting the redundant data and building the tiles for visual display of the results, the log files are sent to the RFC Competence Center located in the Siberian Federal District, where the obtained data is checked for relevance and adequacy, and the measurements are compared with the methodology.
After this, the product of the work of specialists of the radio frequency service is laid out on the site of communication quality .
Finally
As a funny episode, I note that in the territory of the RFTC of the Central Federal District, both in the building and on the street, the Internet and the quality of data transmission leaves much to be desired. The organizers explained this to the huge Etalon City complex being built nearby and to the large distance of the base station on the other side. It seems to be true, but as they say, the shoemaker walks without shoes ...
Instead of an afterword
The cordial leaders of the RFTC of the Central Federal District knew no bounds and for sweetness we were shown mobile radio monitoring stations, which, in general, were not related to the subject of the event (measurement of the quality of communication, if anyone forgot).
Two German-made cars are equipped with high-tech equipment: one is domestic, another is overseas, produced by Rohde & Schwarz.
To be honest, this equipment is impressive where there are more complexes for measuring the quality of communication.
Quite the end
Recognizing some of the existing problems in the implementation of measurements, I want to say that the overall impression of what he saw is positive. Employees of the RFTC of the Central Federal District turned out to be young competent specialists (among whom the girl Irina stood out), and not technicians of retirement age, who were sitting there from the pre-Internet era.
It seems that there is no reason not to trust such measurements, although communication operators with not the best results, of course, will find something to argue ... Business, gentlemen. And for us, ordinary consumers, the site of the quality of communications. The Russian Federation will help in planning its access to the network and choosing a telecom operator.
Source: https://habr.com/ru/post/395991/
All Articles