How to assess the quality of user experience in data transfer?
To date, mobile communications cover 60% of the world's population, with 30% of subscribers of broadband mobile access. The volume of traffic in mobile networks is growing steadily. New applications are emerging. Demands on the quality of services provided by mobile operators are becoming increasingly high.
When the main service of mobile operators was “voice”, the technological assessment of the network was based on the radio coverage area, voice quality (MOS) and the percentage of successful connections, and the subscriber's was based on the ability to “dial” in the New Year.
With the growing consumption of data services and the rapid development of online services, network evaluation, from the subscriber’s point of view, has become critical for operators. The quality indicators have changed, and the assessment itself has acquired a new designation - the quality of the user experience.
This term comes from the area of ​​developing user interfaces, where it characterizes the subjective degree of convenience of obtaining information and functionality when using websites, applications, programs.
In terms of evaluating data services, the quality of user experience can be measurable, as it is a combination of various indicators of network services such as the speed of opening web pages, the number and duration of buffering (stalls) when listening to audio or watching video online, time delays in online games , voice and image quality, as well as the number of “breaks” during a Skype conversation.
Figure 1 Event buffering (stall) when watching a video
Figure 2 “Break” during a Skype conversation
')
Measuring the quality of user experience is a significant problem for operators, because, unlike the “voice and sms” era, the operator does not control the final services provided by service providers. Despite the fact that the operator has subscriber traffic, which can be fully analyzed at the network and partially service levels, this only provides indirect data on what subscribers experience when using online services. Increasing the volume of encrypted traffic further complicates the solution of the problem. For example, mail, social networks, search portals are almost completely transferred to work using the SSL (Secure Sockets Layer) protocol, and the distribution of paid content forces service providers to use different DRM (Digital Rights Management) technologies. As a result, only service providers have the ultimate user experience.
The operator’s need to obtain user experience quality data is also due to the fact that the MNP (Mobile Number Portability) law comes into force, which regulates the mechanism for storing the subscriber number when switching from one operator to another, and information about how “happy” subscriber, can be a key indicator in the fight against churn.
ViGO, focused on creating innovative products in the field of optimization, management and tariffication of mobile network traffic, has developed an end-to-end user experience monitoring methodology - UXZoom. The technique allows for a detailed assessment of the user experience and to determine the factors affecting its quality, all along the way the service is delivered - from the service provider's servers to the subscriber.
At the moment, the user experience when watching a video is the most visible indicator of the quality of the network for the subscriber. For this reason, the methodology is aimed primarily at monitoring video delivery.
UXZoom provides comprehensive analysis of the user experience of data services through direct integration with leading service providers and the infrastructure of the mobile operator. The technique is based on the collection and correlation in real time of the following information:
Figure 3 shows the levels of possible analysis and evaluation of user experience using the UXZoom:
Figure 3 Levels of user experience assessment using the UXZoom methodology
The technique allows the operator to look at the quality of the data service from the point of view of subscribers and answer, for example, the following questions:
* - Viewing is considered “successful” under the following conditions:
a) The total length of buffering when viewing does not exceed 1/100 of the length of the video (excluding buffering when opening video)
b) The video was not stopped (closed) by the user during buffering (video drop)
Moreover, UXZoom allows the operator to analyze and identify the causes of poor quality data services based on the definition:
The operator has the opportunity to take into account data on the quality of user experience in the network when contacting customers, radio planning or analyzing the performance of subscriber devices. Below are examples of the analysis of user experience in mobile networks, obtained using the UXZoom methodology for October 2013 in Moscow.
ViGO quarterly reports on the quality of user experience in Russian mobile networks - coming soon!
Anton Prokopenko
Head of Product Development and Integration
Vigo
When the main service of mobile operators was “voice”, the technological assessment of the network was based on the radio coverage area, voice quality (MOS) and the percentage of successful connections, and the subscriber's was based on the ability to “dial” in the New Year.
With the growing consumption of data services and the rapid development of online services, network evaluation, from the subscriber’s point of view, has become critical for operators. The quality indicators have changed, and the assessment itself has acquired a new designation - the quality of the user experience.
This term comes from the area of ​​developing user interfaces, where it characterizes the subjective degree of convenience of obtaining information and functionality when using websites, applications, programs.
In terms of evaluating data services, the quality of user experience can be measurable, as it is a combination of various indicators of network services such as the speed of opening web pages, the number and duration of buffering (stalls) when listening to audio or watching video online, time delays in online games , voice and image quality, as well as the number of “breaks” during a Skype conversation.
')
Measuring the quality of user experience is a significant problem for operators, because, unlike the “voice and sms” era, the operator does not control the final services provided by service providers. Despite the fact that the operator has subscriber traffic, which can be fully analyzed at the network and partially service levels, this only provides indirect data on what subscribers experience when using online services. Increasing the volume of encrypted traffic further complicates the solution of the problem. For example, mail, social networks, search portals are almost completely transferred to work using the SSL (Secure Sockets Layer) protocol, and the distribution of paid content forces service providers to use different DRM (Digital Rights Management) technologies. As a result, only service providers have the ultimate user experience.
The operator’s need to obtain user experience quality data is also due to the fact that the MNP (Mobile Number Portability) law comes into force, which regulates the mechanism for storing the subscriber number when switching from one operator to another, and information about how “happy” subscriber, can be a key indicator in the fight against churn.
ViGO, focused on creating innovative products in the field of optimization, management and tariffication of mobile network traffic, has developed an end-to-end user experience monitoring methodology - UXZoom. The technique allows for a detailed assessment of the user experience and to determine the factors affecting its quality, all along the way the service is delivered - from the service provider's servers to the subscriber.
At the moment, the user experience when watching a video is the most visible indicator of the quality of the network for the subscriber. For this reason, the methodology is aimed primarily at monitoring video delivery.
UXZoom provides comprehensive analysis of the user experience of data services through direct integration with leading service providers and the infrastructure of the mobile operator. The technique is based on the collection and correlation in real time of the following information:
- from online services - delivery quality events (buffering (stalls)), subscriber actions (play, stop, pause, etc.), content metadata (quality, duration, etc.), indications about server service providers' work;
- from the infrastructure of the mobile operator - data on the load of base stations, subscriber radio conditions, their tariffs, location and other network information.
Figure 3 shows the levels of possible analysis and evaluation of user experience using the UXZoom:
Figure 3 Levels of user experience assessment using the UXZoom methodology
The technique allows the operator to look at the quality of the data service from the point of view of subscribers and answer, for example, the following questions:
- How often is video viewing “successful *”?
- How much time do subscribers spend in the state of buffering?
and
How many subscribers refused to further content consumption for this reason? - What is the quality of the content most in demand on the web?
- How many percent of the downloaded content really look?
- In which area of ​​the city and at what time of the day is the highest demand for bulk content?
* - Viewing is considered “successful” under the following conditions:
a) The total length of buffering when viewing does not exceed 1/100 of the length of the video (excluding buffering when opening video)
b) The video was not stopped (closed) by the user during buffering (video drop)
Moreover, UXZoom allows the operator to analyze and identify the causes of poor quality data services based on the definition:
- Radio conditions of subscribers;
- Base station downloads;
- Problems on the side of the service provider;
- Content characteristics;
- Tariffs of subscribers and their restrictions,
The operator has the opportunity to take into account data on the quality of user experience in the network when contacting customers, radio planning or analyzing the performance of subscriber devices. Below are examples of the analysis of user experience in mobile networks, obtained using the UXZoom methodology for October 2013 in Moscow.
- When analyzing the quality of user experience at the highest load hours (CNN) from 6 pm to 9 pm, subscribers of the Yota 4G (Scartel) operator had better quality than subscribers of other operators — about 64% of all views of online video content were “successful *”, MTS subscribers have 43% of successful views. The number of analyzed views amounted to 741345.
- The average bitrate of the "well viewed" video on the MTS network is 1100Kbps (content quality - SQ (Standard Quality)), on the Yota 4G network - 1450Kbps (content quality - SQ).
- The maximum bitrate of successful views on the Yota 4G network is 5000Kbps (content quality is HD).
- Up to 15% of “junk” video traffic is generated on a 3–4 minute video with Progressive Download distribution.
The volume of “junk” traffic is calculated by adding the sizes of all video buffers downloaded but not viewed in the player.
ViGO quarterly reports on the quality of user experience in Russian mobile networks - coming soon!
Anton Prokopenko
Head of Product Development and Integration
Vigo
Source: https://habr.com/ru/post/202354/
All Articles