How to test online store communication channels before sales
Not so long ago, one of the largest online stores conducted an advertising campaign in honor of the birthday of the company. To receive orders, they have two virtual multichannel phone numbers 8-800 from different telecom operators, and for their processing their own CRM system was written and the work of the call-center was organized at an outsource. The initial forecast of customer visits to the site raised doubts that all systems would cope. In order not to lose the lead, it was decided to test the telephone channels and the CRM system.
IP-telephony when it reaches a certain load begins to distort the voice significantly. And if CRM falls, people just won’t get their orders. Assessing the risk of failure of the action, the customer turned to our company.
Our task was to create a distributed telephone load in Russia: 70% from Moscow, the rest from regions. It was necessary to send a voice message to the incoming stream, and on the side of the call-center it was necessary for the employee to pick up the phone. The result was two streams (outbound and inbound), which could be analyzed for voice quality and compared with reference recordings.
Since the clients of the online store are located in different cities of the Russian Federation, to simulate the load, numbers were allocated for testing. Two different telecommunication operators were used as communication service providers for call centers. The load was generated with an increase of up to 5 CPS (Call Per Seconds) and with a duration of one generated call for at least 180 seconds (3 minutes). Limit the simultaneous number of calls - up to 350.
')
To identify errors in telecommunications equipment and its fine-tuning, we use our own MTT System for generating and collecting statistics . This is a set of software and hardware and methods for generating high-load tests via SIP telephony and subsequent analysis of the results.
Testing was conducted in three stages:
With the help of the load generation system and statistics collection, an increasing flow of calls to our customer's call center number was created in accordance with the testing stage. It worked like this:
To analyze the quality of the transmitted voice from the MTT load generation system in the direction of the customer's call center, it was necessary to provide an answer to the call and play the greeting. To do this, the client’s call center did the following:
As a result of the collection of information, the MTT analytical system calculated the parameters of PDD, PESQ MOS and prepared summary reports. They were built by combining the results into pools of a set of values ​​based on the connection establishment time between the nodes of the MTT network and the customer's call center. The aggregate indicators of quality of service were calculated from the sets of pool values.
Reference: PDD (Post Dialing Delay) - the time between the beginning of a call and the moment when the phone starts to ring on the called party. In other words, the PDD is calculated as the time between the INVITE message sent by the caller and the RINGING message from the receiving party.
PESQ (Perceptual Evaluation of Speech Quality) is a family of standards for assessing voice quality from end to end. Standardized with ITU-T P.862 recommendations. Currently, it is the standard for de facto assessing voice quality. Used by most manufacturers of telecommunications equipment and software developers to make calls.
For testing by CRM, experts wrote a scenario of high load imitation conditions. Within its framework, typical actions of the call-center operator were described: a customer card was raised, a product was selected, customer data was filled in, etc. If the cargo is bulky, the delivery was divided into several parts.
For testing used apache JMeter. CRM-system loaded 500 simultaneous ordering tasks.
Setup Tip: JMeter out of the box uses only 512 megabytes, which is not enough for a load of even 100 threads. This is easily corrected by setting the java machine. In the running Jmeter file, you need to increase the heap size: set HEAP = -Xms512m -Xmx4096m
Input data was taken from a CSV file. We implemented several order scenarios with different quantities of goods and ways of processing. For this, the data lines contained different parameters. Each line is one operator of the customer’s system and the goods of the order that the operator had to arrange.
Each item in the JMeter file is a request from the online store website. Based on the user scenario, we analyzed the actually sent requests and formed a set from them. This set was the possible behavior of the call center operator, the load on which was simulated.
Here are some examples of query generation:
index - page load . Request page after authorization. Here we obtained the data necessary for the further work of the script:
search_by_whole_name - product search function . Here we used the data obtained from the parsing page:
Using if, the test execution script was branched:
Depending on the input data, certain actions were performed.
As a result of testing, JMeter generated reports on each call to CRM.
During the development of the test, we noticed that different operators repeated the value of the thread name, which made it impossible to identify a specific order. This was due to the fact that the thread name was formed from the total number of simultaneous threads specified in the thread group.
We found a simple solution to this problem: we added thread_id in jemeter.properties, which made it possible to filter queries by a specific operator.
Report example:
A telephony test was originally scheduled, along with call-center operators. However, organizationally it turned out to be very difficult to do, so that two parallel tests were simultaneously conducted under conditions as close as possible to the real ones. During testing, the time to call center call equipment response was measured. This parameter is of great importance when conducting load testing. With a large load, call processing delays on the call center equipment greatly increase. The called customer may not even wait for the "beeps". Response codes were also recorded and analyzed. This could be the fact that the call center equipment responded, the call center was canceled, or error codes. According to the analysis of response time and codes, they decided to strengthen telephone channels and modernize equipment. The CRM test showed that the software withstood the load, but, of course, programmers made some changes to the settings to increase stability.
IP-telephony when it reaches a certain load begins to distort the voice significantly. And if CRM falls, people just won’t get their orders. Assessing the risk of failure of the action, the customer turned to our company.
Telephony Testing
Our task was to create a distributed telephone load in Russia: 70% from Moscow, the rest from regions. It was necessary to send a voice message to the incoming stream, and on the side of the call-center it was necessary for the employee to pick up the phone. The result was two streams (outbound and inbound), which could be analyzed for voice quality and compared with reference recordings.
Since the clients of the online store are located in different cities of the Russian Federation, to simulate the load, numbers were allocated for testing. Two different telecommunication operators were used as communication service providers for call centers. The load was generated with an increase of up to 5 CPS (Call Per Seconds) and with a duration of one generated call for at least 180 seconds (3 minutes). Limit the simultaneous number of calls - up to 350.
')
To identify errors in telecommunications equipment and its fine-tuning, we use our own MTT System for generating and collecting statistics . This is a set of software and hardware and methods for generating high-load tests via SIP telephony and subsequent analysis of the results.
Testing was conducted in three stages:
- The first two channels of operators were tested.
- At the third stage, the logic of the IVR call center (CC) was tested by generating a load through the channels of one of the operators.
Testing method
With the help of the load generation system and statistics collection, an increasing flow of calls to our customer's call center number was created in accordance with the testing stage. It worked like this:
- The system alternately substituted a number from the list of Caller ID numbers as a call source.
- Then a call was formed through the communication node corresponding to the substituted caller ID.
- A welcome template file with the text pronounced by the announcer (in the WAV format, 8000Hz, mono) broadcast from the customer’s CC was recorded within 20 seconds after receiving the connection information.
- After that, after a pause of 1 second in the direction of the customer's call center, a template file was played with text that was spoken by the speaker (in the WAV format, 8000Hz, mono).
- As a result of the CDR (Call Detail Records) call, the quality parameters of the connections were recorded and a link to the recording file was created.
- According to the results of the load tests, the analysis of statistical information was made.
To analyze the quality of the transmitted voice from the MTT load generation system in the direction of the customer's call center, it was necessary to provide an answer to the call and play the greeting. To do this, the client’s call center did the following:
- Provided receiving incoming calls via communication channels to the leased phone numbers.
- When a call arrived, I lost a template file with text that was spoken to the speaker (in the WAV format, 8000Hz, mono).
- Made a recording of the voice of the announcer (in the WAV format, 8000Hz, mono) transmitted from MTT in the direction of the CC with the operator of the CC.
- I recorded the results of the call to the CDR (Call Detail Records) with information that would be sufficient to uniquely identify the call generated by MTT and the customer’s call center, as well as a link to the file with the record.
- According to the results of load tests, he provided data for analysis to the system for generating and collecting MTT statistics.
As a result of the collection of information, the MTT analytical system calculated the parameters of PDD, PESQ MOS and prepared summary reports. They were built by combining the results into pools of a set of values ​​based on the connection establishment time between the nodes of the MTT network and the customer's call center. The aggregate indicators of quality of service were calculated from the sets of pool values.
Reference: PDD (Post Dialing Delay) - the time between the beginning of a call and the moment when the phone starts to ring on the called party. In other words, the PDD is calculated as the time between the INVITE message sent by the caller and the RINGING message from the receiving party.
PESQ (Perceptual Evaluation of Speech Quality) is a family of standards for assessing voice quality from end to end. Standardized with ITU-T P.862 recommendations. Currently, it is the standard for de facto assessing voice quality. Used by most manufacturers of telecommunications equipment and software developers to make calls.
CRM Testing
For testing by CRM, experts wrote a scenario of high load imitation conditions. Within its framework, typical actions of the call-center operator were described: a customer card was raised, a product was selected, customer data was filled in, etc. If the cargo is bulky, the delivery was divided into several parts.
For testing used apache JMeter. CRM-system loaded 500 simultaneous ordering tasks.
Setup Tip: JMeter out of the box uses only 512 megabytes, which is not enough for a load of even 100 threads. This is easily corrected by setting the java machine. In the running Jmeter file, you need to increase the heap size: set HEAP = -Xms512m -Xmx4096m
Input data was taken from a CSV file. We implemented several order scenarios with different quantities of goods and ways of processing. For this, the data lines contained different parameters. Each line is one operator of the customer’s system and the goods of the order that the operator had to arrange.
test_crm_user_376;1a6cddfc0c077fe64fbe94983b2e5bde;111;;call;127322;103413001;1;null;1;116131;1;;;;false
test_crm_user_126;8ae9ssc0d73b5305b58714c0c33a3c27;111;;call;127322;111634;1;110951005;1;111117;1;;;;trueQuery examples
Each item in the JMeter file is a request from the online store website. Based on the user scenario, we analyzed the actually sent requests and formed a set from them. This set was the possible behavior of the call center operator, the load on which was simulated.
Here are some examples of query generation:
login — . CSV :
/crm/index.php?login=${login}&pass=${pass}&ANI=${ani}&SLNUM=${slnum}&module=${module}
index - page load . Request page after authorization. Here we obtained the data necessary for the further work of the script:
/crm/index.php?SLNUM=${slnum}&ANI=111436609800&module=call
.*ContactID'\s*,\s*'(\d+)'\);.*
.+CallBegin',\s*'(\d+)'.*
.+'operator_id'.+'(\d+)\|.+
search_by_whole_name - product search function . Here we used the data obtained from the parsing page:
/crm/modules/ajax/get_goods.php?&CONTACT_ID=${contact_id_g1}&IBLOCK_ID=7&BAN_SYM=%2C%3B&REP_SYM=&MODE=SEARCH&p=0&search=${commodity1}&type=&OFFER=1&ORDER_PROP_13=TV&ORDER_PROP_32=${slnum}&coupon=&user_id=0&LID=s1&order_id=&bonus_rubls=&bonus_rubls_status=active&_=1470218107193Using if, the test execution script was branched:
If commodity1 and associated commodity (orderline/upsell/list-top)
${__jexl(${commodity1} != null and ${commodity2} == null and ${commodity3} != null)}
Depending on the input data, certain actions were performed.
Reports
As a result of testing, JMeter generated reports on each call to CRM.
During the development of the test, we noticed that different operators repeated the value of the thread name, which made it impossible to identify a specific order. This was due to the fact that the thread name was formed from the total number of simultaneous threads specified in the thread group.
We found a simple solution to this problem: we added thread_id in jemeter.properties, which made it possible to filter queries by a specific operator.
sample_variables=splitting,order_id_g1,thread_idReport example:
Total
A telephony test was originally scheduled, along with call-center operators. However, organizationally it turned out to be very difficult to do, so that two parallel tests were simultaneously conducted under conditions as close as possible to the real ones. During testing, the time to call center call equipment response was measured. This parameter is of great importance when conducting load testing. With a large load, call processing delays on the call center equipment greatly increase. The called customer may not even wait for the "beeps". Response codes were also recorded and analyzed. This could be the fact that the call center equipment responded, the call center was canceled, or error codes. According to the analysis of response time and codes, they decided to strengthen telephone channels and modernize equipment. The CRM test showed that the software withstood the load, but, of course, programmers made some changes to the settings to increase stability.
Source: https://habr.com/ru/post/321142/
All Articles