Our rake when starting Calltouch Predict: 365 days of speech recognition and machine learning
For a long time, the “calling” market has switched from a “pay for a call” model to a “pay for a call that leads to a sale” model. In the automotive business, these are calls to the sales department, in real estate, calls that bring new customers, in medicine, a primary record of patients, and so on.
The importance of determining target calls lies in the fact that in such areas up to 70% of incoming calls are not interesting for the marketer to set up advertising: calls from current or repeat customers, employees, various spam, etc. Based on the total number of calls, the advertiser will consider it an effective source , which in fact does not lead new customers. In order to optimize the cost of advertising, calls must be divided and laid out on those that lead to the sale, and those that do not. As a result, the company has a choice: to put this manual labor of call tagging on the shoulders of operators or vendors, or to use neural networks and other machine learning methods.
In the middle of 2016, we were the first on the market to launch a technology to automatically determine the quality and outcome of a telephone call. The Calltouch Predict was based on the SpeechKit speech recognition system from Yandex and the company's own algorithms. We have heard quite a bit of skepticism about the fact that this will not work.
')
In general, a year later, we are ready to tell without what call tagging really will not work effectively:
It seems to be an obvious statement that the automatic determination of the quality and result of a phone call is needed precisely in order to level the human factor when the operator works with CRM. If the accuracy is, for example, 80%, then such a system may not be implemented, but be content with the work of the contact center employees. 2 errors for 10 calls is the usual manual error. The technology should provide higher accuracy of call determination. What is needed for this? Everything is simple: the more data for training the model, the higher the accuracy of speech recognition and call typing. Transcribers need to spend time and manually decrypt at least a million calls. This will help to accumulate a large database and improve the language model, thereby reducing the percentage of the word error rate (speech recognition errors) to a minimum. Our customers are satisfied with the bottom bracket of 90%. Therefore, now the system will not even start automatically tagging if the accuracy is lower than this indicator.
A product must have a system that leads all word forms to a certain type (declensions, conjugations, and so on) in order to search for uniform vocabulary structures. A kind of normalizer language. This greatly increases the accuracy of call typing.
In itself, automatic recognition and tagging simply provides a set of data about calls that should be used to optimize advertising campaigns. It is logical that when the system automatically arranges tags for calls, the contextual ad optimizer “pulls up” calls according to tags, and all this happens within the same system. And not the employee pins it with his hands, using different services. What is it for: the value of the target call for automatic bid management is higher than the value of the sales data: it is more important to have a large array of targeted calls that could lead to the sale than a small sample of actually completed transactions. Since often a quality lead can be inefficiently processed in the sales department.
An example of testing (30 days) of the Predict system in tandem with the contextual advertising optimizer from the developer X:
Number of unique calls (calls from new numbers) - 510
Number of uniquely-targeted calls (call from a new number with a specific duration) - 410
The number of calls with the automatic tag "target" - 360
Therefore, the proportion of truly targeted calls among unique ones is 71%, the proportion of truly targeted among uniquely targeted 87%. From this it follows that 29% of calls are not targeted among unique ones, and 13% are not targeted among uniquely targeted ones. With the help of Predict, we were able to determine the proportion of true target calls, and then using the optimizer, we lowered the CPA by 48% and increased the CR by 115%.
It is impossible to ignore the fact that in some industries a high fraud index is not always: a call that allegedly leads to a sale (housing display, recording for a test drive) leads to a deal, because it can be a bargain from an unfair advertising platform. At the time of the phone call, even if we automatically mark it with a “deal”, the client cannot be sure that this is not a fraud call by the script. The market segments with a high percentage of fraud include, for example, real estate sales (9%). For such companies, tagging works most effectively in the chain: “an incoming call is a system for determining the quality of a call — an antifraud — a tagging — an optimizer.”
Customers of the product were required to control the quality of employees who communicate with customers. In order to eliminate manual labor, the automatic tagging system must highlight these calls in a separate category. In addition to the commonly used negatively colored expressions (machine learning elements are used for this, such as hidden semantic analysis, support vector machine and semantic orientation in this area), you can expand the vocabulary of lexical units that carry risks for customers. For example: the court, FAS, Rospotrebnadzor and so on.
It is quite difficult to be the first to introduce new solutions: how much rake will be - all yours. It took us a year to collect the first five. I think that many more discoveries await us ahead :-)
The importance of determining target calls lies in the fact that in such areas up to 70% of incoming calls are not interesting for the marketer to set up advertising: calls from current or repeat customers, employees, various spam, etc. Based on the total number of calls, the advertiser will consider it an effective source , which in fact does not lead new customers. In order to optimize the cost of advertising, calls must be divided and laid out on those that lead to the sale, and those that do not. As a result, the company has a choice: to put this manual labor of call tagging on the shoulders of operators or vendors, or to use neural networks and other machine learning methods.
In the middle of 2016, we were the first on the market to launch a technology to automatically determine the quality and outcome of a telephone call. The Calltouch Predict was based on the SpeechKit speech recognition system from Yandex and the company's own algorithms. We have heard quite a bit of skepticism about the fact that this will not work.
')
In general, a year later, we are ready to tell without what call tagging really will not work effectively:
Recognition accuracy not lower than 90%
It seems to be an obvious statement that the automatic determination of the quality and result of a phone call is needed precisely in order to level the human factor when the operator works with CRM. If the accuracy is, for example, 80%, then such a system may not be implemented, but be content with the work of the contact center employees. 2 errors for 10 calls is the usual manual error. The technology should provide higher accuracy of call determination. What is needed for this? Everything is simple: the more data for training the model, the higher the accuracy of speech recognition and call typing. Transcribers need to spend time and manually decrypt at least a million calls. This will help to accumulate a large database and improve the language model, thereby reducing the percentage of the word error rate (speech recognition errors) to a minimum. Our customers are satisfied with the bottom bracket of 90%. Therefore, now the system will not even start automatically tagging if the accuracy is lower than this indicator.
Normalizer
A product must have a system that leads all word forms to a certain type (declensions, conjugations, and so on) in order to search for uniform vocabulary structures. A kind of normalizer language. This greatly increases the accuracy of call typing.
Optimizer
In itself, automatic recognition and tagging simply provides a set of data about calls that should be used to optimize advertising campaigns. It is logical that when the system automatically arranges tags for calls, the contextual ad optimizer “pulls up” calls according to tags, and all this happens within the same system. And not the employee pins it with his hands, using different services. What is it for: the value of the target call for automatic bid management is higher than the value of the sales data: it is more important to have a large array of targeted calls that could lead to the sale than a small sample of actually completed transactions. Since often a quality lead can be inefficiently processed in the sales department.
An example of testing (30 days) of the Predict system in tandem with the contextual advertising optimizer from the developer X:
Number of unique calls (calls from new numbers) - 510
Number of uniquely-targeted calls (call from a new number with a specific duration) - 410
The number of calls with the automatic tag "target" - 360
Therefore, the proportion of truly targeted calls among unique ones is 71%, the proportion of truly targeted among uniquely targeted 87%. From this it follows that 29% of calls are not targeted among unique ones, and 13% are not targeted among uniquely targeted ones. With the help of Predict, we were able to determine the proportion of true target calls, and then using the optimizer, we lowered the CPA by 48% and increased the CR by 115%.
Antifraud
It is impossible to ignore the fact that in some industries a high fraud index is not always: a call that allegedly leads to a sale (housing display, recording for a test drive) leads to a deal, because it can be a bargain from an unfair advertising platform. At the time of the phone call, even if we automatically mark it with a “deal”, the client cannot be sure that this is not a fraud call by the script. The market segments with a high percentage of fraud include, for example, real estate sales (9%). For such companies, tagging works most effectively in the chain: “an incoming call is a system for determining the quality of a call — an antifraud — a tagging — an optimizer.”
Negative detection
Customers of the product were required to control the quality of employees who communicate with customers. In order to eliminate manual labor, the automatic tagging system must highlight these calls in a separate category. In addition to the commonly used negatively colored expressions (machine learning elements are used for this, such as hidden semantic analysis, support vector machine and semantic orientation in this area), you can expand the vocabulary of lexical units that carry risks for customers. For example: the court, FAS, Rospotrebnadzor and so on.
It is quite difficult to be the first to introduce new solutions: how much rake will be - all yours. It took us a year to collect the first five. I think that many more discoveries await us ahead :-)
Source: https://habr.com/ru/post/337080/
All Articles