How we taught the robot to help support

Truth of life: even an ideal software will not save its developer from client requests to technical support. And if on your software (talking about Parallels Plesk Panel) about 50% of all servers for web hosting in the world are spinning, messages to support rush with terrible force. In addition to the error messages and breakdowns of Parallels Plesk Panel (alas, it happens), support comes to the implementation of custom configurations, the use of non-standard settings and requests of features needed by clients for their future implementation. Customers do not always follow the development of the product and often simply do not know that the subject of their request is implemented in one of the previous updates, and non-standard settings are already described in the article Knowledge Base article for Parallels Plesk Panel. It was necessary just to figure out how to intercept such requests “on the fly” and automatically provide users with answers, thereby unloading the support for really complex cases that are not mentioned in the knowledge base.

This material outlines the experience of Parallels Plesk Service Team (there is Twitter and a group on Facebook ) - a structural unit of a large company. But I'm sure the article under the cut will be useful for startups who already have a finished product or service, but no technical support. And in which product questions are answered by a product manager or a founder, doing so on the way to the office or before going to bed from home. Our system helped to reduce the number of incoming tickets by 3-5%. Parallels Plesk Panel-wide thousands of man-hours.
')

Why did we decide to do this?

During the existence of Parallels Plesk Panel, the technical support team and the Parallels Plesk Service Team have accumulated a knowledge base on various problems and errors, on making some non-standard settings, as well as on where and in which updates a bug has been fixed or a new one has been added. functionality. Knowledge is systematized and published in the form of Knowledge Base articles. Knowledge base update is non-stop. Practice shows that even the best search and the most comprehensive data in the knowledge base do not guarantee that professional clients (in our case, hosting companies) will independently and quickly find a way to solve their problems with the help of relevant articles from the search results.

How to be? We decided to bring the knowledge base closer to the users as much as possible - integrate it at different stages of the problem detection and reporting by the client. In a nutshell: we had to come up with a system (hereinafter, for brevity, we will call it that), which would automatically offer the client solutions to his problem, taken from the knowledge base on the basis of the description of this problem, which he wishes to inform our support service.

The implementation of this system would allow reducing the number of incoming tickets to the technical support service due to the fact that the client would independently solve the arising problems as soon as possible based on the solutions proposed from the knowledge base. For hosters, whose business is in many respects tied to the uptime of client servers, the speed of “fixing” problems is a critical parameter.

What did we start with and why?

To begin with, we created a signature database. Each specific article in the knowledge base was assigned one or several signatures, which most accurately and uniquely describe and represent this or that problem. For example:

Signature	KB article
1. Can't locate object method "get_msgid" via package "Mail::SpamAssassin::Bayes" 2. Spamassassin training 3. Train spamassassin 4. Spamassassin train	kb.parallels.com/112919
1. Can't connect to local MySQL server through socket 2. Trying to establish test connection... ERROR 2002 (HY000)	kb.parallels.com/114289
1. Error code: (2) Could not find a valid SPF record	kb.parallels.com/6051

The accuracy and uniqueness of the signature is the main criterion for the success of the system.

for example

2012-09-11 05:41:13: (mod_fastcgi.c.2746) FastCGI-stderr: PHP Warning: date(): It is not safe to rely on the system's timezone settings.

very bad signature because there may be another time in the report from the client

2012-09-11 05:41:13

or a completely different number

mod_fastcgi.c.2746

An article with a solution to this problem will not be found. Therefore, we tried to find a compromise between the uniqueness of the signature and its universality.

An example of a good signature:

FastCGI-stderr: PHP Warning: date(): It is not safe to rely on the system's timezone settings

Very simple and monosyllabic FastCGI-stderr type or SPF record lead to the fact that the corresponding database article will often be offered completely unreasonably in all cases where these combinations will be mentioned in reports from clients. The accuracy of the search falls, the customer is torn between several articles in the KB, the time to solve a problem grows, the customer’s customers are nervous.

So, the signature database is created. We uploaded it to the Knowledge Base server, and it is ready to go. A tool was needed to use this base. We made it as a separate server for API requests under the code name Troubleshooter.

A reasonable question: why was the search system selected for static signatures, and not more advanced? The answer is simple - this option is the fastest, and the speed of work is the most critical parameter for our system. When a client comes in and fills out a ticket form with his text, he does not have to wait more than a few seconds until this text is checked by the system and the page with the scan results is loaded. After the “running in” of the system, we see how we can improve the quality of search in the database of signatures - for example, by using regular expressions or using well-known semantic analyzers to form more relevant signatures.

Where and how is the system used?

Before we talk about the mechanism of the system, it is necessary to describe three main options for its use:

1. The most important and most basic use of the system is the integration of the system into the form of sending a ticket on the corresponding page of the Parallels website.
2. We have created and released utilities for executing them in the Linux or Windows command line. These utilities are essentially an offline version of the system and are add-ons above the signature database, which searches for known signatures in certain Parallels Plesk Panel log files and, based on the signatures found, offers a list of relevant Knowledge Base articles. In more detail, this utility, called the Spider Tool, is described in the KB article and is discussed in the corresponding section of the Parallels forum .
3. The third use of the system is designed for technical support when working with tickets.

In more detail about how each of the use cases works, I will tell you below.

In the future, we will offer our customers another three possible uses of the system. First, we open the API for those customers who wish to integrate it into their infrastructure. The second is the online form of the Troubleshooter, through which anyone can enter any problem and analyze it for the existence of the relevant article in the Knowledge Base, or download the entire log file and analyze it. And the third is the integration of the system directly into the Parallels Plesk Panel interface.

How it works?

Let's take a closer look at how each of the three existing use cases works.

Option 1.

A client comes to a special page of the Parallels website, where requests are accepted for service, and after going through all the steps, he writes a text describing his problem. Then he presses the button “Send ticket to support service”. But in fact, the ticket in the support does not fly away immediately. At this stage of the ticket (what is written in the Subject field) and the text of the ticket are analyzed by the system for the presence of known signatures. Before the final sending of the ticket to the technical support service, the Knowledge Base articles found on the basis of the signatures are provided to the client. The client views these articles and solves his problem on his own with the help of the proposed articles, without sending a ticket to the technical support service. If none of the articles offered to him helped or no articles were found, the client can download the Spider Tool proposed for his platform and use option 2. Option 2 involves installing the Spider Tool to search for problems in the Plesk logs and get a list of relevant articles in KB to solve them. In the case when the client did not help any of the suggested options, he finally sends the ticket to the technical support service using the Submit ticket anyway button. In the screenshot you can see how it looks in reality.

Option 2.

Install and run the command line Spider Tool. Here is its current version for Linux , and here is the version for Windows (both versions are downloaded by clicking). Distributions for both platforms contain the most recent signature database and a special user space, which is designed to scan the most important Parallels Plesk Panel log files to search for matches with the signature database. As a result, the user receives a list of problems found on his Plesk-server with links to relevant Knowledge Base articles for solving these problems on their own.

The utility has many different options that extend the possibilities of its use. You can read more about it here .

Option 3.

As stated in the description of the first option, the system scans the "body" and the subject "Subject" of the client's letter to the technical support service only at the stage of its creation. All subsequent client correspondence with the support is carried out within the framework of the created ticket and can be scanned at any time by the technical support engineer manually with the help of a special button in the Request Tracker. The system will analyze all correspondence taking into account new details of the problem that have appeared and will offer the engineer Knowledge Base articles that may be useful to him when working on a ticket.

The scheme made on the knee looks very clumsy, but it faithfully reflects the mechanism of the system operation in all three use cases:

findings

The system is in use relatively recently. With its integration, the possibilities were laid for collecting statistical data. This data is collected and analyzed every month. At the moment, we get the values ​​of several important metrics - the effectiveness and quality of Knowledge Base (KB Quality) articles and the efficiency of the system itself (Ticket Avoidance, Accessibility).

According to the most preliminary data, the system allowed to reduce the flow of incoming tickets by about 3-5%.

Further improvement will significantly increase this figure. In addition, the planned removal and other metrics. For example, the rating (top) of Knowledge Base articles, after showing which the client did not press the button for the final sending of the ticket, for a more accurate and detailed assessment and analysis of the system and the effectiveness of existing articles to solve the client’s problems.

Of course, there are more powerful systems for solving the problem. But they are usually intended for industrial search systems. Providing the necessary speed of text processing requires significant resources. In addition, we already had an existing Knowledge Base on Plesk with a volume of about 6 thousand articles, which would also require substantial resources for modification. Therefore, we refused to use the already existing expensive solutions as unjustified for our use, but created our own fast, cheap and lightweight system, which even in the initial version shows a very good result.

UPDATE: Today we launched Troubleshooter for real use online. Use and ask questions!