Trolling detector: how we increased productivity using a speech analyzer

In the modern world, corporate trolling is as mundane as spam. In the rating of time-eaters, trolls are in the first place, and the point is not only that there are too many divorces, but also that, unlike spam (against which very effective tools have already appeared), corporate trolls mimic under the usual office workers, and the themes for trolling are based on what, in fact, is the work. Therefore, it is sometimes very difficult to distinguish the simple desire of an employee to "troll" from a productive dispute on the merits.

The Mail.Ru Group’s department for increasing the efficiency of its internal resources has found a way to distinguish trolling from productive communication: we created the AntiTroll hardware and software system, tested it and made sure of its effectiveness. Details under the cut.

First of all, it is impossible not to mention that communication with the troll reduces the effectiveness of not only the troll itself (which, like a swamp, becomes more and more wide of fresh water poured into it), but also the employees who participate in the discussion. Therefore, a decrease in efficiency is often epidemiological (viral) in nature.
')
In order to determine the level of trolling, we plunged into discourse analysis, studied the language used by members of our team (we wrote about the terms used by our developers here .

In the course of this analysis, both the form of the language and its function were considered. Conversation and written texts were taken as the source material, in which we identified the linguistic features of the perception of various texts and types of oral speech.

The analysis of written texts included the study of the development of the theme and the semantic links (links) between the sentences. The guys from Mail.Ru Search helped us a lot in this direction.



The analysis of colloquial speech was based both on the above-described aspects and on the practices of step-by-step interaction, the opening and closing sequences of social interactions or the structure of the narrative.

The working group developing the “AntiTroll” mathematical apparatus had to seriously study psycholinguistics, sociolinguistics, sociology, hermeneutics, social psychology and even anthropology (as it turned out, such parameters as gender, height and acoustic properties of the articulation (speech) apparatus should also be taken into account when processing information ).

In parallel with the development of the mathematical apparatus, we collected data for analysis.
To this hooked up our usability laboratory. After a short period of adaptation, the experimental group of employees of the game development department began to talk about their common project, and with the help of the inferior troll the discussion moved from the stage of effective communication to the stage of inefficient trolling on a traditional holivar topic (for example, Android vs iOS or Windows vs Linux) .

Gigabytes of data collected by the pulse, respiration, and cerebral cortex data recorders, microphones, and DVRs were obtained. This source information was fed to the script, written on the basis of the above mathematical apparatus.

The result of the script should be a single number - the level of trolling.

After 28 builds (assemblies) of the script, we were close to abandoning this idea: the script produced some strange values, rather, resembling the result of a random number generator. And then we decided to add another weight vector to the mathematical model: “the level of emotional involvement”. It is precisely called a “subjective evaluative attitude to existing or possible situations”. And suddenly, on the 29th build, "it" worked!

The script became quite accurate in determining the level of trolling, and when someone moved from the productive discussion stage to the word procrastination stage, the numerical value of the trolling level grew steadily.

During the last 7 months, we tested this system in 5 departments of the Mail.Ru Group, gradually adjusting the script's work, and at the moment we have achieved the definition of trolling in 96%.

Interestingly, during testing, an analogue of the “Ballmer peak” , known to most habrayuser, was revealed. Up to a certain value, trolling has almost no effect on productivity, but when the value of X is reached, developers have a sharp increase in productivity (the number of commits increases, the number of bugs decreases). However, when the threshold X is exceeded, productivity drops sharply.

Whether this discovery has practical application is not yet clear: it is obvious that it is rather difficult to maintain the level of trolling in the work team at the required level to achieve this peak.

The next step was the creation of the “AntiTroll Alarm!” Software and hardware complex:



The principle is quite simple: a microphone placed in the conversation sends the audio signal to a constantly running server with an AntiTroll script, and if a high level of trolling is detected, then the Arduino connected to the server via USB receives a signal to turn on the AntiTroll Alarm! Relay. As soon as the trolling level in discussions goes off, the siren and flashing light start:



So far, we have tested our new system in several divisions for three months, but we are already seeing positive results: the number of commits has increased by 23%, the number of bugs found by testers has decreased by 14%.

In the near future, we plan to implement the system in all divisions, and then completely make mobile applications that can be used not only by us, but also by colleagues in the market.

Just imagine - you put your phone in the center of the table in front of the scoop, turn on our application, and do not worry about missing the moment when your discussion moves to a non-constructive direction ...

The source code will later be posted on github here .