Chatbot Suzette won the Löbner Prize, was able to deceive the judge

The winner of the chatbot competition in 2010 and the winner of the Löbner Award was the Suzette chatbot. As always, the competition was held in the format of a standard Turing text test. The judges had to lead the conversation, not seeing the interlocutors, and then announce their opinion: which of them is a chatbot and who is a person, and also put down the grades for each program.

This year, the best chatbot even managed to deceive one judge, who took him for a man.

Apparently, the quality of chatbot improves every year. For example, the triple winner of previous years ALICE is not particularly competitive. According to the author of the program Suzette, his chatbot won the qualification by a wide margin (11 points against 7.5 from the closest competitor).

The final of the competition was not without surprises. Adventures began at the stage of installing programs on the computer. Dr. Richard Volles (author of the ALICE program) brought three spare disks, but they all turned out to be empty. He had to download the program via the Internet.
')
The Cleverbot Chatbot is now bundled with 45 million lines of recorded chats (he constantly trains in public online), and this index doubles annually. The UltraHal robot began scanning tweets for self-study, so in addition to 300 thousand chat sessions, he now has 400 thousand tweets taken as colloquial samples (the program has certain criteria which tweets can be taken for analysis, including be enough responses to this tweet).

The contest organizers also messed up a bit. Initially, they wanted to invite professors of the English language and other linguists as judges, but because of organizational problems, in the end, all the judges turned out to be developers from the computer department of the California State University. According to the author of the chatbot Suzette, these guys speak in much the same way as his program.

The people who participated in the testing along with chatbots turned out to be students of the same faculty. They did not quite understand their role - to act as humanly as possible and try to convince the judges of this.

In the first round, Suzettte let us down - in one of the moments she literally repeated the words of the interlocutor (the author still does not understand how such a glitch became possible). This round was a complete disaster. In the second round, everything went fine, and in the third, “Suzette” was again close to failure. The judge stuck to her with the question for whom she would vote in the next election, without naming the election. Obviously, he was referring to the election of the governors of California, but before that it was necessary to guess ! When she said she did not know, he demanded to name the current candidates. When she again confessed her ignorance, the judge continued to repeat the question and insisted on answering. "Suzette" began to show signs of irritation, then became angry, and then she became bored and she threatened the judge to interrupt the conversation. That was the end of the conversation. The developer thought it was a matter of seams, he would not win.

In the final vote, Suzette shared second or third place by points, but then it turned out that the judge from the third round took her for a man! Apparently, this was how the manifestation of emotions affected him. It was an instant victory, because this indicator is a priority, and no other chatbot was able to deceive the judge.