Game AI champion from Ilona Musk killed people in video games? Not so easy

You might not have noticed this, but in the first half of August 2017 a small coup happened over the weekend. On Friday evening, in front of a crowd of many thousands, the AI-bot defeated a professional player in Dota 2 - one of the most popular video games in the world. The human champion, a polite lad Danil “Dandy” Ishutin, surrendered after being killed three times, and said that he could not defeat the unstoppable bot. “He’s a bit like a man,” said the Dandy. “But at the same time it looks like something else.”

The bot's father was none other than the techno billionaire Ilon Musk, who helped finance and establish the organization that developed it, OpenAI. He was not at the event, but he expressed his attitude on Twitter.


OpenAI was the first to beat the best players in the world in competitive e-sports. It is a much more complex form than traditional board games like chess or go.

What is even more interesting, OpenAI itself has learned everything that it knows. He studied, constantly playing with himself, accumulating numerous "careers" in the gaming experience in just two weeks.
')
What follows from this? Was the Friday show more impressive than the Google AI wins in the go board game? In short, probably not, but still it represents a significant step forward - both for eSports and for the world of AI.

Yes, video games are harder than chess

First, you need to consider Mask’s statement that Dota is “a much more complex form than traditional board games like chess or go.” It really is. Real-time battles and strategic games like Dota and Starcraft II are challenging challenges that computers are still unable to cope with. These games require strategic thinking, and, unlike the board, they conceal important information from the players. You see everything that happens on a chessboard, but not in a video game. This means you need to predict and anticipate everything that your opponent can do. It requires imagination and intuition.

In Dota, this complexity increases with the work of people in teams of five people coordinating strategies that vary in the course of an action depending on the characters used. To further complicate the task, the game has more than 100 different characters, each of whom has a unique set of skills; characters can be equipped with different unique items, each of which, when applied at the right time, can lead to a win. All this means that it is impossible to program winning strategies in a bot for Dota.

But the game that OpenAI played was not so difficult. Instead of "5 for 5" he played with people in "1 for 1"; instead of choosing a character, the person and the computer had the same hero — a comrade named Shadow Fiend [Shadow Demon], whose set of attacks is quite straightforward. My colleague Vlad Savov, who hooked up on Dota, who also described his impression of the Friday game, said that the 1 on 1 match represented “only a small fraction of the complexity of the full competition”. So - probably not as difficult as go.

Cannot be considered a better calculator

The second big trick is the advantages that OpenAI has over man. One of the main disputes in the AI ​​community was to discuss whether the bot has access to the Dota API for bots - this would allow it to connect directly to the flow of information from the game, to parameters such as, for example, the distance between players. Greg Brockman from OpenAI confirmed to our publication that the AI ​​actually used the API, and that certain techniques were firmly sewn into the agent, including the items he used in the game. He was also taught certain strategies using a trial and error technique called “stimulated learning”. In general, he was trained a little.

Andreas Theodorow, an AI researcher in games from the University of Bat and an experienced Dota player, explains why it matters. “One of the main features of Dota is that you need to calculate distances in order to know how far some attacks spread,” he says. The API allows bots to estimate distances. So you can say: 'If someone is 500 meters away, do it', but the person has to calculate everything on his own, learning by trial and error. If bots have access to information that a person does not have, this gives them an advantage. ” This is especially important for the game "1 on 1" with such a hero as Shadow Fiend, where players have to concentrate on choosing the right time to attack, and not on the overall strategy.

Brockman says that learning this skill for AI is a trivial task, and it has never been central to research in OpenAI. He says that the institute bot would have done without the information from the API, but “he would simply spend much more time to acquire the skills of vision, which already works, so what's the point?”

Some skills can be learned, but not taught.

Bearing in mind all this, is it possible to postpone the victory of OpenAI? No way, says Brockman. He points out that the way he studied independently was more important than the victory itself. The previous AI champions of the type AlphaGo learned to play games, processing past matches of people-champions, and the OpenAI bot itself learned (almost) everything that it knows.

“You have a system that just played against itself and developed enough robust strategies to defeat the professionals. This should not be taken for granted, says Brockman. - And this is a big question for any machine learning system: how does complexity get into the model? Where does it come from? ”

According to him, the OpenAI bot shows that we don’t need to train computers for complex things: they can do it themselves. And although some of the bot's behavior was pre-programmed, he developed some strategies himself. For example, he learned how to deceive opponents by pretending to launch an attack, but canceling it at the last moment, and forcing a person to repel an attack that was not there - just like a trick in boxing.


Others rate it more skeptically. AI researcher Denny Brits, who wrote a popular blog entry on this topic, told us that it is quite difficult to assess the degree of achievement without knowing the technical details. Brockman says that they will follow, but when I couldn’t say for sure. “Before the release of the work it is not clear what were the achievements from a technical point of view,” says Brits.

Theodorou points out that although the OpenAI bot won Dandy in a competition, when the players looked at his tactics, they were able to outwit him. “If you study their strategies, it is clear that they played, not like everyone else, and won,” he says. The players used non-standard strategies - they would not have surprised a person, but the AI ​​has not yet seen them. “The bot was not flexible enough,” says Theodorow. Brockman objects that after learning new strategies, the bot would not have succumbed to them again.

All experts agree that this was a serious achievement, but the real difficulties are just beginning. It will be a 5 on 5 match, where OpenAI agents will have to not only engage in duels in the middle of the map, but also work on a sprawling, chaotic battlefield, with many heroes, dozens of support units and unexpected turns. Brockman says that OpenAI is now targeting the Dota tournament next year, which will take place in 12 months. And during this time you need to spend a lot more workouts.