AI from Google independently mastered 49 old Atari games

Google has created an artificial intelligence system that plays a better person in many arcade games. The program learned to play, without knowing the rules and not having access to the code, but simply by watching the picture on the screen.

This development is not as frivolous as it may seem. A universal self-learning system may someday be used, for example, in autonomous cars and other projects where it is necessary to analyze the state of the surrounding objects and make decisions. For example, when installed in an autonomous vehicle, the AI ​​will determine by trial and error which traffic light signal is best for passing through the intersection. If no joke, the program is able to find a solution for a wide range of tasks, regardless of the rules and initial conditions.

Another interesting fact is that in 20 games the AI ​​could not beat a man. For example, he seriously screwed up in the game Pac-Man, and not having learned how to plan his actions for a few seconds ahead. He also did not understand that by eating certain magic balls you can devour ghosts. As a result, the program managed to score only 13% of the record set by the best professional player.
')
The neural network training under the name DQN was carried out by the London unit of Google DeepMind. Artificial intelligence was not informed of the rules of the game. The neural network itself analyzed the state and looked for a way to score the maximum number of points. In training and decision making, she considered only the last four frames.

As a result, DQN was able to outperform the best result of human players in 22 out of 49 games and beat any other specialized computer algorithm in 43 out of 49 games.



“This is really the first algorithm in the world that corresponds to the human level on a wide variety of complex tasks,” says Demis Hassabis, co-founder of DeepMind.

The research results are published in the journal Nature.



Trained neural networks are often used in pattern recognition systems, and DeepMind used the reinforcement learning method when the AI ​​gets a “reward” for performing certain actions - and independently improves the result as experience accumulates.

The program showed itself best in simple games like pinball (2439% of the result of a person), boxing (1607%) and in the game Breakout (1227%), where you need to hit the ball, clearing blocks on the screen. She even mastered the trick of professional players when a tunnel breaks through in an array of blocks and the ball starts at the top of the screen!



“It really surprised us,” said Hassabis. “This strategy stems entirely from the underlying game mechanics.”

Computers have long been used to control gameplay, but modern AI systems have reached a new level. DQN self-learning involved analyzing information on the screen in real time, that is, processing approximately 2 million pixels per second. At this rate, AI in the future will be able to learn how to analyze the surrounding reality of the real world in real time, shooting everything around it with the help of video cameras. This opens up completely new applications for it.