A little more than a month has passed since the non-profit organization OpenAI, Ilona Mask, introduced the Universe middleware for training and teaching a strong AI. Theoretically, training can occur on all information of mankind, accessible through the Internet. Using the Universe software platform, an intelligent agent uses a computer in the same way that a person does: he will look at the pixels of a computer screen and interact with virtual keyboard and mouse.

Now to the dozens of games available for AI training, the game Grand Theft Auto V has been added , which is notable for exceptional realism.

You can create an AI unmanned vehicle agent on any machine learning framework and are relatively easy to connect to the game on a computer with GTA V installed.
')

Agent AI is at a crossroads. In the console diagnostics, the status of brake pedal depression (true) and other current autopilot parameters are displayed on the left.

The game GTA V, even with disabled violence, provides rich opportunities for learning AI for an unmanned vehicle. It is one of the largest and richest open worlds. The game takes place on the territory of San Andreas Island with an area of ​​almost 20% of this Los Angeles. Here you can run a variety of scenarios for testing AI. Agents have access to 257 different types of cars, 7 types of bicycles, 14 types of weather, and the environment can be changed directly during the simulation.


San Andreas Island

Thanks to numerous mods , you can load buildings from real cities, real cars, real road signs and other objects into the GTA V game. Accordingly, your AI agent will learn how to drive in real road conditions.

Universe Platform

The discovery of the universal universal platform Universe is a continuation of the planned actions of OpenAI to create a worldwide open universal AI. In April of this year, the organization released a public beta version of the OpenAI Gym toolkit for developing and comparing reinforcement learning algorithms. The "Gym" OpenAI Gym consists of a large number of environments (from a humanoid robot simulator to Atari games ). There is a site for comparing and reproducing the results .

OpenAI believes that reinforcement learning is an important way of machine learning that will greatly improve AI. In the process of learning by this method, the subject system (agent) learns by interacting with a certain environment. In contrast to traditional teaching with a teacher, reinforcement signals are a response to AI decisions, while some reinforcement rules are dynamically formed and difficult to understand for a person, that is, they are based on the simultaneous activity of formal neurons.

Running AI agent on your computer with GTA V

The Universe integration with Grand Theft Auto V was created and maintained in the framework of the DeepDrive project, which has now been switched to open source. The project provides for the possibility of launching the Universe Agent on its own computer with the installed copy of the game.



Through the Universe, the artificial intelligence agent gets access to the realistic three-dimensional gaming world of GTA V. The following video shows footage from the game, which were transferred to the AI ​​agent for processing (artificially slowed down to 8 FPS, upper left), diagnostic information from the agent and the environment (lower left) and a view from the camera that is convenient for viewing by a person (on the right)

The integration of AI into the game prohibits any violent actions in GTA V.


To start the agent on your computer, you need to start the server process GTA V. You must first install the Python library universe , and then connect the agent with the following code.

import gym import universe # register Universe environments into Gym from universe.spaces import joystick_event env = gym.make('gtav.SaneDriving-v0') env.configure(remotes='vnc://$host:$port') # point to the GTA V Universe server observation_n = env.reset() while True: steer = joystick_event.JoystickAxisXEvent(-1) # turn right throttle = joystick_event.JoystickAxisZEvent(-1) # go in reverse # Alternatively, use WASD to steer: ('KeyEvent', 'w', True) action_n = [[steer, throttle] for _ in observation_n] observation_n, reward_n, done_n, info = env.step(action_n) env.render() 

As usual for Universe agents, AI uses a virtual keyboard, but in this case a virtual joystick is also available. The best results AI demonstrates with a joystick.

Deepdrive

DeepDrive is a specialized platform for developing open source AI for unmanned vehicles. It uses modder frameworks and special techniques to turn GTA V into a normal car simulator. Pre-trained agents with transport management capabilities and data sets on which they were trained are available.

Although the DeepDrive platform appeared earlier than the Universe, but now its developer has decided that it would be reasonable to transfer its work to this universal open platform. It was the right move. Previously, an agent was required to run a computer under Windows and many hours of setting up the environment (it used to intercept DirectX to capture the screen, and to write agents it was necessary to use the C ++ interface to Caffe under Windows). Now DeepDrive is installed in 20 minutes, it works under Linux and OS X, and you can write agents on any framework for machine learning.

The source code and AMI for GTA V, previously trained agent-driver, are published in open access. His training lasted 21 hours (600 thousand images). The agent has basic driving skills and is a good start for your own experiments.


With the joint efforts of thousands of AI agents, you can create a truly sophisticated driver - a program that can drive a real unmanned vehicle in the real world.