Machines are like children: can AI learn to predict the consequences of its actions?

Young children are well aware of what will happen if you turn a glass with juice. But no car. Despite all the diversity of modern algorithms, the computer is not able to predict the consequences of one or another of its actions. Of course, if this computer is not specifically train.

A group of researchers from the Paul Allen Institute for Artificial Intelligence (Allen Institute for Artificial Intelligence, AI2) has developed a program that helps a weak form of AI to “understand” what will happen when performing a particular action. The software “predicts” the future, showing how this or that object can behave in certain conditions. This, scientists say, will help AI make fewer mistakes. For example, an autonomous car can “predict” the consequences of the development of a given situation on the road.

The system developed by Ruzbeh Mottaghi (Roozbeh Mottaghi) and his colleagues demonstrates the features of a number of objects. The software platform created by scientists uses machine learning and 3D modeling. Researchers have converted more than 10,000 images into three-dimensional scenes. For this purpose, a specialized 3D engine was used.
')
After the transformation, the original images and their volumetric analogues were loaded into neural networks. As a result, the computer system was gradually trained on the example of simple 3D objects and the movements of these objects. After the system received a certain number of “image-3D model” pairs, it learned to independently assume what forces can be applied to a three-dimensional object, and what the consequences of the manifestation of these forces can be.

This is not to say that the work went very smoothly. But the situations in which a computer could predict the behavior of a three-dimensional object when interacting with it turned out to be more than situations when the machine did not understand what could happen. The AI ​​“understood”, for example, that if the stapler lying on the edge of the table was pushed, then it would fall. The system also successfully showed the situation with a coffee table and a sofa. The AI ​​was able to “understand” that if the coffee table moved towards the sofa, the table would certainly rest on the sofa and would not be able to move further.

“Our goal is to study the dynamics of the physics engine. You must learn to predict the possible behavior of the objects in the scene, ”says the project manager.

The results of this work can be useful in many areas. So, the trial and error method is far from being applied everywhere. In the case of a mobile, this is absolutely excluded. The computer should predict the possible development of the situation, and do it very quickly. Collecting and analyzing data in such situations is very important. Here is another example. The customer service system in the store also cannot push goods from the shelves in order to understand the consequences of their actions. It will cost the store owner a lot and scare customers away.



The work that scientists do is part of the Project Plato project. Its goal is to enable robotic systems to understand the consequences of their actions without testing it in practice. In particular, systems with a weak form of AI, which are used in the project, understand how a skier going down a mountain will move. They also understand how the football that the football player has just flown will move. Such events and their consequences AI must learn to understand in real time.

In recent years, computer systems have noticeably become smarter. They can already analyze images with the subsequent breakdown by categories, identification of elements of images and tagging. Brendan Lake , a specialist from New York University, believes that the Ai2 project is important for the modern world. “Understanding the scene is much more difficult than object recognition,” says Lake. "When a person sees a frame of a scene, he is able to tell the whole story about what is happening or what can happen on the specified frame." Ideally, the car should be able to do the same.

Of course, for the time being, a person greatly exceeds the capabilities of the machine in the example given. But the task of scientists is to teach computer systems to analyze the possible consequences of their actions. The fact that computers in this sense become equal to man or transcend him is not yet in question. But at the current stage it can be very useful for a number of areas.

Scientists participating in the project do not hide the results of their work. Source code, data set, and everything else can be obtained from this link for self-study.