3D modeling of the environment with a video camera

New technology from MIT allows you to simulate a 3D-map of the area / room using a conventional video camera.

The bottom line is that a regular camera removes the surroundings and remembers it using a special algorithm. As soon as the camera returns to the starting point of the route after its circular journey, the algorithm understands that it is the same place, and quickly “stitches” the ends of the space, forming a coherent 3D map.


“I have a dream to make a complete model of the entire Massachusetts Institute of Technology,” says John Leonard, who works in computer science at the MIT artificial intelligence laboratory. “With this 3D map, applicants would be able to“ swim ”across MIT like fish in a large aquarium. There is still much to do, but I think it is doable. ”

Leonard, Whelan and other members of the team - Michael Kayess of MIT and John MacDonald of the National University of Ireland - will present their work at the 2013 International Conference on Intellectual Work and Systems in Tokyo.
')

Device, principle of operation

The problem of millions of points

The Kinect camera captures a color image, and a depth sensor measures the distance to each pixel of the resulting image. This data can be processed by the application to create a 3D map.

In 2011, a group from King’s College London and Microsoft Research developed applications for such 3D modeling, called KinectFusion, which successfully builds 3D models in real time. Very detailed models of space are obtained, with a resolution of less than a centimeter, but only for a limited fixed region in space.

Whelan, Leonard and their team attempted to develop a technique for creating 3D maps with equally high resolution over hundreds of meters, in different conditions and in real time. The goal, they note, was ambitious in terms of data: the modeled environment will consist of millions of 3D points. In order to create an accurate map, it would be necessary to determine which homogeneous areas can be leveled without compromising the quality of millions of other different areas. Previous research groups have solved this problem with repeated surveys - an impractical approach if you want to create a map in real time.

Instead, Whelan and his colleagues came up with a much quicker approach in which the camera shoots space in two stages: using the first camera on the front of the device and the second camera on the back of the device.

In front of the device, researchers have developed an algorithm for tracking the position of the camera at any time along its route. Since the Kinect camera does 30 frames per second, the algorithm shows how much and in what direction the camera moves between frames. At the same time, the algorithm creates a 3D model consisting of a cloud of small pieces — cross sections of thousands of 3D points in the immediate environment. Each piece of cloud is associated with a specific camera position.

As the camera moves down the corridor, the pieces of the cloud are included in the global 3D map.

The camera in the back of the device again removes the surroundings, finds familiar pieces corresponding to the camera position and completes the missing pieces. Thus, the device automatically assigns the location of the cloud of pieces to the position of the camera instead of memorizing the location of each of the pieces individually.

The team used its technology to create 3D maps of the center of the Massachusetts Institute of Technology, as well as indoor and outdoor locations in London, Sydney, Germany and Ireland. In the future, the group suggests that the method can be used to give robots much richer information about their environment. For example, a 3D map will not only help the robot decide whether to turn left or right, but also provide more detailed information.

“Imagine that a robot could look at one of these cards and tell where a fire extinguisher is located in a fire, or take other reasonable actions based on the situation,” says Whelan. “These are systems of the type“ saw and made, ”and we feel that there is great potential for this type of technology.”

Costas Danilidis, a professor of computer and information science at the University of Pennsylvania, considers the research team's device as a useful method for using robots in everyday tasks, as well as in building inspection.

“A system that minimizes errors in real time allows the lawn mower or vacuum cleaner to return to the same position without the use of special markers. The same idea applies to the navigation of the rover. ”Says Danilidis, who did not participate in the study. “This device makes it possible to use accurate 3D environment models for verification in architecture or infrastructure. It would be interesting to see how the device behaves in difficult climatic conditions. "

This study received support from the Science Foundation of Ireland, the Irish Research and Management Council for Naval Research.