MIT has developed an inexpensive 3D camera that can see through translucent objects

The existing 3D sensors based on measuring the time of passage of a ray of light , like Kinect, can quite well map the depths of the image if there are no translucent and reflective surfaces in the scene that refract and distort the path of the light rays. Rain, jets of water, fog, or objects made of glass or transparent plastic just obscure more distant objects, since the sensor only recognizes the very first reflection of the signal.

At the Siggraph Asia conference, which was held in Hong Kong last week, scientists from the MIT Media Lab presented a new development based on the same principle. It uses a conventional laser diode and an inexpensive sensor. The cost of equipment was only about $ 500. The only significant difference between the prototype and commercial counterparts, such as Kinect, is firmware.
')
Instead of simply periodically modulating infrared laser pulses, the MIT prototype uses specially selected pulse sequences that have an autocorrelation function with a very narrow single peak (such signals are used in echolocation and telecommunications to accurately measure the signal delay time. For more information, see “ Basic Principles digital wireless communication. Likbez ", in the section" Autocorrelation function. Barker codes "). The camera clearly distinguishes the first reflections from the surface of a transparent object and the reflections following them from farther objects. She is even able to get a clear image of the inscription, closed by a matte screen.

The development of MIT is based on previous work: a camera with a picosecond time resolution, and a system created on its basis that can capture an image of objects that are behind an opaque obstacle. The system is based on measuring the time of arrival of the reflected signal from a femtosecond laser flash. Lidars, radars and echo sounders work in a similar way. However, the cost of a prototype camera, capable of looking around the corner, was about half a million dollars; it requires lengthy calibration and signal processing.

The videos that were posted on Youtube by the participant of the project Asuta Kadambi clearly demonstrate how the camera sees the advance of the light pulse front of the scene. First, the camera perceives reflections and glare from the surface of a transparent object, and then the image of other objects hidden behind transparent ones.



In the video with a mirror in the frame, the advance of the light front is seen especially well:



The new system has very good prospects for commercial use, since it can work on the already existing low-cost and massively sold hardware. The early prototype spends about four seconds to process one frame, so one can only hope to work in real time after serious optimization. Sensors built according to the new principle can theoretically replace very expensive lidars in unmanned vehicles and thus bring their mass implementation closer, they can be used in medicine, and in general, noticeably improve almost all the characteristics of 3D cameras without changing the hardware.

On the project page you can download videos and PDF with a detailed description of the technology.