Single-pixel camera with active backlight accelerated 50 times

Samples of photographs taken with the new single-pixel camera with illumination of just 50 patterns from the random Bernoulli distribution (M = 50)

In traditional digital photography, it is customary to use a lens that generates luminous flux and directs it to a matrix of photosensitive elements — a CCD or CMOS sensor of a digital camera. It is believed that the more elements in the matrix, the better the picture: 20 million elements are better than 13 million. This is one of the main indicators of photo quality, together with the lens aperture and the density of the elements themselves (the lower the density, the larger the physical size of the matrix, the less distortion and interference the elements give).

But what if the camera has no lens at all, and the photosensitive matrix consists of 1 (one) pixel? Modern digital signal processing technologies do allow for very quickly collecting and processing the light flux with a single light-sensitive element. In this case, the camera does not need a lens or a large matrix. That is, the photographic equipment is replaced by computer calculations.

The single-pixel photography technique is part of a new approach called computational photography . This includes other innovative computational methods for manipulating the luminous flux: gluing panoramas, expanding the dynamic range due to multiple exposures with different exposures, cameras of the light field, even the notorious slit photography can be attributed to this category.
')
In a computational photograph, the subject is measured , which is then decoded to produce an image. One of the fundamental tasks is to get rid of the lens altogether, since this is the most expensive and heavyweight part of the camera. With a sufficiently powerful processor lens is not needed.

One of the most promising areas of computational photography is measurement compression (compressive sensing). It is the use of such technology that makes it possible to design a single-pixel camera, first described in the literature in 2008 . The original model of a single-pixel camera is an optical computer consisting of one photosensitive diode, a digital micromirror (DMD), two lenses and an analog-to-digital converter.


Original model of single-pixel camera in the laboratory (2008)

Despite this primitive design with a single photodiode, the camera quite recognizably captured objects when shooting. In this case, the digital micromirror acts as a spatial light modulator (PMS) - a necessary element of any single-pixel camera. In the role of the ICP there may be another device that imposes a certain spatial modulation on a beam of light under the control of a computer. So the signal is encoded, and after registering and recording it is decoded back and forms an image.


Reconstruction of the measurements made by the photodiode, in the form of black-and-white and color images of 256 × 256 pixels (the second and third photos) in the 2008 sample model. On the first image - the original image.

A single-pixel camera requires thousands of successive measurements, where speed is key. The faster the measurements are taken, the better (it is desirable that the object does not move during the measurements).

Technological progress does not stand still, and the sensors of the latest generation are capable of recording the signal much faster than the previous ones, which opens up new opportunities for creating single-pixel cameras of better quality. In a recent issue of IEEE Transactions on Computational Imaging for March 20, 2017, a modern single-pixel camera with an illuminated object is described, which works on a time scale of picoseconds.

The authors of the scientific work from the Media Lab laboratory at the Massachusetts Institute of Technology explain that the picosecond interval makes it possible to distinguish individual photons arriving at the sensor from different points of the object with millimeter resolution. At the same time, the sensor registers more information from this photon than just the fact of arrival (as opposed to a regular pixel), so that fewer masks are required as PMS.

Single-pixel cameras with an interval of picosecond or less have been created before, but they suffered from low sensitivity. Single Photon Avalanche Photodiode (SPAD) demonstrated good sensitivity - at the level of modern CMOS, but it had a low speed of operation, dozens of picoseconds between measurements.

The new research paper describes a device that combines the advantages of all single-pixel cameras created earlier: it works on a picosecond scale and demonstrates sensitivity as in SPAD. Experiments have shown that in order to obtain high-quality images on such a time scale of measurements, they require 50 times less illumination patterns than in traditional single-pixel cameras. That is, you can create an image 50 times faster.


The principle of operation of the ultra-fast single-pixel camera of 2017 sample: (a) backlight with wavefront modulation; (b) measuring the time of arrival of reflected photons with an ultrafast omnidirectional sensor

The principle of operation of the camera is shown from the illustration. You need to understand that every point of the photographed object reflects the light cone. By registering photons from a cone, it is possible to draw a conclusion about the shape of the whole object and the distance to each point.


Registration of photons from the light cone at different points in space (above) and signal analysis (below)

The camera demonstrates the excellent quality of photography. The following samples compare the photographs of the sample (above) with a conventional single-pixel camera with 50 and 2500 backlight patterns (second row), as well as a one-pixel camera of a new design (third and fourth row). The backlight patterns are selected from the Bernoulli random distribution in {−1, 1}, as in the experiment with a conventional single-pixel camera. Please note that only 50 templates are used in the samples from the new camera.



As you can see, the new development has a much higher quality of shooting, even with a dramatic decrease in the backlight patterns.

The design of a single-pixel camera with active backlighting far exceeds that of single-pixel cameras in a number of technical specifications. This makes it possible to expect that in the future such cheap cameras without moving parts may find practical application in everyday household appliances.

The scientific work was published on March 20, 2017 in the journal IEEE Transactions on Computational Imaging (doi: 10.1109 / TCI.2017.2684624, pdf ).