Hybrid film and CCD / CMOS matrix is ​​not possible? Try saying this to the Foveon X3!

Lyrical digression.

One interesting site caught my eye - www.foveon.com . It contained information that Foveon was developing three-matrix systems (also called 3CCD) for Sigma digital cameras from Sigma Corporation . I was very surprised, both by the technology in general, and by the fact that at Habrahabr there were still no references to this remarkable development. All information is taken on the website www.foveon.com

Image Sensors

Foveon X3 is the most high-tech and advanced color sensor direct image registration, which has ever been developed. His development allowed to make a giant leap in digital photography, because The Foveon X3 sensor combines the advantages of both digital photosensors and the principles of preserving the image of a conventional photo film.

The Foveon X3 Direct Image Sensor is an image sensor that can capture both red and green and blue light at the same point during a single exposure. While working on the Foveon X3 sensors, a lot of research has been carried out in the field of image acquisition using CCD / CMOS sensors and the principles of image acquisition using a conventional photographic film.
')


The first figure shows the principle of obtaining images on film. (The bottom line is that red, blue and green rays are recorded at different depths on photographic film, due to the fact that after refracting light by a lens, the rays of the blue part of the spectrum will focus closer to the surface of the film than the rays of the red part of the spectrum. Ie the red rays fall a little deeper into the thickness of the film, while the blue rays are focused almost at the surface itself. This spread is about 5 μm)



The second picture shows the visual scheme of the work of the conventional CCD / CMOS Matrix. (Drawn ray passing through Bayer Filter )

In the last picture, the Foveon X3 matrix is ​​provided, and it is visually illustrated how it combines the principles of operation of the CCD / CMOS matrix and film.

Film.

For more than 100 years, color film has been the gold standard of photography. It is the photographic film that makes it possible to obtain natural warm tones, incredible detailing and color reproduction, which we have long been accustomed to. The film allows you to get this result due to the presence of a three-layer structure (three layers of emulsion, each of which is sensitive to a specific part of the spectrum, blue, green or red), to most fully capture all shades of color at any point in the image.

CCD matrix.

The CCD matrix was developed approximately 30 years ago, but even at the peak of its development, it still does not give an opportunity to get the same colorful and detailed image as on the photographic film. This is due to the fact that at one point in time digital CCD / CMOS image sensors are capable of registering only one color at one point in space.

Foveon X3

The developers of the Foveon X3 sensor have combined the best of what was in the principles of image registration with a film and a CCD / CMOS matrix. The result is a three-level Foveon X3 sensor. The structure is similar to film (contains three layers, each for registering the rays of its part of the spectrum), but instead of a photosensitive emulsion, in each layer of three, the Foveon X3 sensor has a matrix of pixels that register its part of the spectrum. Layers with pixels are arranged in such a way as to use the fact that red, green, and blue light are focused in a silicon matrix at different depths, which allows you to create one sensor sensitive to all three parts of the spectrum at the same point of the matrix. This made it possible to obtain a digital matrix that is capable of registering an image of the same quality and the same detail as a film could have done.

Comparison of pixel locations in a conventional CCD / CMOS sensor and in the Foveon X3 sensor

Before the invention of the Foveon X3, the ratio between the number of pixels and the number of their location was 1: 1 for a traditional CCD CMOS image sensor. Given this relationship, the generic term “pixel” was commonly used to refer to both a pixel (photo sensor) and its location in the matrix. Foveon X3 is a new type of sensor that includes three pixels instead of one at each point in the image. Accordingly, the definition of the pixel itself is preserved, but now implies that the ratio between the number of pixels and the number of their location will correspond to 1: 3. This means that at one point of the image there will be not one pixel, which is capable of registering only one third of the color information, as in the CCD / CMOS matrix, but as many as 3 pixels in different layers of the same matrix, which will allow to get a more complete color picture of the photographed image.

The better three-matrix system?

What is so good three-matrix system Foveon X3 ? And it is good because for registration of the same amount of light a three times larger area is used. After all, the matrix Foveon X3 essentially consists of three separate matrices located on top of each other:

• Blue - thickness 0.2 µm from the surface;
• Green - thickness 0.4 microns;
• Red - the depth in the thickness of the crystal is more than 2 microns;

Let me use the services of Wikipedia on this issue -



The thickness of each layer was chosen according to the results of experimental studies on the depth of penetration of quanta of the corresponding spectral range into silicon. The layers in which the photoelectric effect occurs are separated by additional thin zones of low-alloyed silicon and have separate signal outputs.

Due to the small (less than 5 microns) thickness of the sensor, the possible influence of chromatic aberrations on the image is minimal. However, as in other varieties of matrices, the absorption of the red part of the spectrum occurs at maximum depth. As a result of the parasitic diffusion of photoelectrons and illumination with oblique rays in the region of maximum wavelengths, additional image blurring occurs. In particular, the same effect makes it difficult to further (compared with the current matrices) reducing the size of the element and increasing the resolution.



If we consider the usual CCD / CMOS, knowing that the pixels in it are located only in one layer and one pixel can contain information only about one part of the spectrum, we can go so far that 50% of the green color falls on the green (the highest sensitivity of the human eye falls on the green part of the spectrum, so the green pixels are 2 times larger), and the blue and red accounts for 25 percent of the total number of pixels. It turns out that not only does the usual CCD / CMOS matrix have a total area of ​​all pixels less by two thirds, so the number of pixels of an individual color is even smaller with respect to the total area of ​​the matrix.

Step by Step Comparison of CCD / CMOS and Foveon X3 Matrices

Here are two images:
from a conventional camera with a CCD matrix and made with a camera with a Foveon X3 matrix



On the images marked areas on which we will compare the quality of work of both matrices.

Sharpness

CCD / CMOS matrix Foveon X3 matrix

Detailing color objects

CCD / CMOS matrix Foveon X3 matrix

The presence of artifacts

CCD / CMOS matrix Foveon X3 matrix



Interactive demonstration of the differences between CCD / CMOS and Foveon X3 .

Variable Pixel Size VPS (Variable Pixel Size)

Foveon X3 has another interesting feature - it can change the pixel size (in the future we will call this technology VPS). VPS technology opens up new horizons for video cameras and cameras in which they are installed. The uniqueness of the technology lies in the fact that the matrix can change its resolution by combining signals from several pixels into one, which allows to perceive a group of pixels as one big pixel.

For example: a sensor of 2300 x 1500 pixels contains approximately 3.4 million pixels. If you combine all the pixels in groups of 4 x 4 pixels using VPS technology, then we would end up with a matrix of 575 x 375 pixels, but each of them would be 16 times larger than the nominal pixel. That would significantly increase the amount of light received by one pixel, and would inevitably lead to an increase in the sensitivity of the Foveon X3 matrix.
The most common possible pixel group configurations are:

• 2 x 2 pixels
• 4 x 4 pixels
• 1 x 2 pixels

Managing groups of pixels occurs through a complex scheme integrated in Foveon X3

Because the Foveon X3 captures all of the visible range at each pixel location, the pixels in the combined groups are full-color “super pixels”. No other sensor can boast such capabilities.



Interactive Demonstration of VPS Technology

Grouping pixels increases the “Useful Signal / Noise” ratio, allowing you to take full color shots in low light with minimal color distortion. Thanks to the VPS technology, it became possible to increase the pixels, reducing their total number and resolution, but at the same time being able to process video image frames more quickly, thereby increasing the speed of video recording.

This allows you to shoot higher-quality digital video, which, in turn, allows you to use two functions in one device - video shooting and photo shooting. Due to the fact that the rearrangement of pixels using VPS technology in Foveon X3 matrices is instantaneous, it became possible to take instant photos directly in video mode without losing the quality of work of both modes.

Foveon X3 Work with "windows"

Working with “windows” in Foveon X3 is that any rectangular area on the matrix is ​​available for creating a new image, since The technology allows direct access to any rectangular area of ​​the array of pixels.



CCD sensors, this technology is impossible. because Initially, CCD technology does not support access to individual pixels or groups of pixels. At best, they offer a limited fixed set of possible options for trimming the finished image. With Foveon X3 , digital camera manufacturers are not limited by anything, which allows you to use your camera to get a new image by cutting out any rectangular area from a finished photo and saving it in a separate file. This allows you to give much more freedom to the end user when working with the camera with the Foveon X3 matrix.

The main feature of the Foveon X3 - when resizing images - the lack of artifacts, because each pixel stores full color information, unlike CCD matrices, in which one pixel can store information only about one color.

Thanks to VPS technology, metering can now be carried out more accurately in any light conditions, since the matrix at the time of the exposure metering enters the mode of increasing pixels, which leads to a significant increase in sensitivity. The same function allows for faster focusing, single-zone exposure metering and many other things not available to cameras with conventional CCD matrices.

X3 Fill Light - light correction function.

The first photo is the original image, the second is the image after X3 Fill Light lighting correction.

Essence - the new technology X3 Fill Light automatically corrects the lighting problems in digital photos.

If the photo was underexposed (for some reason), then the X3 Fill Light feature will help to correct the situation. The X3 Fill Light software works on the principle of modulating the natural conditions for normal exposure. Ie, artificially lightened components are added while maintaining the same ratios of brightness and contrast of the main image array.
The X3 Fill Light technology avoids irregularly bleached areas by thoroughly adjusting the parameters of each pixel relative to other pixels surrounding it. For imaging, the Foveon X3 processor generates an X3F file, which is then processed by the X3 Fill Light software on a personal computer.

As a result, if the photo was taken mistakenly at a slow shutter speed, or due to bright background lighting it turned out darker than desired - there is always the opportunity to correct the error using the X3 Fill Light function and continue to enjoy high-quality images.

Thanks for attention!