Image Enhancement with Neural Network
Today, I want to talk about an interesting approach to improving image quality. The official name of the Super Resolution approach. Image quality improvement by software methods has been known since the beginning of the emergence of digital images, but in the last 3 years there has been a qualitative leap caused by the use of neural networks.
An example of improving image quality using Super Resolution technology.
There are four basic approaches to image improvement: prediction models (predictive models), edge based methods, image statistical methods, and patch based (or example-based) methods (pattern-based methods). The best quality is provided by patch based (or example-based) methods (methods based on patterns).
In article [1] for the first time it was proposed to move from the traditional approach, using a set of filters to a single convolutional neural network that works on the principle of end to end.
')
A convolutional neural network approach combines functions:
1) Extract and display patches.
2) Nonlinear mapping.
3) Reconstruction.
Visualization of the principle of the neural network and communication with the method of sparse coding [1]
The approach based on the convolutional neural network has improved the quality in comparison with the known methods while maintaining a high response rate.
Quality and response time depending on the number of filters.
According to the authors of the study, the network architecture influenced the quality more than the size of the training sample.
A continuation of research in the field of image quality improvement is the work [2] in which deeper architectures were considered with the help of which it was possible to restore the “corrupted image”.
On the left is the “damaged image”, the center is the reconstructed image, on the right is the original.
In [3], the problem of image enhancement is solved using the GAN.
Visualization of the principle of GAN in the task of improving image quality [3]
The use of GAN made it possible to improve the quality of the texture and made the processed images so photorealistic that it is difficult to visually distinguish them from the original.
Comparison of the image restored using the GAN and the original.
Super Resolution technology is already used in image and video processing. For example, Yandex has improved the quality of old Soviet films in cinematography, and Robin Video uses Super Resolution to improve the quality of the image in the cloud video surveillance, so that customers use simpler and cheaper cameras while preserving the quality of the picture.
In the following articles I will try to tell you in detail about the technical implementation of the Super Resolution approach.
An example of improving image quality using Super Resolution technology.
There are four basic approaches to image improvement: prediction models (predictive models), edge based methods, image statistical methods, and patch based (or example-based) methods (pattern-based methods). The best quality is provided by patch based (or example-based) methods (methods based on patterns).
In article [1] for the first time it was proposed to move from the traditional approach, using a set of filters to a single convolutional neural network that works on the principle of end to end.
')
A convolutional neural network approach combines functions:
1) Extract and display patches.
2) Nonlinear mapping.
3) Reconstruction.
Visualization of the principle of the neural network and communication with the method of sparse coding [1]
The approach based on the convolutional neural network has improved the quality in comparison with the known methods while maintaining a high response rate.
Quality and response time depending on the number of filters.
According to the authors of the study, the network architecture influenced the quality more than the size of the training sample.
A continuation of research in the field of image quality improvement is the work [2] in which deeper architectures were considered with the help of which it was possible to restore the “corrupted image”.
On the left is the “damaged image”, the center is the reconstructed image, on the right is the original.
In [3], the problem of image enhancement is solved using the GAN.
Visualization of the principle of GAN in the task of improving image quality [3]
The use of GAN made it possible to improve the quality of the texture and made the processed images so photorealistic that it is difficult to visually distinguish them from the original.
Comparison of the image restored using the GAN and the original.
Super Resolution technology is already used in image and video processing. For example, Yandex has improved the quality of old Soviet films in cinematography, and Robin Video uses Super Resolution to improve the quality of the image in the cloud video surveillance, so that customers use simpler and cheaper cameras while preserving the quality of the picture.
In the following articles I will try to tell you in detail about the technical implementation of the Super Resolution approach.
Links to articles
[1] arxiv.org/pdf/1501.00092v3.pdf
Image Super-Resolution Using Deep Convolutional Networks
[2] arxiv.org/pdf/1606.08921.pdf
Image Restoration Using Convolutional Auto-encoders with Symmetric Skip Connections
[3] arxiv.org/pdf/1609.04802.pdf
Photo-Realistic Super Image Resolution Using Generation Adversarial Network
Image Super-Resolution Using Deep Convolutional Networks
[2] arxiv.org/pdf/1606.08921.pdf
Image Restoration Using Convolutional Auto-encoders with Symmetric Skip Connections
[3] arxiv.org/pdf/1609.04802.pdf
Photo-Realistic Super Image Resolution Using Generation Adversarial Network
Source: https://habr.com/ru/post/359016/
All Articles