Restore defocused and blurred images. Increase quality
I present to your attention the final article from the trilogy "Restoring defocused and blurred images." The first two caused a noticeable interest - the area is really interesting. In this part, I will look at the family of methods that give the best quality compared to the standard Wiener filter - these are methods based on Total Variaton prior.
Also, by tradition, I posted a new version of SmartDeblur (along with the sources in open-source) in which I implemented this method. The final quality was obtained at the level of commercial counterparts of the Topaz InFocus type. Here is an example of processing a real image with a very large blur:
I will not describe the basic theory of deconvolution here, it was written in great detail about it in previous articles. Those who have not read or forgotten them, I recommend to get acquainted with them first, in order to understand the terminology and classical approaches:
Part 1. Theory ;
Part 2. Practice .
')
Before proceeding to the description of Total Variation (hereinafter TV prior), it is necessary to understand what are the disadvantages of algorithms such as the classic Wiener filter? The most basic ones are the ringing effect (a periodic halo at the edges of objects) even with a small noise level, blurring of borders and small details, as well as poor noise reduction from the point of view of human perception. All this greatly hinders the practical use of the Wiener filter, limiting its application to the tasks of technical image restoration, for example, for reading inscriptions of interest.
Therefore, in recent times a large number of various methods have been developed, the purpose of which is to improve the visual quality. It should be noted that the number of parts at the same time, as a rule, does not increase.
The main quality of Total Variation prior in terms of result is the preservation of sharp edges and the smoothing of artifacts of deconvolution. It is written as follows:
Unfortunately, the calculation of this functional cannot be done in a simple way, since it requires the use of very complex optimization techniques.
Alternatively, you can use the smoothed functionality instead of the absolute value:
When epsilon tends to zero, the result tends to the original definition of Total Variation, but the optimization process becomes more complex. Conversely, with sufficient large epsilon, the optimization result will resemble a Wiener filter with blurring of the edges. Unfortunately, the above formula has a non-quadratic form, so it cannot be simply calculated in Fourier frequency space, as was the case with Wiener and Tikhonov filters. Therefore, one of the step-by-step optimization methods is needed to find an approximate solution — for example, the classical gradient descent method:
Where tau is calculated by the following formula:
And the gradient of the smoothed functional is defined as:
The number of iterations should be large enough - a few hundred.
This is the most basic approach in implementing TV prior, which is called head-on. Nevertheless, even he gives very good results. On the basis of it in scientific publications, many studies have appeared that are trying to further improve the quality and also reduce the calculation time.
The formulas described are, in principle, uncomplicated, although very cumbersome to implement. The main problem is to achieve high speed, because The number of iterations is very large and each iteration contains many complex actions. Namely - a few convolutions of the entire image, the calculation of the full gradient and divergence.
I’ll say right away that I haven’t yet managed to achieve a good work rate; on the image of several megapixels, the final calculation time is 2-3 minutes. But Preview is fast - about 0.2 seconds.
The build for Windows can be downloaded at:
github.com/downloads/Y-Vladimir/SmartDeblur/SmartDeblur-1.27-win.zip
Sources (licensed under GPL v3) are available at: github.com/Y-Vladimir/SmartDeblur
The main changes from the previous version, which was described in the second part:
C ++ using Qt.
Well, now the most important thing is what quality you can expect when processing blurred images. We will compare it with Topaz commercial Topaz InFocus. The remaining analogues (such as FocusMagic) have not been supported for a long time, or they give completely unacceptable processing results. So let's go.
First, take an example of advertising from the Topaz InFocus website: www.topazlabs.com/infocus/_images/licenseplate_compare.jpg
Here is the result from Topaz InFocus:
But the result of SmartDeblur with the following parameters:
Type: Motion Blur, Length: 10.1, Angle: -45, Smooth: 60%
As you can see, the results are very similar. And not so obvious that better. Apparently, Topaz InFocus also uses an algorithm similar to TV prior plus post-processing in the form of sharping edges. It should be noted that the given source blurred image, with a very high probability, is synthetic. Those. an undistorted image is taken and a motion blur filter is applied. This can be seen in almost perfect recovery, as well as in suspiciously whole distortion parameters - an angle of 45 degrees and a length of 10 pixels.
Now let's take a real image, which yesterday I took on my Canon 500D with a manual focus shift:
Result from Topaz InFocus with the following parameters:
Type: Out-of-Focus, Radius: 5.5, Suppress Artifacts: 0.34
SmartDeblur result with the following parameters:
Type: Out of Focus, Radius: 5.9, Smooth: 60%
There is a draw, you can say. The parameters in each program were selected to ensure the best quality.
Another real example I shot:
SmartDeblur result with the following parameters:
Type: Motion Blur, Length: 6.6, Angle: -37, Smooth: 53%
The third final article has come to an end. It turned out not very big, but I hope it will be useful. As you can see, the quality of processing obtained is quite acceptable for real use. The main problem that remains is in places where there are light objects, after processing, a noticeable ringing effect is obtained. I think this is due to the fact that in the light areas the linearity of the display of the brightness of the pixels is disturbed, which gives an incorrect interpretation of its real brightness. Perhaps you need a logarithmic pre-processing of brightness, or something else.
Once again I will remind:
The build for Windows can be downloaded at:
github.com/downloads/Y-Vladimir/SmartDeblur/SmartDeblur-1.27-win.zip
Sources (licensed under GPL v3) are available at: github.com/Y-Vladimir/SmartDeblur
And as usual - I will be very happy with comments and suggestions on SmartDeblur!
Who will try the program - please note that the Smooth quality parameter in the preview mode and in the High-Quality mode behaves very differently. Therefore, the final result of the smoothing slider can be evaluated only after the completion of High-Quality rendering.
PS A huge request to everyone who writes me a mail. After the publication of the two previous articles, I received (and continues to receive) a large number of letters with a request to restore the machine numbers in the frames from surveillance cameras, when the entire number occupies an area of a few pixels.
I do not do that! SmartDeblur does not know how to do this either. This is a completely different task, namely Super-Resolution, when from several low-resolution images you get a high-resolution image with new details that were not on the original data. Maybe someday she'll do it, but definitely not in the near future.
UPDATE Continuation Link:
Blind Deconvolution - automatic recovery of blurred images
Also, by tradition, I posted a new version of SmartDeblur (along with the sources in open-source) in which I implemented this method. The final quality was obtained at the level of commercial counterparts of the Topaz InFocus type. Here is an example of processing a real image with a very large blur:
Introduction
I will not describe the basic theory of deconvolution here, it was written in great detail about it in previous articles. Those who have not read or forgotten them, I recommend to get acquainted with them first, in order to understand the terminology and classical approaches:
Part 1. Theory ;
Part 2. Practice .
')
Before proceeding to the description of Total Variation (hereinafter TV prior), it is necessary to understand what are the disadvantages of algorithms such as the classic Wiener filter? The most basic ones are the ringing effect (a periodic halo at the edges of objects) even with a small noise level, blurring of borders and small details, as well as poor noise reduction from the point of view of human perception. All this greatly hinders the practical use of the Wiener filter, limiting its application to the tasks of technical image restoration, for example, for reading inscriptions of interest.
Therefore, in recent times a large number of various methods have been developed, the purpose of which is to improve the visual quality. It should be noted that the number of parts at the same time, as a rule, does not increase.
Description TV prior
The main quality of Total Variation prior in terms of result is the preservation of sharp edges and the smoothing of artifacts of deconvolution. It is written as follows:
Unfortunately, the calculation of this functional cannot be done in a simple way, since it requires the use of very complex optimization techniques.
Alternatively, you can use the smoothed functionality instead of the absolute value:
When epsilon tends to zero, the result tends to the original definition of Total Variation, but the optimization process becomes more complex. Conversely, with sufficient large epsilon, the optimization result will resemble a Wiener filter with blurring of the edges. Unfortunately, the above formula has a non-quadratic form, so it cannot be simply calculated in Fourier frequency space, as was the case with Wiener and Tikhonov filters. Therefore, one of the step-by-step optimization methods is needed to find an approximate solution — for example, the classical gradient descent method:
Where tau is calculated by the following formula:
And the gradient of the smoothed functional is defined as:
The number of iterations should be large enough - a few hundred.
This is the most basic approach in implementing TV prior, which is called head-on. Nevertheless, even he gives very good results. On the basis of it in scientific publications, many studies have appeared that are trying to further improve the quality and also reduce the calculation time.
Practical implementation
The formulas described are, in principle, uncomplicated, although very cumbersome to implement. The main problem is to achieve high speed, because The number of iterations is very large and each iteration contains many complex actions. Namely - a few convolutions of the entire image, the calculation of the full gradient and divergence.
I’ll say right away that I haven’t yet managed to achieve a good work rate; on the image of several megapixels, the final calculation time is 2-3 minutes. But Preview is fast - about 0.2 seconds.
The build for Windows can be downloaded at:
github.com/downloads/Y-Vladimir/SmartDeblur/SmartDeblur-1.27-win.zip
Sources (licensed under GPL v3) are available at: github.com/Y-Vladimir/SmartDeblur
The main changes from the previous version, which was described in the second part:
- Two methods of deconvolution are added: TV prior and Tikhonov filtering
- Added support for restoring Gaussian blur
- Improved speed (about 2.5 times)
- Reduced memory consumption (about 1.5 times)
- The maximum size of the image processed by default is 3000 (but can be changed in the settings)
- Added settings section
- Added Updates Checker
- Drag & Drop support
- Added Help Screen with sample image and customization tips.
- Fixed a bug with ripples in preview mode
C ++ using Qt.
Comparison
Well, now the most important thing is what quality you can expect when processing blurred images. We will compare it with Topaz commercial Topaz InFocus. The remaining analogues (such as FocusMagic) have not been supported for a long time, or they give completely unacceptable processing results. So let's go.
First, take an example of advertising from the Topaz InFocus website: www.topazlabs.com/infocus/_images/licenseplate_compare.jpg
Here is the result from Topaz InFocus:
But the result of SmartDeblur with the following parameters:
Type: Motion Blur, Length: 10.1, Angle: -45, Smooth: 60%
As you can see, the results are very similar. And not so obvious that better. Apparently, Topaz InFocus also uses an algorithm similar to TV prior plus post-processing in the form of sharping edges. It should be noted that the given source blurred image, with a very high probability, is synthetic. Those. an undistorted image is taken and a motion blur filter is applied. This can be seen in almost perfect recovery, as well as in suspiciously whole distortion parameters - an angle of 45 degrees and a length of 10 pixels.
Now let's take a real image, which yesterday I took on my Canon 500D with a manual focus shift:
Result from Topaz InFocus with the following parameters:
Type: Out-of-Focus, Radius: 5.5, Suppress Artifacts: 0.34
SmartDeblur result with the following parameters:
Type: Out of Focus, Radius: 5.9, Smooth: 60%
There is a draw, you can say. The parameters in each program were selected to ensure the best quality.
Another real example I shot:
SmartDeblur result with the following parameters:
Type: Motion Blur, Length: 6.6, Angle: -37, Smooth: 53%
findings
The third final article has come to an end. It turned out not very big, but I hope it will be useful. As you can see, the quality of processing obtained is quite acceptable for real use. The main problem that remains is in places where there are light objects, after processing, a noticeable ringing effect is obtained. I think this is due to the fact that in the light areas the linearity of the display of the brightness of the pixels is disturbed, which gives an incorrect interpretation of its real brightness. Perhaps you need a logarithmic pre-processing of brightness, or something else.
Once again I will remind:
The build for Windows can be downloaded at:
github.com/downloads/Y-Vladimir/SmartDeblur/SmartDeblur-1.27-win.zip
Sources (licensed under GPL v3) are available at: github.com/Y-Vladimir/SmartDeblur
And as usual - I will be very happy with comments and suggestions on SmartDeblur!
Who will try the program - please note that the Smooth quality parameter in the preview mode and in the High-Quality mode behaves very differently. Therefore, the final result of the smoothing slider can be evaluated only after the completion of High-Quality rendering.
PS A huge request to everyone who writes me a mail. After the publication of the two previous articles, I received (and continues to receive) a large number of letters with a request to restore the machine numbers in the frames from surveillance cameras, when the entire number occupies an area of a few pixels.
I do not do that! SmartDeblur does not know how to do this either. This is a completely different task, namely Super-Resolution, when from several low-resolution images you get a high-resolution image with new details that were not on the original data. Maybe someday she'll do it, but definitely not in the near future.
UPDATE Continuation Link:
Blind Deconvolution - automatic recovery of blurred images
-Vladimir Yuzhikov (Vladimir Yuzhikov)
Source: https://habr.com/ru/post/152885/
All Articles