Coloring images

Hello, Habrahabr. Today we will color.
What will be here? It will search for a color image with similar colors in black and white and the method of transferring colors from first to second.

Color Image Search

What do we have? Suppose we have a bunch of color images and one black and white, we want to choose from this heap one that best suits our black and white colors (if they had colors, he would have). Each image in the election will represent its signature.

The signature is 128 floating-point numbers, they are obtained from the histogram of the l (brightness) image (from the lab color model) normalized to one, that is, the sum of all 128 numbers is equal to 1 (normalization is needed to remove the image size effect on signature). Schematically getting the image signature looks like this:


A similar approach is described in [2], there are other options for signatures and experimental results. Now every color image has a signature, and black and white also has it, how do we choose an image? We calculate the correlation between the signature of our colorless experimental and color candidates, and we get the winner by the largest value. Correlation, by the way, is calculated like this:
')



Here H1 and H2 are signatures, N = 128 is the number of signature elements. If d = 1, then the maximum match, -1 is the maximum difference, 0 means no correlation. Thanks for the note of Assorium , the deductible is really equal to 1/128 in this case.
At this stage, we already have a black and white image and an image of the source of color, let's call it that. In the first picture, these are left and right images, it remains to get the central one.

Transfer color from color to black and white

The stages of color transfer are as follows:
1. We superimpose a 15x15 grid on the source color image, in each cell we take a random pixel (this is called jittered sampling )
2. Consider a neighborhood of 25x25 pixels, calculate the expectation and variance (working with channel l).
After the first two steps, we receive 225 color samples from the color image, each of which is characterized by the following: two values ​​of channels a and b; mat. waiting and dispersion.
15x15 and 25x25 were obtained during the experiments.
3. Pixel bypass our black and white image, calculate the variance and mat. wait, compare with samples, transfer the values ​​of color channels a and b from the most appropriate sample.

This method came up with Welsh with his friends, you can read more with him [1].

As a result of these simple manipulations, quite colorful pictures are obtained:



Well, as without a red cat:

findings

Well, the advantages here are obvious, we do nothing with our hands, everything happens beautifully, and what’s the disadvantages? For images where the main tones are not two or three, but many more, the results can be very inadequate, this is eliminated very simply in words and very difficult in practice: using recognition, for example, using the SURF method and contour analysis to search for a machine instead:



get a much more believable picture:



but that's another story.

PS

I will be extremely grateful for alternative approaches to coloring in the comments, at least at the level of ideas.

Used sources:
1 Welsh, T., Transferring color to greyscale images
2 Vieira, LFM at al. Fully automatic coloring of grayscale images