Neural network learned how to grow old and rejuvenate people by photo

Original photo (a), initial reconstruction (b), two options for optimizing reconstruction (c) and the result of aging / rejuvenation of persons with the second option of IP optimization, that is, with the best preservation of face recognition (d)

Facial aging (age synthesis) with photo morphing is an important task that has many practical applications. This aging should be done for the correct operation of facial recognition systems. It is necessary when searching for missing children years or decades after the disappearance. And of course, face morphing is used in the entertainment industry - for example, in cinema. Probably, mobile applications with such a function may become popular. It is interesting for everyone to see how this old teacher looked like when he was young or what your beautiful fellow classmate would be like in 40-50 years.

Traditionally, face aging programs are based on one of two approaches: prototyping and modeling . In the first case, the estimate of the average person in each age group is calculated, as well as the difference between the average persons in different age groups. This difference is then used to morph a specific face. Prototyping is a simple and quick approach, but it completely ignores the individual traits of a person. Therefore, the images are not very realistic.

In contrast, modeling involves the construction of a parametric model that simulates the mechanisms of aging on the muscles, skin, and skull of a specific individual. This approach also has a flaw. To create a good model, you often need photos of the same person at different ages. It is not always possible to obtain such data.
')
Both of these approaches do not cope well with the additional modifications that occur with an aging face. For example, the appearance of a beard, bald head or glasses.

Methods of natural image generation have been studied for many decades, but they received a significant boost after 2014, as a result of serious progress in depth learning. Neural networks have learned to generate very high quality images and resolution. The key technology in this area is generative adversarial neural networks (Generative Adversarial Network, GAN), which have been repeatedly described on Habré and GT . The GAN network teaches the loss function, the purpose of which is to classify an image as “real” or “fake”. Simultaneously, the generative model is being trained to minimize this function.

Unlike autoencoders, GAN cannot generate a blurry image because it will not pass the classification test as “real.” GAN is a kind of “war” of two neural networks, a generator and a discriminator, where the first one tries to deceive the second one to generate a photo that will come off the real one at the discriminator.

Generative competitive neural networks have shown themselves to be excellent in different programs for changing faces — for example, in imposing hairstyles and changing hair color or simply changing a person’s age without clear age criteria. But in most cases, these programs had a problem that, along with the aging on the face, the characteristic individual features inherent in a particular person disappeared. This problem was tried to be solved by the staff of the research team under the leadership of Grigory Antipov from the company Orange Labs (France). Their scientific article was published on February 7, 2017 on the preprint website arXiv.org (arXiv: 1702.01983).

Antipov and his colleagues developed the acGAN (Age Conditional Generative Adversarial Network) generative consensual neural network, trained to grow old or rejuvenate a person while maintaining recognition . This is the first neural network that generates high-quality individuals in any given age category. In addition, the neural network is able to restore the original photo while maintaining human awareness.

The authors write that after learning the neural network aging occurs in two steps. At the first step, the optimal eigenvector for the reconstruction is calculated. At the second stage, the image generator is turned on, which simply creates a photo of the specified age taking into account its own vector.


The algorithm of the generative concurrent neural network acGAN

For the conditions of the acGAN generator, six age categories have been established: 18 years, 19-29, 30-39, 40-49, 50-59 and over 60 years. The neural network was trained on a set of IMDB-Wiki photos cleaned, containing at least 5000 images in each age category. Thus, acGAN conditions are six-dimensional direct vectors.

In total, the set for training was 120 thousand photos. Of these, 110 thousand were used for training, and the remaining 10 thousand were used to test the operation of the neural network using two alternative methods for calculating the optimal eigenvectors Pixelwise and Identity-Preserving .


Examples of generating synthetic images in six age categories using two random optimal eigenvectors

A comparison of the results of the work of the Pixelwise and Identity-Preserving methods showed that the second of these methods shows the most optimal results. Evaluation of face recognition was carried out using OpenFace software - one of the best open source facial recognition software.

The authors of the scientific work believe that the innovative Identity-Preserving method is suitable not only for aging people, but also for other face transformations with preservation of recognition. For example, for the imposition of beards, glasses and other transformations. This technology can be used in various face recognition systems and entertainment applications.