Face recognition

“My light, mirror! tell me
Yes, tell the whole truth:
I am the sweetest in the world,
All blush and whiter? "

A.S. Pushkin

Little by little magic things from fairy tales are realized in real reality through the use of new technologies and scientific discoveries. Such devices as carpet-plane (aviation), boots-walkers (cars), an apple on a saucer (netbook with internet), a ball that shows the way (GPS-navigator) and other necessary things are already implemented and actively used. We tried to implement the system for assessing the beauty of a person’s face mentioned in the “tale of a dead princess and seven heroes” using artificial intelligence and machine vision, because we believe that the author of the epigraph actually meant a tablet with a front-facing camera and special installed software.

The question of what exactly makes a person's face attractive is the subject of research by physiologists, biologists, philosophers, art historians, and specialists in plastic surgery for a long time. At present, it is considered established that people, in addition to individual preferences, are also affected by the general biologically-motivated principles of beauty assessment [1-2]. Among the possible candidates for typical signs, physiologists distinguish the symmetry of facial features [3], the difference between a facial image and averaged image of a large number of people [4], the correspondence of facial proportions to the “golden section” [5], etc. For example, [4] shows that, on the one hand, symmetrical facial features correspond to genes less susceptible to mutations and therefore people with such facial features are more resistant to mutations and diseases, and on the other, people with more symmetrical facial features receive higher beauty ratings when evaluating them otografy experts.
')
In recent years, several pioneering works have appeared on computerized beauty recognition systems based on the use of machine vision systems [6–8] and trainee classifiers. These works can be considered as an attempt to endow robotic systems with the ability to “see beautiful”. In [6], the proportions of facial features are used as signs, with the key points on the face being highlighted manually. In [7], in addition to the proportions, the method of principal components was applied to extract features. In [8], deep neural networks were used for the beauty recognition task.

We have developed an automatic beauty assessment system based on the method of extracting key points on the face using the tools of the OpenCV machine vision library and a neural network trained on the expert assessment data and conducted an experimental evaluation of the quality of its work.

Image database for training

We have collected our own image database, consisting of 180 photos of young women, the images were taken from open sources. Were selected photographs of faces in the frontal projection with a neutral facial expression, without glasses and jewelry. To make the sample representative, we tried to include in the database examples of both beautiful and ugly faces (Fig. 1).



Fig. 1. An example of a photo of persons from the database of images

Unlike the work [7], the collected base includes photographs of women of different races, skin color, and their age ranges from 18 to 35 years. After the images were collected, the group of experts was asked to set subjective ratings of aesthetic appeal for each of the photos on a scale from 1 to 7. In total, 8 experts, 4 men and 4 women aged 16 to 63 were involved in the labeling of photos. grades were set independently. According to the conditions of the experiment, before the beginning of the scoring process, each expert was presented with all the photos for initial review. To check the consistency of the sample was carried out correlation analysis, its results are presented in Table. one.

Table 1. Pairwise correlations of assessments of various experts



The average correlation of the sample was 0.7, which makes it possible for a neural network to be trained on such data and roughly corresponds to the results of other researchers [4.7].

The general scheme of the algorithm

The beauty recognition system receives as input an image containing a frontal photo of a person’s face (Fig. 2).



Fig. 2. Scheme of the face beauty recognition algorithm

Before the start of the algorithm, we assume that the face in the image has already been allocated earlier and occupies most of the image area. Next, using the Viola-Jones standard boosting classifier, which is part of the OpenCV computer vision library, the areas on the face corresponding to the right and left eye, nose and mouth are highlighted.
On the basis of these coordinates, the basic proportions of the face are calculated, which are then used as a feature vector for the neural network. The neural network is first trained on this input using expert evaluations as a target sample, and then it can be used to recognize new data that has not been previously seen by the network.

Feature highlighting

We conditionally divided the signs we distinguished into two groups: the ratio of the distances between the selected key points and the ratios of the found face sizes.

The group of signs 1 is shown in fig. 3, left: AB / CD, AC / BC, AD / BD, EC / ED, EC / AB, AC / AD, BC / BD. The group of features 2 is shown in Fig. 3, right: L / R, Mw / Mh, Nw / Nh, Mw / Nw, Mh / Nh. The final feature vector consists of the combined features of both groups. Before submission to the neural network, the data were brought to the range [1; 1].



Fig. 3. Calculation of feature vectors for selected key points on the face

Training neural networks

As a trained neural network, we used the standard multilayer perceptron [10, p. 219] with one hidden layer containing 5 neurons in the hidden layer. The hyperbolic tangent function was used as the activation functions of the neurons of the hidden and output layers. The neural network was trained using the advanced Kalman filter [10, p. 219], [11], which is currently one of the most effective second-order learning methods for neural networks. Before training, the sample was divided into 2 parts: the training (110 examples, 60% of the sample) and the examination (70 examples, 40% of the sample). The learning outcomes are presented in Table. 2

Table 2. Neural network learning results on the beauty recognition task



We believe that the obtained correlation result of 0.5 on the exam sample that was not used in training is very good for the small amount of information supplied to the neural network as features. In fact, the neural network makes a decision based on the analysis of the structure of the bones of the skull, ignoring other data that a person takes into account when solving a similar task.
In the future, we plan to improve the algorithm by expanding the base of images for training, highlighting new key points on the face and incorporating a smooth skin detector into it.

Original article (ours): Chernodub AN, Pashchenko Yu.A., Golovchenko K.A. Neural network system for determining the attractiveness of the human face // XV All-Russian Scientific and Technical Conference "Neuroinformatics-2013", Moscow, January 21-25, 2013, p. 254 - 259.

Bibliography

1. Kovach, FJ Philosophy of beauty // Norman: University of Oklahoma Press. 1974.
2. Grammer K, Thornhill R. Human (Homo sapiens) // J Comp Psychol, 1994. V. 108. No. 3. P. 233-242.
3. Rhodes G. The Evolutionary Psychology of Facial Beauty // Annu. Rev. Psychol. 2006. V. 57. P. 199-226.
4. Sheib JE, Gangestad SW, Thornhill R. Facial attractiveness, symmetry and cues of good genes // Proc Biol Sci. 1999 September 22; 266 (1431). R. 1913-1917.
5. Holland E. Marquardt's Phi Mask: A Beautiful Face // Aesthetic Plastic Surgery, 2008. V. 32, No. 2. P. 200-208.
6. Aarabi, P., Hughes, D., Mohajer, K., Emami, M. The IEEE International Conference on Systems, Man, and Cybernetics, 710 October 2001, Tucson, USA. V. 4. P. 2644-2647.
7. Eisenthal Y., Dror G., Ruppin E. Facial Attractiveness: Beauty and the Machine // Neural Computation, 2006. V. 18. No. 1. P. 119-142.
8. Gan J., Li L., Zhai Y. Deep self-taught by facial beauty prediction // Neurocomputing. DOI: 10.1016 / j.neucom.2014.05.028
9. Gray D., Yu. K., Xu W., Gong Y. Predicting Facial Beauty without Landmarks // Computer Vision - ECCV 2010, Lecture Notes in Computer Science, 2010, V. 6316/2010. P. 434-447.
10. Khaikin S. Neural networks: a full course. M .: Williams, 2006.
11. Calman Filter: A comparative study // Optical Memory and Neural Networks, 2014. Vol. 23, Issue 2, pp 96-103.