Neural network predicts the first impression of a person on his face

The result of processing photos of Julian Assange and Benedict Cumberbatch by the neural network to predict the first impression of a person. There is a slight lack of subjective intelligence and dominance of the actor compared with the hero he played in the film “The Fifth Power”. The rest of the image Assange passed quite correctly. Illustration: University of Notre Dame, Harvard University

Our brain is able to quickly form an impression of a stranger on his face. For a few seconds, the key characteristics of a person are determined: his attractiveness, intelligence, age, reliability, social status (dominance), sociability and level of morality. It is clear that this impression is inaccurate, and sometimes completely wrong. Absolute accuracy is not required here. Speed ​​is important. This is an absolutely necessary social mechanism, without which it is difficult to survive in society, a vital skill. Therefore, the first impression is so strong and important. Change it then very difficult.

Leaving behind the obvious lack of objectivity of the first impression, it is important to teach computers to evaluate faces in the same way that other people do. Such attempts have been repeatedly made. Researchers tried to algorithmically categorize various emotions across a person’s face, including using neural networks . Published work on the automatic determination of the attractiveness of the person , as well as the presence of a sense of humor and other personal characteristics of a person using neural networks.

All these studies are an absolutely necessary part of the work on creating artificial intelligence at the human level. The AI ​​must be able to read faces no worse than a person, and to make subjective evaluations in the face no worse than a person.
')
Now a group of researchers from the University of Notre Dame (USA) and Harvard University (USA) has conducted a new study that will complement the scientific developments in the field of social psychology. Their work is devoted to the automatic compilation of the first impression that arises about a person among others. The analysis is performed by a neural network trained on sample photos.

According to psychologists who specialize in studying the system of social assessments in people, the first assessment of a stranger in the face is two characteristics: reliability and dominance. According to Alexander Todorov, all other characteristics can be derived from these two main ones. Psychologists have long been a set of basic characteristics that demonstrate the dominance of a person over others, including the inclination of the head, gestures of eyebrows and mouth.

Scientists from the University of Notre Dame and Harvard University have trained the neural network on a sample of 6300 photos from the publicly available Annotated Facial Landmarks in the Wild . These photos were annotated by users of the crowdsourcing platform TestMyBrain.org on three social attributes: dominance, reliability and IQ, as well as an additional attribute - age.

6000 photographs were used for training, another 200 - for fine-tuning and fine-tuning, and the last 100 - for testing the neural network.


Almost perfect match profiles of Edward Snowden and playing his actor Joseph Gordon-Levitt in the film "Snowden". Illustration: University of Notre Dame, Harvard University

Scientists note that the attributes IQ and age, they could make annotations with real, not subjective characteristics. But then the neural network would learn to determine the real IQ and age of a person by his face. As mentioned above, the task was completely different, therefore, for learning neural networks of the category IQ and age were also assigned using crowdsourcing.

The illustration shows individual photos from the base that was used for training. Samples are selected among holders of minimum, medium and maximum characteristics in all categories. As it is easy to see, young children have the lowest scores in the categories of IQ, dominance and age. A person with an ornate face is one of the minimum marks for reliability.



The distribution of indicators in the training dataset is close to the normal distribution.



The table shows the average ratings of photos in four categories, based on user ratings. The minimum and maximum marks are indicated, as well as the average number of marks given. Each person received, on average, 32 ratings for dominance and reliability, as well as 15 ratings for age and IQ.



After training, the neural network showed a fairly accurate assessment of individuals close to the results of the assessment of users of the site.



The authors hope that their neural network will be useful in the development of computer vision programs. Such a system is able to analyze the video stream in real time. For example, to show on the screen in the form of a graph, how the subjectively perceived intelligence of the person of the host of a television program changes over time. That is, how clever his face looks at certain points in time. Imagine that the program will compile a digest of the most interesting moments of the transfer, when the presenter’s face looked most stupid.



During video processing, the neural network processes each frame separately.

You can think of other applications of this technology. For example, to select the actors who most accurately convey the subjective impression of the appearance of their characters in real life, as shown in the previously mentioned examples with Julian Assange and Edward Snowden.

Probably, such a neural network will find application in personal growth programs: by changing the facial expression in front of the camera, a person will practice how to become more solid and intelligent in the eyes of others - by observing the assessment of his face by the neural network.

One can imagine that some firms will start recruiting employees, taking into account the results of the subjective assessment of the candidates' faces. That is, when training a neural network, the boss chooses several dozen people he likes according to IQ and reliability characteristics, and the program finds candidates with exactly those characteristics that the boss needs.

Researchers only warn that the accuracy of the neural network directly depends on the quality of the source data, that is, the assessment of individuals. It is important that this assessment is carried out by representatives of the target audience. The fact is that the impression about a person strongly depends on the personality of the person who assesses: his culture and nationality. Roughly speaking, a very authoritative and wise person in the Zulu tribe will not be perceived as the dominant leader in the community of Oxford aristocrats.

The scientific work was published on October 25, 2016 on the site arXiv.org .