Video analytics combines: what the brain and machines do with our faces

The skill to see and quickly recognize faces is a super ability. No need to spend time on the analysis, to study wrinkles, folds and ovals. Face recognition occurs instantly and effortlessly. It is so easy that we do not realize how we do it.

Think about how different people look like each other - two eyes, mouth, nose, ears sticking out on each side, each time in the same order (most often). It is incredible that we analyze the object with such ease.
')
We are “programmed” to recognize faces from birth, but now people have achieved more - they taught the machine this skill. How will the widespread introduction of person recognition and identification systems affect society?

Pareidolia: automatic face search

People in the "automatic" mode can distinguish familiar images on any surfaces. Only three architectural elements of the building are perceived as the face of a surprised duck. This is an example of paradolia.

The word pareidolia comes from the Greek words para (para - near, near, deviation from something) and eidolon - image. This is the name of optical illusion, image perception or meaning, where they are not really present. For example, a face on a tree trunk or animal figures in the clouds is a paradeolia.


More of these photos can be found on thingswithfaces.com

The faces of people and faces of animals we see in any geometric figure. The whole Emoji culture is built on this principle. :-)



The phenomenon of pareidolia is easy to translate into the language of algorithms. Artists Shinseungback Kimyonghun photographed clouds, for a moment merging into human faces, using a script with the library OpenCV.

Thatcher's illusion: systemic biological errors

There is a biological bug that shows the greater importance of the recognition skill . Most of the objects around you - a chair, a table, a computer, are easy to see and correctly identify from any angle. Just not the face.

An inverted face produces a malfunction in the brain, called the Thatcher effect (illusion). A phenomenon describes a state in which it is difficult to detect local changes in an inverted portrait photo.


The first photo seems normal, but if you turn it over, the wrong position of the eyes and mouth immediately catches the eye. Man and artificial neural network perceive images in different ways. It is surprising that the “neural network” between our ears is so easy to deceive.

Thatcher’s illusion demonstrates some of the basic mechanisms by which our brain processes information. The brain reads a collection of individual elements: a pair of eyes, nose, mouth, ears. In addition to the individual characteristics of facial features, their relationship with each other and location are taken into account. That is, the person is perceived as a whole system.

Therefore, when we are shown an inverted face, it is more difficult for the brain to assess the image as a whole — information is “collected” separately for each element: the eyes are in place, the mouth is like a mouth. However, as soon as we are shown the correct face, the perception of a single system suddenly reconnects and problems begin: it becomes clear that the usual features are interconnected in an unusual way.

Why is it important? The human brain is able to recognize the smallest differences in facial features due to the integrity of perception. The area of ​​the cerebral cortex recognizes the face and determines the direction of the gaze, the amygdala and the islet lobe analyze facial expressions, and the area in the prefrontal area of ​​the frontal lobe and the brain system responsible for the sense of pleasure evaluate its beauty.

Bug as a feature: Chernov faces

( c )

The peculiarity of human perception is used to analyze aggregated multidimensional data using "individuals". The American mathematician Herman Chernov in 1973 outlined the concept of using "persons" to identify characteristic dependencies and study the complex relationships between several variables.

The Chernov data is reflected in the form of face-pictograms, where the relative values ​​of the selected variables are presented as the shapes and sizes of individual features: nose length, angle between eyebrows, face width — up to 36 variables in total. Thus, the observer can identify visual characteristics of objects that are unique for each configuration of values.

A quick glance at a diagram made up of individuals allows you to quickly determine whether the profile characteristics differ significantly (coincide). A detailed review of the facial features makes it clear in what features (each facial feature is a separate feature of the original data set) the similarity, and what the difference is. For example, in the illustration above it is easy to notice the difference between countries in sad and cheerful emoticons.

Why does the car need your face

The skill of fast face recognition helps to pick up your child from kindergarten, choose a partner, express emotions correctly and appropriately. But what happens when a person transfers this ability to an artificial neural network?

The idea may cause rejection. Not everyone is ready to easily accept technology that stores data, monitors movement, analyzes purchases and emotions. The transition from simple video surveillance to personalized video analytics entails a significant increase in responsibility.



Today, algorithms such as DeepFace determine the similarity of individuals with accuracy higher than that of people. The Nvidia algorithm itself creates the faces of non-existent people in a few seconds. The people on the collage above were generated by the StyleGAN neural network, which trained on a data set of 70,000 images. They look frighteningly realistic.


Demonstration of the SearchFace algorithm

At first, Facebook's face recognition algorithm caused heightened vigilance, but then everyone got used to (or left the social network). The FindFace service to search for people on photos on VKontakte received ambiguous responses and was used to harass, but the closure of a similar project SearchFace has already caused a negative reaction from users - after all, if the data is available, then let them be open to everyone.

Trading networks are installing facial recognition technology to prevent theft, collecting data on age, sex, and even customer emotions. In the end, the goal is to improve customer service and capitalize on it. When customers realize that the system is beneficial to them personally, many will agree to the introduction of new technologies.

Given the increasing number of cases of identity theft - credit card fraud and personal data fraud, consumers will prefer a system that is at the right moment. correctly identifies them.

At present, algorithms help solve the problems of poor illumination of the frame, low resolution and masking such as glasses, wigs and multi-day bristles. Systems work at amazing speeds and match a person with a database of millions of people in just one second.

Some stores in the United States offer the option of being suspected of theft: allow yourself to take a picture or get a formal crime charge. The thief gains freedom along with the ban on shopping, and his photo officially enters the database. Files containing images of people are encrypted and accessible only to the owner of the system.

Who makes profit from recognition

Most stores have already installed security cameras. For video analytics, an iron update is not required - just connect the cloud service. And with the Ivideon video analytics service, the input threshold is almost absent. The cost of the solution from 1,700 rubles per camera opens up access to software for any entrepreneur.

The main motive of retailers to use face recognition technology is to prevent theft. According to the National Retail Foundation, a retail association, in the US alone, about 1.33% of all goods in 2017 were lost due to theft — no less than $ 46.8 billion in damage.

Face recognition technology reduces the number of thefts in stores by more than 30%.

Often the amount of damage is influenced by minor factors: employee negligence, poor security training, the desire to save. These and other problems must be solved with the help of cameras and cloud technologies.

The facial recognition system facilitates quick work with blacklists: it compares a client's photo with a database of unreliable persons and, if it coincides, sends a corresponding warning to the guards.

Analytical software significantly enhances the security of the store. An experienced thief is able to notice the "blind zones" of cameras. In this case, the guard can use his phone to take a picture of the suspect, and then check if the person is in the database.

Brands have been using mobile marketing for a long time - they send SMS, push notifications and show targeted advertising. For traditional retail, recognition systems offer the same capabilities as online vendors with cookies.

The same platform that is used to identify thieves, helps sellers to figure out which storefronts better attract buyers. The recognition system helps to identify the VIP client right at the entrance to the store. Using the data from CRM, the seller can quickly make the customer a favorable offer.


In the Seoul International Financial Center, cameras on real-time information boards determine the age and sex of a person, and offer advertising according to the parameters identified.

Customer information activates a powerful tool to increase sales and assess audience needs. The cameras will help you set up video ads for a specific visitor depending on their gender, age and emotional state, and will also become data providers for calculating advertising effectiveness.

The above options for retailers often sound like annoying advertising noise. Abstracts of "profit growth" and "audience needs" accompany any IT-tool on the market - from ERP to the electronic price tag. Are there more to face recognition systems than pure marketing about artificial intelligence and future technologies? We will answer this question through examples of using real systems in existing stores.

"Work in the field": who in real conditions recognizes faces

7-Eleven is the world's largest retail chain, uniting under the control of Seven-Eleven Japan over 36,000 small stores in 18 countries. The company recently installed software in 11,000 of its stores. The technology of face recognition and analysis of behavior in the trade network is used to identify loyalty card holders, monitor customer traffic, determine the level of stocks of goods in warehouses.



Saks is a centenary network of premium stores, which is currently owned by one of the oldest companies in the world (founded in 1670), Hudson's Bay Company. Video analytics is used in Saks mainly to prevent theft. The software verifies the photos of the suspects in the theft of a database of known shoplifters. The cameras are networked, so the results can be viewed at Saks headquarters in New York.

According to the Guardian, premium stores and hotels in Europe regularly use face recognition technology, tracking VIPs and celebrities to provide them with the most comfortable conditions.

In the US, the CaliBurger burger network uses face recognition technology in a loyalty program. The interactive kiosk “recognizes” customers, remembers orders and offers favorite dishes, accepts payments with identification by person.

The system eliminates the threshold for entering the bonus program for the elderly, who may find it difficult to use the mobile application, bonus points and credit cards.

Face recognition systems are massively used in Asia, especially in China, where it is common for them to pay for food, withdraw cash from an ATM, or even take loans. The accuracy of face recognition in China exceeds the capabilities of the human eye. This is also connected with the large-scale transition of China from 2D to 3D recognition.

In the first case, the algorithms use for analysis two-dimensional images accumulated in databases. 3D recognition analyzes reconstructed three-dimensional images and demonstrates much higher accuracy. In China, you can make purchases by scanning individuals (for example, paying for orders at KFC), making payments, and entering buildings.



In Alipay, you need to smile , so that the payment recognition system understands: it’s not a photo, but a living person. It is argued that it is impossible to deceive Alipay: changing the color of hair, makeup, using a wig does not change anything. The system uses a set of distinctive features that take into account the geometry of the face and the location of certain points on it.

findings

The scale of direct investment by Western companies and China in face recognition technology is enormous. Nevertheless, in Russia the implementation of such projects is a matter of time. Large commercial companies already understand the benefits and economic effect. If we consider facial recognition as a product, it is important to understand that each business segment has its own specifics, including pricing. The larger the enterprise, the more cameras and analytics modules may be required. Solutions for large businesses are always complex customized projects, and customization requires additional resources. Medium and small businesses can easily do with a single camera with a face recognition module connected. In this case, the cost of the solution is comparable to the use of cloud surveillance.