How Amazon uses the new face recognition system, and why it is hated

Amazon has developed a powerful new real-time face recognition system. Neural networks are “fed” by photos and videos, and they determine what (or who) is shown there. Any AWS user can use it. In the US, with its help, the sheriffs are already successfully catching criminals, and television stations find celebrities in live broadcasts. But technology has had ardent opponents. They write letters to Bezos and call on Amazon to immediately stop developing the system, otherwise the consequences could be most unpredictable.

Amazon Rekognition allows you to embed image and video analytics into your applications, created by deep learning algorithms. It is enough to give your video or a series of pictures to neural networks, and they will understand what is depicted on them. The system is able to recognize the faces of people, their actions, objects, environment. Detect pornography or the scene of the murder.

Technology for the photo was introduced in 2016, and for video - in 2017. Since it is peculiar to neural networks, the product has matured well, wised up and learned new tricks. For example, at the initial stage, Rekognition was not able to determine the context of a snapshot — what does a person now do and what does his facial expression show (pleasure, fear, anger?). Then, this technology Amazon often compared with Microsoft Cognitive Services , and said that Microsoft is doing better.

But in the end, the AWS system became much more famous than its competitor. Perhaps, the role played by the fact that Amazon cloud services account for 33% of the market, and Microsoft - only 13%. Amazon itself says its service is more powerful because it has accumulated more data that can be fed to neural networks for their training. Be that as it may, Rekognition is now for the public and authorities in the United States is the face of deep learning to recognize photos and videos. And now it brings a lot of problems to the company.

How it works

Rekognition allows real-time search for individuals in collections that hold several tens of millions of individuals. First you need to create such a collection (or use the option provided by Amazon). Then the service conducts a quick search on it to find faces that have a visual resemblance to the faces of the people in the photo or video.

Amazon's computer vision experts have developed this system for the daily analysis of billions of images and videos. According to rumors (still unconfirmed), it is she who is used in the first stores without Amazon Go cash desks . In the video, she can even remember where a person went when his face is not visible. Technology promises a great future - for example, in the search for missing people or in the automatic identification of suspected of crimes.

Rekognition can also be used to filter inappropriate content. Now for this you have to use the team of moderators - or expect that users themselves mark photos / videos with violence or scenes of a sexual nature in applications and in social. networks. Amazon now allows you to specify which things need to be automatically removed. And they won't even appear in the output: the system will filter out already at the loading stage.

The service constantly learns about new data, expanding its abilities and recognition accuracy. Moreover, due to the phenomenal power of AWS, the technology is stable with any number of requests. Its delay will remain unchanged even with an increase in the number of requests to tens of millions.

For individuals or companies, the service is rented at a price of 10 cents per minute of recognition of archived or streaming video (plus a free analysis of 1000 minutes of video per month in the first year of use). But Amazon itself sees it in the state. structures. Under a special program, police in the United States can get Rekognition for as little as $ 6 a month to scan hundreds of thousands of potential criminals. The first few sheriffs have already established such technology in their department.

For example, last July, a man entered a store in Oregon. I took the basket, scored the most expensive goods. And he went out without going to the cash desk of self-discussion. The store had a camera that took a couple of photos of the thief, but in a normal situation, they most likely would never have found it - the police have enough other tasks. Unfortunately for the criminal, the sheriff's office, tied to the case, has recently connected to Amazon technology. The shoplifter’s face was driven through a database of 300,000 photos of criminals from their district. She gave four people with similar faces. The detective searched them on Facebook - and found the same person, with the same features and in the same sweatshirt. Everything, the case is closed.

Police say that once a person has already committed a crime, there is a big chance that he can do it again. This is where their old photo comes in handy. Oregon has a database with photos of all the criminals caught in the last 10 years. Now it is used mainly to search by name, surname and place of residence. It’s almost unreal to find someone in her eyes, especially considering that over time people grow beards, make new hairstyles and so on. But neural networks cope with the task in a few seconds. The county police say that the Rekognition capabilities are now being used 20 times a day.

Royal wedding

Another version of how Rekognition can be used for the good is demonstrated last Saturday. Millions of people were chained to television screens, watching Prince Harry marry American actress Megan Markle. The wedding gathered three times as many spectators as the top episodes of Game of Thrones. For many, the truth is, it was a pretty boring sight. Watching how some lords, ladies and British celebrities of various calibers take turns gathering and entering the cathedral. A crowd of people, of which you, God forbid, you know a dozen. Brr

But the British television network Sky News has found a way out of this situation. Viewers of her livestream did not need to guess who it was that was wearing a hat. All data went through the AWS cloud. And Rekognition recognized all the celebrities live with the video, and under each sign appeared with his name, surname and title. In the window next to it was possible to see who this man has for Harry and Megan, and find out more about him. How it worked in real time, you can see here . On the right, you can choose which celebrity you want to see, and you will be switched to the moment when Rekognition has detected its arrival. Cool.

The path to authoritarianism

Not everyone shares the bright feelings about the new technology. This Tuesday, the American Civil Liberties Union (ACLU) and the 40 organizations that supported it released a statement telling about the hidden dangers that Rekognition poses. They also sent an open letter to Jeff Bezos, urging him to stop putting technology in the hands of third parties and stop developing new “surveillance systems that are detrimental to civil society.” Face detection, of course, is not something revolutionary, but, according to the authors of the letter, the way Amazon does it is extremely dangerous and can set a precedent.

The essence of this. After the recent protests of black Americans, who believe that the police treat them with bias, all the law enforcement officers in the United States were forced to wear cameras on themselves. So that in which case it was possible to determine whether the officer was to blame for the situation. Amazon is actively promoting the use of Rekognition with these cameras. So that, say, in real-time, the system could whisper to the police in the ear, "To the right in a black jacket and a white cap is a suspect in three murders." But Malkiya Cyril, the leader of the Black Lives Matter movement, says that he does not believe in the new technology, and it, on the contrary, will make the life of minorities worse:

This is a recipe for authoritarianism and disaster. Amazon shouldn’t even get close to doing this, and if we can do something about it, they won’t.

The cameras on the body of the police do not follow the police. They are watching those who are looking at the police - for us, for communities, for people like me.

Technology is a tool. By offering such a tool in the context of extreme racism and cruelty, this racism and cruelty can only be strengthened. Given Amazon’s computing power, you essentially stop feeding this discrimination system with a battery, and instead connect it to a nuclear power plant. You increase not only the speed, but also the scale of how the state can interfere in our lives.

Rekognition can allow the police to detect in real time who joined the protest. Or who is an illegal immigrant. "This represents a fatal threat to communities," organizations write in their letter to Amazon. “People should be able to walk freely on the street, without fear of being observed by the state.”

Attached to the letter, the ACLU published a series of internal letters between Amazon and law enforcement in Oregon, which it received through an official request. There is nothing criminal there, of course, but it is interesting that even the sheriffs themselves, while communicating with representatives of the Amazon, were slightly nervous about “how the public can perceive it.” And they asked to slow down the introduction of new systems in order to give people time to adapt.

There is also a correspondence with the police in California and Arizona, who asked Oregon how effective their new system was, and how quickly it could be implemented. It turned out that the Rekognition installation cost just $ 400 to the sheriff's office - to upload 305,000 photos with profiles to the system. And $ 6 per month to continue using the service. Cheaper than two big poppies.

AWS responded to the claims of public organizations in the statement that if customers do not follow the letter of the law, they are disconnected from the service. And they promised to continue to closely monitor how their technology is used.

From the Amazon statement:

The quality of our life today would be much worse if we forbade every new invention whenever some people could find a way to abuse it. Imagine if you could not buy a computer, because it can be used for illegal activity?

But reassuring people is not easy, and society’s fears about surveillance systems that use deep learning algorithms continue to grow in the States. A few weeks ago, about a dozen Google employees quit as a protest against the search engine’s decision to unite with the US Department of Defense in the development of the Maven Project. This project creates an AI that will analyze and identify objects in the field of view of the drone (for example, when searching for ISIS terrorists). Google employees were against their technology being used to kill people.

Rekognition, meanwhile, already exists, and in many ways turns out to be much more powerful. The system does not care what content is “fed” to it - video from the drone, TV program, camera recording from the store. She can recognize up to 100 faces in crowd photos in a split second and determine who owns them. The only thing she needs is a fairly serious database with which these individuals can be compared. Fortunately (for Amazon, but not for human rights defenders), the police have such a database. In 2016, researchers from Georgetown University found that there are every second adult in it - 117 million people. And most of these records are not advertised and are not regulated by anyone.

Racial issue

Another big criticism of all American face recognition systems, and Rekognition in particular, concerns their “bias” in relation to women, blacks and Muslims. In short, the bottom line is that neural networks are trained mainly in photos of white men - simply because there are more of them. As a result, technology learns to very well distinguish such men from all others, and works for them practically without errors.

But for women and blacks, these neural systems tend to work worse. The MIT study, for example, showed that one similar system using deep learning algorithms thought that black women were men in 35% of cases — although there were almost no errors for white women.

Rekognition is, of course, much smarter. Still, if on each photo white suspect, on average there are three or four "similar" individual in the database, for the Negro she can give twenty or thirty results. Of which it will be easier for the police to find "that person", even if in reality it is not a criminal. As a result, experts fear, the percentage of searches and arrests among racial minorities with such a system can only increase. And now only this was not enough for America.

A successful example of similar technology in China, where the police monitor real-time criminals and identify “people with low social capital” using their photos and fingerprints, critics of Rekognition are not impressed. They say that in China, with its homogenized population, it is easier to work with such systems than in America, given its cultural and racial diversity. Plus, perhaps, Chinese civil society doesn’t like such state intervention in the private life of people, it’s just not organized enough to protest.

The final argument of opponents Rekognition - FRVT. For more than a year, the National Institute of Standards and Technology in the United States has been testing systems from dozens of companies developing their face recognition technologies. To check which one works more precisely, and who has some disadvantages compared to the others. The project is called Facial Recognition Vendor Test. It is from it that information is mainly taken that all non-white non-males recognize the system worse (of the 62 samples last year, only 4 showed almost no deviations depending on the race). Amazon does not provide its system for testing the institute - which gives skeptics a reason to say once again that "something is wrong here."

And which side are you on, ACLU or Amazon? Would the state give its face to the database?

PS You can ship purchases from Amazon and other US stores from Pochtoy.com. We have the lowest prices, from $ 8.99 per pound. And $ 7 at the expense of all readers who are registered with the code Geektimes.