Shop of the future. How the first supermarket without cash registers operates in America

In 2016, Amazon tested the “store without cash desks and queues” on its employees. And now such a supermarket was first opened to the public. It is equipped with cameras and sensors that look what the buyer takes, and when you exit the store money is automatically deducted from the card. The opening was expected at the beginning of 2017, but the technology had to hone an extra year. It was difficult for her to distinguish many buyers of similar build, and when, during the test period, people came with children, they caused chaos by incorrectly folding goods. But now, after five years of work, Amazon says that the technology is ready, and "will become a new achievement in computer vision and machine learning."

Prior to this, similar projects already existed - for example, Moby Mart , which was tested in China in 2017, and the “staff-free cafe” from Alibaba. But for Amazon, the system works on a completely different scale, pleases (quite numerous) buyers, promises to bring profit, and is already ready for expansion. And we are looking at how it all works, and why millions of people are up in arms in the US.

Big Brother is watching

At the entrance to the store you are greeted with rows of turnstiles. It may seem that you are going down to the metro station. Pass through the turnstile is only possible with a smartphone. Access is opened when it scans the QR code generated by your Amazon Go application . At this point, the Amazon account is tied to your physical person, and the cameras (THOUSANDS of cameras) begin to track your every action.

The Amazon Go app is linked to an Amazon account, and all purchases will be deducted from there. A couple of minutes after you leave the store, you will receive a receipt on your smartphone with what you bought. If the system suddenly brought something wrong, you can indicate this, and the money will be returned back to your account. It seems that the system is vulnerable to those who want to cheat it, but Amazon representatives say they have countermeasures for this. What specifically, the journalists are not informed, but the simplest thing you can think of is that if a person constantly writes everything wrong, Amazon Go simply blocks him. Like, and why do you continue to go to the store, if you didn’t work out with him?

The store itself, if you do not look up, looks like a rather ordinary small American supermarket. Clean, spacious enough, with a maximum of wood in the interior (like Whole Foods) and shelves for chips, drinks, bread, salads, vegetables ... The most noticeable difference is that Amazon Go has no carts or baskets for shopping. They are simply not needed: visitors immediately put the goods in their bags (or pockets). At this point, things are recorded in the "virtual basket". Big Brother remembers what you took from the shelf. If then you decide not to buy the product and put it back, even to another place, the system will take this into account and automatically delete the record from its memory.

And what about those who go shopping with the whole family? Here Amazon came up with two solutions. First, everyone can sign up for your Amazon Go account from their smartphone. Secondly, you can skip the whole family forward on the turnstile using your device. The result is the same: the system remembers everyone, and everything that they take with them is combined in a common account. If you buy with a friend and each have their own Amazon Go account, the money is read from the person who takes the goods from the shelf.

To the exit

It is calculated that in the store you can take all the products to which buyers are accustomed. There are shelves of milk, yogurt, cookies, pastries, snacks. A separate section for organic food chain Whole Foods (which Bezos bought for $ 13.7 billion last year), and a section with Fresh Meal Kits - sets launched on Amazon this summer, allowing home cooking for two. The store also has its own kitchen. You can take a soup or a sandwich or have a snack in a small cafeteria. Prices for everything are the same as everywhere else, neither more expensive nor cheaper. Obviously, Amazon does not expect to "beat off" all the costs of technology by one supermarket, and average prices will allow it to compare the activity and the level of customer satisfaction.

Behind the corner of the supermarket is the only place where the buyer needs to interact with the person: a section with beer and wine. There, an Amazon employee checks your ID to see if you are 21 years old. After that, you can grab the alcohol and leave the store, just like with any other product.

The exit is carried out through the same turnstiles, this time without any authorization or any actions on your part. Sound or light signals either. Just go about your business. No queues. And then the check comes to the smartphone.

The first buyers, who experienced the first supermarket without cash registers, say that the sensations are very strange (more precisely - “dumb”). When you go out the door, it seems that you stole food from the store, and then it’s hard to get rid of this feeling. Perhaps this will be the main trick of Amazon Go: when a person does not feel like he pays, he tends to spend a lot more.

Chamber work

The developers of the system say that the “Just Walk Out” technology in the store uses “computer vision, depth learning algorithms and a combination of sensory data from various sources”. The main "source" - hundreds of specials. cameras mounted in the ceiling. They track every point of the store from several angles. Basically, these are ordinary RGB cameras, only with built-in development boards from Amazon, thanks to which they independently perform basic tasks: they track movement, find an object, try to distinguish it from its neighbors. Nearby are slightly more advanced cameras capable of measuring the depth, marking the flight time of the signal to each point of the image.

When you stand in the supermarket and you look up, you hardly see these cameras: just gray blocks embedded in the dark ceiling. It may seem that this is a design element. In a regular store, cameras stand out a lot more.

The resulting images are sent to the “central processor,” which Amazon doesn’t say anything about. He performs the main work: the identification of who is who, in real time. Which person picked up what item, what was put back on the shelf, and so on. This processor is the main feature of the whole project, without which nothing would have happened. It is difficult for even the human eye to understand which of similar people wearing a gray T-shirt and blue jeans took exactly which yogurt from a dozen almost identical white packages. And then the system should determine this lightning-fast, unmistakably, and in a hundred places at the same time. Earlier on such a scale and such speeds nobody even wanted to take on the task.

Amazon, as usual, does not disclose its secrets, but you can look at similar projects - to present how it all works. For example, the startup Standard Cognition recently developed a computer vision system that tracks customer movements and identifies the products they take. And then, for example, sending this data to the guard’s smartphone. It turns out pretty simple when compared to “Just Walk Out” in Amazon Go. But the company expects to sell its development to supermarkets and large retailers. To become successful, it is enough for her to buy one network for 2-3 thousand stores.

Bodega is working on the same idea with kiosks. Three or four shelves, five cameras, one buyer at a time. For office, cafeteria, cafeteria at the university. Well, in exactly the same direction as Amazon, Aipoly is moving - trying to create "fully autonomous stores" without sellers and cashiers at the expense of computer vision .

The main problem is the accuracy and speed of data processing. The basis of the system, in the end, probably, could be done by a student in a few weeks, if he had enough time and resources. But she correctly handled, say, 80% of cases. And in order to bring it to an acceptable 99.9%, millions of dollars and many, many years of work are needed.

Moreover, the Amazon system, according to the developers, does not use facial recognition. The company did not want to be accused of violating privacy, and tried to avoid criticism to the maximum. Those customers are not even remembered. Instead, “Just Walk Out” uses other visual signals (gait, limb length, etc.), and also monitors your movement through the cameras so that each of them “takes over” you from the previous ones. A person is not required here: your person will never lose sight of, which means that you do not need to re-identify you.

If one of the chambers breaks, or something pollutes its lens, this does not affect the operation of the technology. The system has already been tested without part of the cameras, and with a degraded view. According to the calculations of engineers, it is necessary from 10% of broken cameras, so that the quality of recognition of customers and their purchases in the store is somehow significantly reduced. In addition, cameras help weight sensors in the shelves. They know the exact weight of each product, so grab two yoghurt at a time, hide one in the sleeve, and so the system will not deceive, even if you are David Blaine.

How to avoid theft

There are no employees in the store, with the exception of people filling the shelves. But Amazon is not worried about attempting to steal goods. Theoretically, if you know the exact weight of the goods you want to steal, you can do in the style of Indiana Jones. Put sand in a bag and put it on the shelf at the same moment when you pick up the product from there. And at the same time, cover yourself up with some particularly large hooded cloak. But it requires much more dexterity than ordinary shoplifters. And expensive things in Amazon Go is not for sale. The skills of such home-grown Jones will be more efficiently spent somewhere in the bank or jewelry boutique.

The fact is that most of us do not steal. It is unpleasant for us. And with a small loss of money due to kleptomania or poor people, ordinary shops were reconciled, taking them into account when drawing up business plans. Most of the thefts in the supermarkets are carried out by the employees themselves , and another 34% of the lost goods are due to mistakes (or abuses) of management and suppliers.

You can steal goods in a regular supermarket. I took what you want, tore off the label, put it in my pocket, went to the exit. With Amazon, on the contrary, it’s harder to do it, wherever you put the product, you won’t be fooled by the system. If you do not find any critical vulnerability, "Just Walk Out" can be advertised as the most "theft-proof" model of the work of supermarkets.

Another thing is that there are mistakes in technology, and they were noticed on the very first day. Tweets began to appear from people from whom the store did not withdraw money for a particular product. Amazon responds that they may regard this as a gift from the supermarket. The percentage is quite low, and the company does not have the opportunity to completely eliminate such errors, so it does not even bother. And buyers still enjoy unexpected “gifts” and share their impressions on the net.

Technology did not exist when Amazon launched its project five years ago. All created by themselves, from scratch. Therefore, in the first days, when the supermarket is full to a cue ball, a couple of mistakes can be forgiven (the developers admit that it’s harder for the system to track many people standing together in a group). The complexity of monitoring does not increase with an increase in square meters, you just need more cameras and more computing power. And the technology will continue to be honed.

World without people

The American working class is worried that soon there will be no place for them. There is a joke that “soon Beosos will work in America, and the remaining tasks will be performed by robots”. In every joke there is some truth: Amazon excludes the human factor to the maximum, if possible. In 2012, it bought Kiva Systems, a machine manufacturer, for $ 775 million. Now in its warehouses about 50 thousand of their robots, similar to huge orange rumba, work. They are gradually replacing storekeepers. In the US, about 3.5 million cashiers, and their work in the world with Amazon Go, it seems, is no longer needed. Where to put the released labor force?

Amazon says the problem is contrived. In their “store without cashiers and cashiers,” in fact, there will be as many people working as in a regular market, just buyers will not see them. These are people filling the shelves with supplies, cooks, making sandwiches and dishes that customers take with them, and, of course, technicians that keep the webcams and turnstiles in proper condition. Bezos likes to blame for depriving people of work, but in fact, Amazon is now expanding at a frantic pace, hiring thirty-forty thousand people every year, and under a hundred thousand more during the holidays.

But the work of the cashier in the near future, apparently, will really disappear. Advances in this direction are being made even in Russia , where fast foods and restaurants have begun to replace people with self-service kiosks, automatons and other digital platforms. Accuracy and speed of work grow, and systems do not have to pay salary, which allows them to pay for themselves in less than a year.

Who needs it

Opponents of the idea say that the aggregate number of jobs will go down, and the rich will again become richer, and the poor - poorer, so the new technology will not bring benefit to society. The ubiquitous observation, hundreds of cameras that monitor your every movement, are not to everyone’s liking, either. And all for the sake of saving "a couple of seconds" when leaving the store. A huge amount of resources and brain power of smart people is spent on what is already working normally.

Supporters of “Just Walk Out” note that the strategy will reduce costs and increase the carrying capacity of stores. Savings are not only on wages, but also on the collection of cash, at the work of cash registers and payment terminals. Overhead is reduced, and thus the cost of goods in the store is reduced compared to competitors who have live cashiers.

Another important point is that the simplification of the payment procedure in the store according to statistics increases the size of the average check. The easier it is to part with money - the more goods a person buys. In Amazon Go, you don’t have to imagine how each of the things you bought will be taken in turn by the people at the checkout.

Plus - as with the giant Bezos clock , immured in the mountain for 10,000 years, in the process of developing a new technology, you can find solutions to other problems.

So far, Amazon has not announced plans for a massive introduction of Just Walk Out, but many industry experts have no doubt that it will revolutionize the service industry. Hundreds of thousands of Americans who worry that they spend more time on the queues at Walmart and Costco than on collecting products themselves are already inspired by technology. This can be a huge step in the field of offline sales, it is compared with the invention of shopping carts in the 1930s. What in the end will be more, good or bad, time will tell.

PS

From Amazon and other American stores, you can order goods directly to Russia from Pochtoy.com . The prices for equipment in the USA, as a rule, are much lower, and there is more choice. Shipping cost for 0.5 kg starts at $ 8.99. And when registering with the code Geektimes, you are charged $ 7 to your account.