Artem Kukharenko, the founder of NTechLab - about face recognition, the potential of neural networks and business

Everyone heard about NTechLab at that moment when the FindFace identification algorithm demonstrator became available on the Web - people were shocked that just by a few clicks, it was possible to determine the network coordinates of almost any person from a photograph: a passerby, a passenger opposite, and so on.

Although the path of Artem Kukharenko on the way of recognizing faces began long before that, and the most significant point, in fact, was the victory in the competition of Washington University MegaFace - yesterday the company announced its cloud SaaS product FindFace.pro , which already promises a great future.
')
We talked with Artyom about algorithms and business, as well as the prospects of technologies for intelligent recognition of objects.

Artem Kukharenko received a higher education at Moscow State University. Mv Lomonosov, where he graduated from the Faculty of Computational Mathematics and Cybernetics. As a student, he worked at the Laboratory of Computer Graphics and Multimedia at Moscow State University.

2013 - began to collaborate with the e-Lab of American University Purdue.

2014 - Artem Kukharenko became a member of the Moscow Research Center Samsung. At the same time, he acted as the author of several scientific articles on the subject of neural networks.

What you do is unlike most startups. What was the trigger to start doing this project?

There are many factors there. I had a diploma in facial recognition at Moscow State University. At that time, we were just starting to study neural networks and basically used so-called hand-crufted features — algorithms, for example, SVM over LBP or HoG descriptors that arose even before neural networks.

Then I worked in different companies. They then began to actively use neural networks. We realized that using neural networks can improve the quality of work in various fields.

In my free time I developed an application that recognizes the breed of dogs and showed it to friends. They introduced me to investors. Together we came up with several options for applying technology and understood how to make money with it. As a result, came to recognize faces.

Who are your investors?

These are private individuals. Now one of them has become a co-founder. Others are also close to this. Now they devote 25-50% of their time to the project.

Then, probably, it would be more correct to call them partners?

Yes. In addition, a cool team of development engineers was selected. Initially, we were engaged in the development of three. It was then that we managed to invent and implement a face recognition algorithm that won the The MegaFace Benchmark competition. It was conducted by the University of Washington. It was necessary to recognize faces in a database of 1 million people. Previously, such contests were held on small bases, and in MegaFace everything was close to real conditions.

More than 100 teams from all over the world naturally participated. And we also sent our own algorithm - just to understand what stage of development we are at. And suddenly they took first place. From that moment, there was a lot of interest from customers, investors, and the press. And we began to actively develop b2b products.

FindFace appeared a little later?

Yes. Initially, we did a search engine to search through large arrays of photos (up to 1 billion). Perhaps that is why we won first place. Search accuracy is the first important criterion. And the second is the search speed. If the algorithm will search for half an hour or even a minute, this is not particularly interesting to anyone.

How much do you think about privacy per se? Everything is already a long time ago on the Internet. There are no people who are professionally engaged in espionage. When giving away your photos to Facebook, Google, VK, we need to understand that at some point they will search through these photos. What is your opinion about the negative consequences of the implementation of recognition algorithms?

When we created the algorithm, we, the engineers, were just interested in making it efficient. And only then we began to think about how to apply it correctly and produce it in the form of products for a wide audience. But we decided that when powerful technologies emerge, people should be aware of them. If we gave the technology only to special services that would use it, and no one would know about it, this is one thing. And we laid it out for public use - this is different. People know about it and make conclusions, what to lay out in the network, and what is not.

Public opinion will adapt itself, knowing the existence of this thing.

Nevertheless, we received many letters of appreciation from the Ministry of Internal Affairs, the police, and so on.

Watching your Facebook profile, I noticed that after the appearance of FindFace, you changed the photos, the general background happened such a “high point”. After MegaFace came a lot of orders, customers. Is the company starting to grow at a serious pace or are there structural barriers to the commercialization of these solutions? What are your business plans?

We are growing all very quickly in all directions. The company has about 25 people. Before the new year will be about 30. You need to quickly dial a team, "zaskeilitsya." It seems we are coping well with this. Our company is divided into three logical components - a business unit, a product team (makes the final product) and a laboratory (improves algorithms, search quality, etc.).

We now have more than 500 applications from customers. At some point we were faced with the fact that there are a lot of orders, but there is no product for business as such.

In what form can it exist in the market? SaaS, Black box serverside app, mobile app?

FindFace is just a demonstration of our algorithm. We do not plan to earn money on it. Now we are launching the FindFace Cloud API (SaaS) cloud service, it allows you to upload a photo and search for it.

On October 18, NTechLab presented a business solution - a product of FindFace.Pro.

The product allows you to integrate technology for facial recognition with third-party applications through the "cloud", according to the description of the service. With it, you can detect faces in images, identify and form the basis for further use.

The product is focused on shops, casinos, companies that organize concerts and other public events, and other enterprises. When concluding an agreement, NTechLab credits the client with an account of $ 100, for which he can make 18 thousand requests for face recognition or create a gallery for 10 thousand people for three months.

The service is in demand in many industries - from social networks and dating services to retail and security systems.

This is applicable in retail: for example, there is a store and a database of regular customers. In addition, the technology is used in the banking sector - confirmation of payments for selfies, authorization.

There are all sorts of applications in the field of entertainment: for example, the search for similar people. In the dating services it can be used for moderation of photos, since the use of other people's photos is an actual problem for all social networks.

You were on CyberCrime - this is also one of the examples of use.

Yes. There was also a project with Alfa Future People - there are a lot of photographers there. They had a chat bot, which showed users photos from all the events that a particular person attended.

Our second product is the SDK. The library is written in C ++, it provides mechanisms for detecting faces, building the feature vector, base and search in this database. This is interesting to all sorts of integrators, who will build SDKs under license into their products.

It turns out that we have SaaS, SDK, as well as custom solutions for large projects.

Not so long ago, Facebook launched a video inspection service for duplicates. Each video is checked to identify whether the copyright holder laid it out or not. Do they use technology similar to yours?

As I understand it, they are looking for full compliance. And we find one photo is not the same, but a set of similar ones. This may work in the fight against fake accounts. We are now communicating with several major social networks. They want to embed our algorithm. They have just the problem with the search for large databases of photos. In the meantime, they have an "army" of moderators who do it manually.

Your target consumer is the state. They need these products the most. But no politics has happened to you yet? Will there be any problems if you sell your products not only in Russia, but also in the USA for example?

Not yet. We have a purely market history. There is no preference. Moreover, we are building a global company. We made SaaS just for this. It is easier to sell all over the world.

What will happen to NTechLab by 2020? How many employees will there be? Will your office be in Moscow only, or will you open headquarters somewhere in the states?

Office in the states we will have until the end of 2016. We have a plan - to integrate our algorithm into all cameras by 2020.

In all videos and cameras that are commercially available?

Yes. But perhaps we will still be engaged in other tasks: for example, the use of deep learning in medicine.

Is this what Watson is doing now? Diagnostics mostly, yes?

Yes. Medicine is the first thing that comes to mind. But next year we have enough current tasks, recognition of emotions, gender, and so on.

How do you plan to recognize emotion? Also conduct training: sad person / cheerful person?

Yes. We have it already. We are now testing this feature. Perhaps soon it will appear in SaaS. It is clear that there will not be the same as in the TV series Lie to me. There will be no magic, but at the level of a person who does not try to hide emotions, recognition will occur.



I know that face recognition technologies before the advent of neural networks were based on fixing, determining the position of certain points on a person’s face. How is this done with you?

We are not trying to find any points. The neural network receives the entire face and the correct recognition responses: “This is Vasya, this is Peter, this is Masha”. In the process of learning on the last layer forms a vector of signs, which contains all the information about the person. On the intermediate layers, important in the opinion of the neural network are calculated.

If we set the structure correctly, on the last layer we get a feature vector from 80 numbers.

That is, 80 parameters that determine the uniqueness of a person?

Yes, 80 numbers is less than half a kilobyte. Therefore, we effectively get to do a search on large databases and economically spend memory on the storage of signs.

What power is needed for this?

When the network is trained, it can work efficiently on a regular computer. For example, FindFace runs on five Amazon EC2 servers — and not on top machines. The system holds 50 requests per second when searching the database of 250 million photos.

And if we need to train her?

We teach it on the GPU, because it is best suited for training neural networks. One training on 20 million photos goes on average a month in our case. But we still have servers here, on which there are about 20 experiments in parallel.

That is, you do not need one full GPU supercomputer, is enough of a cluster?

Yes.

How much harder to work with video than with a photo?

Video from the photo differs only in the number of frames. Therefore, when working with video, the speed requirements of the algorithm increase. From the point of view of learning, nothing changes.

SaaS, which you run, will work with video?

No, but the SDK works with video. To send the entire video stream to the cloud does not look very reasonable. There will be a module with a face detector. Therefore, for SaaS, you need to select frames and send them separately to the cloud.

How expensive is the development of such software solutions? How high is the price of the product for the consumer?

It all depends on the project. SaaS per thousand requests cost from $ 1 to $ 5.

So this is a mass product?

Of course. That is, for some event or a store, it will cost about a few hundred dollars a month. If there are large individual projects, the price will be different.



More about technology. SaaS and SDK are based on the same algorithm. The SDK is written in C ++. It is clear why: it works quickly and reliably. What other languages, technologies do you use?

For training we use standard libraries - Caffe, Theano, Torch7. We write in Python, C ++ and the Cuda platform. With Cuda on GPU, learning is much faster. We need Python mainly for experiments, so that you can write something, try it. The cloud is also written on it, because it is more convenient.

What requirements do you make to programmers in R & D?

Here it is not the languages ​​and technologies that are more important, but the knowledge of algorithms, in general, the requirements for mathematical preparation. Of course, we need some knowledge in the field of machine learning and C-like languages.

If we talk about the product team, then, of course, you need C ++.


Photo: Alexander Utkin

Question about the database. Where is it all stored?

If we are talking about the search index, this is our development. It allows you to search in large databases in less than a second. This is part of the product.
If we talk about the storage of information about users, the database does not have much importance here. We use Mongo. But there is nothing related to the search.

Many people started offering money after you became known?

A lot, but we have not taken money from anyone yet. In the near future we will close the round.

You went to the USA not so long ago. Was it your first trip?

Yes.

Was there a lot of interest directly to you? Who did you communicate with? Do I understand correctly that you combined both leisure and business there?

We traveled in the spring. Recently, my partners went to the USA without me. The interest is huge: we talked to both investors and customers. We have good relations with some of them, we work, we sent them an SDK.

Did you manage to communicate with the developers? Invite someone to work?

Not. But we have all the development now in Russia, and there are a lot of advantages in this, in particular, economic ones. We will open offices in the Valley, and after some time in Europe.