MWC-2017. New technologies for mobile recognition (3 videos + links to the demo on the App store and Google play)

Next week, the largest event in the world of the mobile industry will take place in Barcelona - the international exhibition Mobile World Congress 2017 (from February 27 to March 2, 2017). The Smart Engines team will show new developments in the recognition and processing of video streams on mobile devices: ID recognition of different countries, detection of holograms and dynamic panning of documents.

We have prepared a small review of new technologies (+ video), which we will show in Barcelona. Welcome under the cut!

1. Recognizing the ID of different countries on mobile devices

Currently, in the world, the identification of identification documents in mobile applications is often limited to extracting data from a special machine-readable zone (MRZ), which is present on some types of documents. At the same time on many types of IDs and passports this zone is not. And there are hundreds of such documents in the world.

In this regard, we took up the task of developing an extensible solution that allows you to recognize, in addition to extracting MRZ data, all the relevant human-readable textual information, as well as to look for photographs and signatures on a wide class of documents.

In Smart IDReader, ultra-fast identification of the document type before recognition, sub-pixel localization of information (text and graphic) elements, and text recognition by ultra-light artificial neural networks (ANN) are implemented.

As we know, when recognizing directly on a mobile phone, key issues - speed and energy efficiency, and the usual approach “first we recognize everything, and then we comprehend the text” does not lead to an acceptable result. Knowledge of the type of document can dramatically reduce the computational complexity, but the problem is that the number of these types in our case amounts to hundreds. Therefore, a key requirement for a document type classifier is its sublinear complexity in terms of the number of document types.

High performance of our INS is provided both by deep technical optimization of computations at a low level for each computing platform, and by using advanced technologies of generalizing training and augmentation of data using distortion modeling.

We built a unique “one-shot augmentation” technology that allows you to create high-precision INS based on a small number of real data instances.

In addition to Russian documents (Passport, SNILS, Driver's Licenses, CTC), documents of the USA (Passport), Great Britain (Driving Licenses, Passport), Germany (Passport, ID Cards, Driving Licenses), Austria (ID Cards, Driving Permits), Spain (ID cards), Philippines (Passport), Malaysia (MyKad Identity Cards), Kazakhstan (Identity Cards), Japan (Driving Permits, Health Insurance Card), China (Passport), as well as any documents with machine readable area (MRZ) and banking cards (credit and debit). The list of documents is expanding.

The demo program Smart IDReader can be downloaded in the App store and Google play .

2. Detection of holograms in a video stream on mobile devices

A person can easily distinguish the original of a hologram document from his photocopy, simply by rocking the document in front of him. Until recently, there was no product on the market that could carry out this check automatically. This is due to the fact that without matching technology in a video stream with sub-pixel accuracy, it is extremely difficult to distinguish a hologram from the motion of a static motley picture.

We managed to achieve the required geometric accuracy indicators, as well as solve another problem - to consistently distinguish changes in illumination from hologram overflows. The theory for such problems has been known for a long time and is called color constancy, but it requires an already calibrated instrument, which is impossible in the case of mobile phones used by end users.

The video demonstrates the detection of a hologram on the passport of the Russian Federation.

3. Dynamic thinning and panning of documents in a video stream (video scanning)

The task of dynamic panning of a conditionally flat underlying surface often arises during aerial photography. At the same time, the speed of image displacement in the frame should not be constant. Especially significant speed fluctuations occur when shooting with copters. The technology of dynamic thinning is the removal or averaging of frames with small spatial displacement. As a result, a video stream with a smaller volume is generated, aligned with the observed speed and lower noise level in areas where averaging was possible. In parallel, it is possible to build a single panorama image. This task seems to be the most relevant when shooting with restrictions on the volume of drives and the width of the communication channel.

The video demonstrates the use of such a video scan to obtain a high-quality image of a check of a non-standard size.

We are waiting for you at our booth at MWC 2017 (Hall 7 Stand 7G84) and will be happy to tell you later how it was!