Hi, Habr! In previous articles, we have already told you about how we managed to turn the input of passport data on mobile devices from a routine into a simple and fast procedure. The next logical step was to turn our Smart PassportReader SDK into a server component, thus making it easier for large financial organizations to work with documents in back offices. Finally, having shown inventive ingenuity and engineering approach, we managed to develop a software and hardware complex (looking ahead, let us introduce its name - Smart PassportBox ), which allows us to optimize the work of front offices and access control solutions. Therefore, if you are wondering how many directors, programmers, soldering irons, jigsaws and screwdrivers are required to create a full-fledged PACK, welcome under cat.

Immediately I want to clarify - we are not declaring that we invented the wheel. The first automated jobs for the recognition of passports of the Russian Federation appeared more than 10 years ago (software for solving the problem there is like the well-known document recognition giants Abbyy and Cognitive, and relatively small companies such as PassportVision).

At first glance, it seems that such a simple and understandable problem has been solved. Take it and use it. However, upon a more detailed examination, a number of limitations arise that you encounter during integration. Let us examine two main problems.
')
First of all, however sad it may sound, most of the recognition software works only on PCs running MS Windows. However, in recent years, in connection with the active development of open source software communities, Linux can be seen more and more often on operator workstations. Consequently, passport-recognizing software should also follow this trend.

Secondly, for the qualitative recognition of passports requires a scanner (you must somehow get a graphic image of the document). And here the consumer usually faces a choice: cheap, long, with average quality or expensive, fast, and with a bunch of extra lotions. Let us explain the current problem in a little more detail. To date, it is customary to receive an image of a document in one of two ways: using a flatbed scanner or using a special passport device. Flatbed scanners with their relative cheapness (the price for compact models of size A5 starts from about 12,000 rubles) allows you to get a picture suitable for recognition in 10-15 seconds. At the same time, it is not a fact that recognition will be performed the first time and no re-scanning will be required (due to page bending or incorrect orientation of the document). During this time, an experienced typist can manually enter passport data. You can speed up the process with the help of specialized devices (usually allowing, in addition to instant scanning, to perform verification, recognition of information on an embedded chip, etc.), the main disadvantage of which is cost (about 100,000 rubles).

After analyzing, we realized that the market for inexpensive but effective passport recognition solutions had not yet been fully developed, there was still room for new engineering and high-tech solutions.

So we were inspired to create a software and hardware complex ...

Optical Scanner Development

As you probably already understood from the introduction, the optical device is a fundamental part of the entire software and hardware complex. Therefore, we generously allocated $ 100 to this device without reducing the requirements for the quality and speed of image acquisition. The first (and most obvious) that occurred to me was the use of a good webcam. Moreover, we can recognize the video well (read our previous post ). And yes, Habr, we succeeded quite successfully! The combination of webcam + Smart PassportReader allows you to solve the problem of automated document input. However, it turned out to be not so easy to master the work with such a PAC for the following reasons:

• a certain skill is required to learn how to quickly demonstrate a document in front of a webcam at a certain distance so that there are no significant geometric distortions;
• it is necessary in each case to take into account the peculiarities of the workplace when the web-camera is located, since otherwise possible glare caused by ambient lighting may adversely affect the quality of document recognition.
• It’s impossible to recognize the whole turn in front of the camera, since the passport is by nature a book and all the time it strives to “fold” in half.

Having a little thought, we decided to install the camera inside a closed box with a glass “lid”, about which everything is known: the distance from the camera to the document (you can fix the focal length), the lighting parameters are always the same and do not depend on external factors, the passport is pressed against the glass surface, which makes it easy to avoid the effect of the book.
During the long cold winter holidays, armed with boards and screwdrivers instead of snowboards and skis, we began to create a device ... Made of wood ...


Despite the apparent simplicity, the resulting device has its own nuances:

• the relative position of the camera and the LED illumination should be chosen on the basis of reducing the possible reflection from the glass working surface;
• LED backlight power should be sufficient for normal camera operation, while document areas should not be illuminated;
• the distance from the camera to the working surface must be chosen so as to obtain the document in the maximum resolution.

As a result, from the third attempt (thanks again to the leadership of the country for such a long vacation) we managed to assemble a scanner box that returns, though projectively distorted, but quite suitable for recognition using our SDK video sequence.

Below is a brief estimate of building materials that went to manufacture. As you can see, we saved the cherished $ 100 with a margin.

No	Component part	Cost, rub.
one	Furniture panels (chipboard) 16 mm	200.00
2	Window glass 4 mm	100.00
3	Accessories (self-tapping screws + holders + a level)	200.00
four	LEDs, batteries, wires	500.00
five	FullHD webcam	5000.00

Software development

If the creation of a box for our organization (which by the nature of its activity is engaged in developing software) is a kind of challenge, then the preparation of a recognition program based on our SDK itself is actually a rest. So, in accordance with our goal, the discriminating software must meet the following requirements:

• cross-platform (in particular, support for MS Windows and Linux);
• natural integration with an optical scanning device;
• high recognition speed.

Accordingly, when developing software, we used the following techniques:

• all programs are written in C ++ using cross-platform third-party libraries if necessary (for example, Qt for writing components with a user interface);
• natural integration with an optical device is essentially writing a webcam control module (which, unfortunately, had to be written individually for each operating system) and correcting the resulting geometric distortions;
• high recognition speed is provided by the core itself - on an average PC, both passport pages are recognized by the Smart PassportReader SDK for about 200 ms.

What we got

Following the saying “It’s better to see once ...” we decided to make a small video about the resulting hardware-software complex.

Category “Questions and Answers”

In this section, we have the courage to predict your questions and formulate answers to them in advance. However, we look forward to other questions in the comments to the article.

1. So, after all, how many directors, programmers and tools did it take to create such a PACK?
All joinery materials can be bought in a large hardware store (in our case, Leroy Merlin), there you can also carefully cut boards of the required size (so as not to mess with the jigsaw at home), webcam - in the Yulmart electronics store, wires and LEDs - Chip and dip. Additionally, it took 1 director (it turned out that directors, by their nature, have a good screwdriver and soldering iron) and 1 C ++ programmer (to create a software connection between the web-camera, Smart PassportReader SDK and the output interface).

2. Judging by the images, the PAC has impressive physical dimensions. Is it convenient to work with him?
At numerous demonstrations, potential customers are really confused by the height of the scanner (and it is now 276 mm). But we must understand that this is just a prototype, made entirely from household parts. If you make a plastic case, use a small wide-angle camera, the height of the device can be increased to 120 mm, which is quite comparable with existing passport scanning devices. But even with such “huge” sizes, the device can be effectively used: no one forbids hiding this “box” inside the table or screwing it to the side (as shown in the figure below).



3. All the same, even with such a compact version, there is plenty of free space inside the “box”. You did not think what you can fill it?
We didn’t just think, but already started to fill the “box”. The remaining space can be effectively used for location inside the microcomputer, which itself will deal with the recognition of documents (since our SDK is perfectly optimized for the ARM platform). In this form, the passport recognition PAC will be localized within one device, without using any external computing power.

Conclusion

Of course, our wooden box can hardly be called a finished industrial design. Nevertheless, already at the current stage of our development work, we can formulate the main advantages of the resulting hardware and software complex that solves the problem of recognizing passports of citizens of the Russian Federation:

• high passport recognition speed (less than 1 second per turn on a computer with an Intel Core i5 processor along with scanning );
• high quality of recognition (balance of hardware and software components);
• simplicity of the device (body + camera + software, does not require special drivers and other system software);
• ease of operation (device design prevents typical operator errors);
• not a single moving part (in fact, nothing to break);
• simple API with various interfaces (C ++, ActiveX, Java, C #);
• domestic development.

PS The concept of a hardware-software complex described in this article is protected by patent laws of the Russian Federation.