How ABBYY Cloud OCR SDK helps to catch cheap flights

As our regular readers know, some time ago we had ABBYY Cloud OCR SDK , recognition service for developers, the “cloud brother” ABBYY FineReader Engine . The service works so simply that your editor (tm) rarely succeeds in persuading developers to write a post about him on Habr - technical details are not enough for an entire article. That was really the very first post about beta testing .

Today we will tell how the service is used by one of our clients - Aviasales. This is one of the largest ticket search engines in the world (in foreign markets, the company operates under the name JetRadar), about 10 million people use it a month, and the number of search sessions per day is close to a million. Already quite a long time ago Aviasales , like any self-respecting service, has released mobile applications for searching and buying air tickets.

I must say that “mobile travelers” are different from those who plan their trips on a computer. Often, air tickets are sought and paid on the run, so entering your passport information is slower. It takes 3 to 5 minutes to fill in standard fields and in a more comfortable environment. And this is not counting the usual mistakes like confused names and surnames or incorrectly specified passport validity.
')
In general, the developers of Aviasales decided to add the function of recognizing passport data from a photo into a mobile application and, after analyzing the offers available at that time in the market, addressed us to this task. Passport data can recognize several of our products, partners have stopped at the ABBYY Cloud OCR SDK web service. You can read about how our technology works here .

Thanks to the Cloud OCR SDK web API, it was easy to add recognition functionality to the mobile version of Aviasales. Since the application only needs to be able to send HTTP requests, recognition can be screwed to the application on any platform - our partners first added it to the mobile version of Aviasales for iOS, in the plans - Android. Since the recognition process itself takes place on the service side, this functionality does not increase the size of the distribution kit and does not land the battery.

In short, the process is this: in the Aviasales application, when filling in passenger data, you can take a photo of a passport or select a photo from the gallery, a file with the image of the passport page is sent to our service, it recognizes a machine-readable zone (machine-readable zone, MRZ) and returns as XML (last name, name, passport number, date of issue and expiration). The application parses the answer and saves the data in iCloud, so that you can then use auto-complete with every purchase. Since MRZ has a uniform international format, recognition works on the passports of all countries in which this line exists.

Some difficulties arose at first with the recognition of Russian internal passports. The data of the internal, general passport is needed to purchase tickets for domestic flights, and the passenger's first and last name on the ticket must be written in Latin. The Latin spelling of the name and surname is still taken from the machine-readable line (MRZ), which appeared in Russian internal passports in 2011. The problem was that the information in this line is encoded in a special way: some letters of the Russian alphabet are replaced by numbers or letters of the Latin alphabet (this encoding is called “upgraded cler”). For example, the letter turns into 7, and - turns into Q. And this is how the ABBYY technology recognized these symbols. This problem was solved simply: the developers of Aviasales wrote a bit of code with a simple script with a dictionary that turns the modernized claire into Latin.



For the user, the new functionality looks like this: you need to go to the “Passengers” section and click on the “New passenger” button. A window will open with a form, in the upper right corner of which there is a camera icon. Clicking on this icon and taking a picture of a passport (you can simply choose a photo from the phone’s memory), you need to wait a couple of seconds, and after that passport data will appear in the corresponding fields. The passenger only needs to check the data and click on the “Add” button so that the program remembers it.






Yes, this is our girl and she does not mind that her passport is shown here







Colleagues from Aviasales have injected several small but nice details into the application, among which there is a user notification about the expiration date of the document - several months before this period.

Passport data is stored in iCloud for synchronization between user devices. It is absolutely safe, data is not transmitted or stored on Aviasales servers. When a customer goes from the Aviasales page to the website of the airline ticket sales agency, he is invited to click on the “fill in” button in order not to enter all data manually. But the substitution in the form when filling on the sites of partners occurs only at the request of the user and is approved by him every time.

In conclusion, we say that the developers of Aviasales have integrated the function of extracting data from passport photos in just 2 weeks.

Colleagues from Aviasales say that autocomplete increases the chance to “catch” a cheap flight ticket until someone else has intercepted it. We did not check, but, theoretically, a few minutes of the odds in some situations can really play a role, given the fact that the cost of tickets may change every 15 minutes.

Of course, scanning documents is, as they say now on the Internet, wow-service and not all use it, but in the first 4 months after the appearance of the auto-fill feature, more than 10 thousand travelers used it - not so little. For the user, filling out a form with a large number of fields is usually not a very pleasant experience. And when there is an opportunity to simply take a picture, and the data will be filled in itself, it is tempting to add your data to the application.