A prototype project worth $ 86 million in 57 lines of code

Today, a lot of good competitive systems are created by individual developers, not just big companies. As a rule, this is a copying based on a reference solution (a product of a corporation) with subsequent optimization. In this case, everyone benefits: companies from healthy competition, developers from developing technical skills and gaining experience, consumers from new opportunities and solutions.

I introduce myself: my name is Ivan Grekov, I'm from the Badoo front-end team. I really liked this post, I just could not pass by and did not translate it. This post is a great source of inspiration for device prototyping fans and open-source solutions. The translation is published with preservation of the author's point of view and, I hope, will be of interest to you.

The Victoria Police is the main law enforcement agency responsible for law enforcement in the state of Victoria in Australia. Over the past year, more than 16,000 vehicles worth about $ 170 million were stolen in this state. Therefore, the police department is experimenting with various technological solutions to reduce the number of hijackings.

They are already using the web service VicRoads , designed to help prevent fraudulent sales of stolen cars. In the service you can check the current status of car registrations. Also, the police department purchased a stationary license plate scanner - a camera on a tripod that removes passing cars and automatically detects stolen cars among them.

Don't ask why, but once I wanted to create a prototype license plate scanner mounted on a car. This scanner would notify about stolen or unregistered cars in automatic mode. I understood that the individual components of the desired product were already implemented by someone, and I was interested to find out how difficult it would be to combine them into a single system.

But, having searched the network, I learned that Victoria’s police had recently tested a similar device, the estimated cost of which is about $ 86,000,000. It’s not hard to see that if you install it on 220 police cars, the cost of one installation will be $ 390,909 .

I am sure that you can find a better solution.

One of the stationary license plate scanners

Success criteria

First of all, let's formulate the basic requirements for the product architecture.

Local video processing

Streaming video for centralized processing seems to be the least effective solution to the problem. In addition to the huge bills for data transfer, you need to take into account the network delay, which will significantly reduce the processing speed.

Although centralized machine learning algorithms are getting better, I wanted to find out how acceptable the solution would be using local processing.

Work with a fairly low quality image

Since I do not have a camera for the Raspberry Pi or a regular webcam, I will use the recordings from the DVR. This is the perfect source for testing. In addition, the quality of recordings from DVRs is comparable to the quality of cameras installed on police cars.

The product must be created using open-source technologies.

If you rely on proprietary software, you will constantly rest on the need to ask developers to make changes with all the attendant problems. With open-source technology this will not happen.

Decision

In general terms, I decided to take the image from the recording from the DVR, run it through the open-source license plate recognition system installed on the device, request the service of checking the vehicle registration, and then display the results on the screen.

The data returned to the device installed in the police car contains the make and model of the car (for checking situations when only the license plate itself was stolen), the status of the car registration and the notification that the car is listed as stolen.

It sounds simple, and so it is: for example, an image can be processed using the openalpr library. To recognize the characters on the license plate, nothing is needed:

 openalpr.IdentifyLicense(imagePath, function (error, output) { // handle result }); 

A little explanation

There is no public access to the API VicRoads , so in my prototype, license plate verification is performed via the content grabber. Although in general it is a censured decision, it’s still a matter of verifying the concept’s performance - I’m not dropping server draws.

Here is my proof of the feasibility of the idea:

 // Open form and submit enquire for `rego` function getInfo(rego) { horseman .userAgent('Mozilla/5.0 (Windows NT 6.1; WOW64; rv:27.0) Gecko/20100101 Firefox/27.0') .open(url) .type('#registration-number-ctrl input[type=text]', rego) .click('.btn-holder input') .waitForSelector('.ctrl-holder.ctrl-readonly') .html() .then(function(body) { console.log(processInfo(body, rego)); return horseman.close(); }); } // Scrape the results for key info function processInfo(html, rego) { var $ = cheerio.load(html); var vehicle = $('label.label').filter(function() { return $(this).text().trim() === 'Vehicle:'; }).next().text().trim(); var stolen = $('label.label').filter(function() { return $(this).text().trim() === 'Stolen status:'; }).next().text().trim(); var registration = $('label.label').filter(function() { return $(this).text().trim() === 'Registration status & expiry date:'; }).next().text().trim(); return { rego, vehicle, stolen, registration }; } 

results

I must say, I was pleasantly surprised.

I expected the open-source recognition of numbers to be rather rude. In addition, the algorithms are probably not optimized for the Australian number plate format.

However, the system was able to recognize the numbers in wide-angle images.

Annotations are added by me for beauty. Despite the glare and optical distortion, the number was successfully recognized

However, there were problems with certain letters.

Number is read incorrectly, instead of M, H is recognized

But in the end, the system did recognize them correctly.

After several frames, the letter M is determined correctly, with a high degree of confidence

As you can see, recognition accuracy jumped from 87% on the first frame to 91% on the second.
I am sure that accuracy can be improved by increasing the sampling rate followed by sorting according to the degree of reliability. Or to check the numbers, you can set a threshold of acceptable reliability, say, at the level of 90%.

For this, quite simple code corrections are sufficient and it is not necessary to restrict the learning of the application to recognize only the numbers of your country.

The question of 86 million dollars

Honestly, I have absolutely no idea what articles are included in this amount, and I also cannot compare the recognition accuracy of the BlueNet pilot system and the open-source tool that has not been trained in working with Australian numbers.

I assume that part of the budget is planned to replace several outdated databases and applications with new solutions that support a high processing frequency and provide low latency when processing multiple license plate recognition requests.

On the other hand, the cost of $ 391 thousand per car looks pretty strong, especially if BlueNet is not very accurate and if the price does not include the implementation of large-scale IT projects for decommissioning or updating interconnected systems.

Future applications

In order not to dwell on the idea of ​​creating an ubiquitous network of number plate thieves, according to Orwell, many more positive applications can be devised for this technology. Imagine a passive system scanning vehicles on the road and automatically warning the authorities and owners about the current location and direction of the stolen vehicles.

Tesla cars are already crammed with cameras and sensors that can receive OTA updates. And what if to turn them into a virtual fleet of good Samaritans? Uber and Lyft cars can also be equipped with such devices that will dramatically increase the coverage area.

With the use of open-source technologies and ready-made software and hardware components, it will be possible to offer a solution that will provide much higher returns on investments that are much less than $ 86 million.