Do not deign to violate again, sir (ma'am)?

About a week ago, Vasyutka published a post about recognizing numbers and promised that we will post the software that we can try. We cheated a little, not telling all the details. Well, otherwise there would be no intrigue! We did the core of the recognition algorithm six months ago, but the project for which it was done did not move. He stopped somewhere in the alpha version. Having thought and conferred with the customer, we decided that it was not good when such an algorithm was gathering dust on the shelf and did not benefit the society. On the other hand, I did not want to lay out all the source code of the recognition part. Then the idea was born, which will be discussed in this article.

We decided, in fact, to make an open server that would recognize the numbers coming in by http requests. Since the whole idea is non-commercial, the server is, of course, weak, but I hope that this will be enough for the people to play.
And here is a sharp turn of the plot. This post is not about the protocol for accessing the server. This will be the next post. This post is about a program that we started to do as an example of working with a server, but which resulted in an interesting concept in our opinion. The article will tell about the Android application, which you can take a picture of the car number ... And curse the owner with a dirty word! Well, or see the statistics of how many times this motorist cursed.
Those who are unbearable will be able to pick up work with the server from the source at the end of the article. The rest will hold a couple of days in suspense, and then lay out the finished manual.

In principle, the idea of ​​such software slipped periodically. But in the performance of Android - an application where the process of shooting would be automated, I did not see it. Perhaps our first implementation.
Vasyutka and I have never been android programmers, so everything is a little crooked. As the server side. We will be grateful for the advice.

A few words about the program

The program is available on Google Play. Called "Recognitor". It works on all the phones that we were able to get and test. On some tablets, in order to display the video correctly, you need to call up the menu and click “flip” there. The program uses OpenCV, which drags the library. Therefore, such a size (11 mb). After starting the program, the video stream from the camera will go on the screen. To start detection, just click on the shooting button. Allocation of the number takes place on the phone, recognition - on the server.

When shooting, the number should be visible to the eye in the video. This is achieved at a distance of no more than 7-8 meters. It is also required that the angle of inclination of the horizontal number does not exceed 15-20%, and the car was at an angle of no more than 30-40 degrees to the one removing. After shooting, the program tries to contact the server to recognize the number (information is transmitted on the order of 10kb - the cut fragment with the number).

Recognition is not debugged and it takes about 1 second for the number + time is spent on transmission, so after a couple of seconds, if the number was detected, the following menu is displayed:

Our recognition is multi-threaded, it accepts several numbers in parallel. On the left, the footage number that the program has detected is displayed. On the right is its textual representation, which can be edited. The center number is displayed again. If you poke the buttons that make up the number, you can correct it by letter without calling the keyboard.

Bottom left shows how many times the driver was scolded with this number. From the bottom right there is a button with a bad word that can curse a person. If the number is corrected, then the database is accessed for new information.

Algorithm

After a person takes a picture, a frame is selected from the video stream and processed by the Haar classifier. If such a number was found, it is sent to the server for further recognition. If the number is not found - the program continues.
After the last article, the trained classifier itself was asked several times. In the source given at the end it is. In a couple of days there will be an article about the protocols for accessing the server, it will be attached there with a separate file with explanations.
The number on the server is processed by our algorithm, a little bit about which was described in the last article . Searches for a detected number in the database. The user is sent the recognized number and the number of its detections in the database.
If the user changes the number, then re-request by car number.
In short, the concept itself can be represented as follows:

')

What is not done

Not done dofig in part related to android and server:
• Design of the program as in junior school
• Instead of a full-size photo, a frame is taken from the stream.
• Very weak infrastructure of the project, in fact, you can only curse the person, but you can not say for what, you can not curse in other words
• Not all communications with the server are separated into separate streams; as a result, if the connection is lost, the program hangs for 10 seconds after clicking on some of the buttons.

Instead of listing disadvantages, it is easier to say that we have nothing but the basic framework of the program.

With recognition, everything is much more rosy, we are slowly closing it up, now it remains:
• Only white numbers of the type are recognized: “letter - three numbers - two letters - region”.
• During the initial allocation, there are problems for numbers on white cars (where the number is whiter than the car).
• Periodic disruptions in the processing that affect the stability of the algorithm (about 3-5% of the frames do not allocate any number).

Thinking out loud

It seems to us that the concept of software presented here is quite viable and has the right to exist. Besides, sorry for the pathos, it is socially oriented. If a person will often be scolded - maybe he will reconsider his behavior on the road and when parking?
But we do not have the strength to make a full-fledged application with a branched structure, interface and website (we will maintain the current maximum, if we use it). If suddenly someone has such powers, or comes up with another interesting and useful application, then we will maintain and improve the current recognition server.

Actually links:

The program itself on the android - everything they could have checked. It seems to work. Feel free to use the number of my typewriter, shot in the article :)
Project sources - on android, under the eclipse. OpenCV libraries included.
Vasya's article on number recognition.

PS Now we look what comes to the server. People, the number should be in focus! Strongly smeared numbers are not recognized by us!

PSSPublished Continuation - Server Access Protocols.