Number recognition: from A to 9. Part 3
A week ago we published an article about an open server for recognizing images of license plates. Now, as promised, the article is about how to send your photos with numbers to it. Our goal, as you remember, was not to swear at each other with indecent words, namely, to make a functioning server on the Internet that copes with photos and sends back the result of recognition.
(part of the photos sent during the week)
I would also like to tell you about how we, programmers who use the nose of Internet technologies and Linux, solved the problem with the server.
All thoughts about the present noisy computer under the ear, pulling the cable into the kitchen and negotiations with the provider about the real IP, were rejected as they do not correspond to the new realities (they are only talking about cloud services and other innovations from all sides). But I also wanted the convenience of the familiar Windows, dotNET, and indeed the ability to lively debug on the server. Therefore, it was decided: a virtual server with Windows Server and remote desktop.
I want to convey a huge thank you to the patient and polite guys in tech support! So they did it.
')
Yes, yes, this is how it all looks. This printskrin with remote access to a virtual server (so do not consider it an advertisement for Windows Server 2012 R2).
Then you had to write the http responder. I wanted as simple as possible and not to get involved with IIS, it was necessary to meet a couple of days for development. But it turned out to be very simple to download the SimpleHttpServer example and to the function:
enter the desired processing. Hope we didn't violate any license.
And to those Web security specialists who now have hair on their backs from such a realization ... a huge hello and an invitation to do everything clever for us!
The recognition server works as a very simple http site. The user sends to the page a post-message in the format of http, which contains only one parameter - an image. In response, receives the result of recognition.
To query from the database, if there is a need for this, you need to send 2 lines: the car number in text form and a unique ID.
In the Android program there were 3 requests, their code is as follows:
1) sending a pre-allocated number to the server:
2) sending a request by number:
3) “swearing” to the number:
In my opinion, there is nothing to comment on. HttpPost file and HttpPost two text strings.
Do not forget that in the conditions of using the mobile Internet, you have to send an area with a pre-detected number using a cascade detector Haar.
An example of the Haar selection code using OpenCV on Android Java:
Here, notice an important trifle: after detecting a rectangle, the numbers of its border expand somewhat, since the detector can be mistaken with a scale with a certain probability.
And at the request of workers added http call on the function of search and recognition of the number in the whole frame: 212.116.121.70:10000/uploadimage
In response, you will receive a list of the numbers found and a certain recognition quality criterion for each (more is better):
x000xx99 90%
a111aa197 75%
lines separated by "\ r \ n"
Found 2 rooms, the first higher quality (90%), the second less (75%).
Now you can not select the image with Haar, but send the entire image at once. It is easier to organize automatic testing of algorithms.
On other platforms, the code should not be much more complicated.
A few words about the three days of the flight license plate server
Android program for swearing on cars Recognitor we posted on May 13th. My feelings are mixed: from pride in the fact that it works, to burning shame for the errors in the recognition algorithm, when a clear, clear number comes right before our eyes, but the user is returned with abracadabra.
Number of images sent to server: 1700
Of these, it turned out the numbers of the Russian Federation: 1370
Number of recognized: 830
(up to 10 specified)
Here it is worthwhile to separately explain “of them turned out to be the numbers of the Russian Federation”. We did not take into account that Habr is well read in the territory of the CIS and never indicated that the numbers should be the Russian Federation. Naturally, the errors of the not perfectly trained cascade detector, which was often mistaken in an unusual shooting situation from a monitor, also belong here. And there were several dozens of mirrored numbers, i.e. the user did not select “Flip” in the menu. Also, well, very smeared (not readable eyes), I also took it here.
In the intermediate result, the result is not fantastic, we made conclusions, we have already released 2 Android updates of the program, correcting the jambs and giving the user a new magic function to select the number area with a finger. Changed the algorithms on the server. That we have changed the interesting in the algorithms themselves, in my next article (we used a couple of alternative methods from my previous article ).
But, in spite of the non-ideal work, the users of the application liked it! Ratings on GooglePlay pleased.
And yes, of course, we will encourage the undisputed winners:
P494KE_197 - called 226 times (of course, this is ZlodeiBaal )
X777XX_77 - Called 21 times (in the top of the Yandex request for a “number” request)
They even caught A362MP_97, A231MP_97 and A869MP_97 (possibly also from the Internet).
In general, the algorithm was trained on very dirty winter rooms (and, paradoxically, it does not always work steadily on clean ones), so here its advantages are worth looking for. And yes, indeed, often blurry and very dirty numbers could be recognized:
References:
Part 1
Part 2
Updated Android project sources
Update:
1) It turns out that the cascade we trained on Russian license plates was locked up into the main version of OpenCV
2) when pre-allocating the number, we expected a rather large image in the uploadimage, now corrected, everything is reduced to the same scale. Should earn on small pictures from the Internet.
(part of the photos sent during the week)
I would also like to tell you about how we, programmers who use the nose of Internet technologies and Linux, solved the problem with the server.
All thoughts about the present noisy computer under the ear, pulling the cable into the kitchen and negotiations with the provider about the real IP, were rejected as they do not correspond to the new realities (they are only talking about cloud services and other innovations from all sides). But I also wanted the convenience of the familiar Windows, dotNET, and indeed the ability to lively debug on the server. Therefore, it was decided: a virtual server with Windows Server and remote desktop.
I want to convey a huge thank you to the patient and polite guys in tech support! So they did it.
')
Yes, yes, this is how it all looks. This printskrin with remote access to a virtual server (so do not consider it an advertisement for Windows Server 2012 R2).
Then you had to write the http responder. I wanted as simple as possible and not to get involved with IIS, it was necessary to meet a couple of days for development. But it turned out to be very simple to download the SimpleHttpServer example and to the function:
public override void handlePOSTRequest(HttpProcessor p, StreamReader inputData) { Console.WriteLine("POST request: {0}", p.http_url); string data = inputData.ReadToEnd(); p.outputStream.WriteLine("<html><body><h1>test server</h1>"); p.outputStream.WriteLine("<a href=/test>return</a><p>"); p.outputStream.WriteLine("postbody: <pre>{0}</pre>", data); } enter the desired processing. Hope we didn't violate any license.
And to those Web security specialists who now have hair on their backs from such a realization ... a huge hello and an invitation to do everything clever for us!
Access to server
The recognition server works as a very simple http site. The user sends to the page a post-message in the format of http, which contains only one parameter - an image. In response, receives the result of recognition.
To query from the database, if there is a need for this, you need to send 2 lines: the car number in text form and a unique ID.
In the Android program there were 3 requests, their code is as follows:
1) sending a pre-allocated number to the server:
HttpClient httpclient = new DefaultHttpClient(); final HttpParams httpParameters = httpclient.getParams(); HttpConnectionParams.setConnectionTimeout(httpParameters, 10 * 1000); HttpConnectionParams.setSoTimeout (httpParameters, 10 * 1000); // Http HttpPost httppost = new HttpPost("http://212.116.121.70/:10000/result"); InputStreamEntity reqEntity; httppost.setEntity(new FileEntity(new File(FileName), "application/octet-stream")); // try { HttpResponse response = httpclient.execute(httppost); HttpEntity responseEntity = response.getEntity(); ans = EntityUtils.toString(responseEntity); String[] strs=ans.split("\r\n"); if(strs.length>2) { ans=strs[0]; // timesWas=Integer.parseInt(strs[1]); // ID=strs[2]; // ID } } catch (ClientProtocolException e) { e.printStackTrace(); ans = "NOT CONNECT"; } catch (IOException e) { e.printStackTrace(); ans = "NOT CONNECT"; } 2) sending a request by number:
HttpClient httpclient = new DefaultHttpClient(); final HttpParams httpParameters = httpclient.getParams(); HttpConnectionParams.setConnectionTimeout(httpParameters, 10 * 1000); HttpConnectionParams.setSoTimeout (httpParameters, 10 * 1000); HttpPost httppost = new HttpPost("http://212.116.121.70:10000/checkplate"); InputStreamEntity reqEntity; try { httppost.setEntity(new StringEntity( editText1.getText().toString()+"\r\n"+ID)); HttpResponse resp = httpclient.execute(httppost); HttpEntity ent = resp.getEntity(); String ans = EntityUtils.toString(ent); timesWas=Integer.parseInt(ans); textView.setText(" : "+Integer.toString(timesWas)); } catch (ClientProtocolException e) { e.printStackTrace(); } catch (IOException e) { e.printStackTrace(); }catch(Exception e) { e.printStackTrace(); } 3) “swearing” to the number:
HttpClient httpclient = new DefaultHttpClient(); final HttpParams httpParameters = httpclient.getParams(); HttpConnectionParams.setConnectionTimeout(httpParameters, 10 * 1000); HttpConnectionParams.setSoTimeout (httpParameters, 10 * 1000); HttpPost httppost = new HttpPost("http://212.116.121.70:10000/swear"); InputStreamEntity reqEntity; try { httppost.setEntity(new StringEntity( editText1.getText().toString())); HttpResponse resp = httpclient.execute(httppost); HttpEntity ent = resp.getEntity(); String ans = EntityUtils.toString(ent); textView.setText(""); } catch (ClientProtocolException e) { e.printStackTrace(); } catch (IOException e) { e.printStackTrace(); }catch(Exception e) { e.printStackTrace(); } In my opinion, there is nothing to comment on. HttpPost file and HttpPost two text strings.
Do not forget that in the conditions of using the mobile Internet, you have to send an area with a pre-detected number using a cascade detector Haar.
An example of the Haar selection code using OpenCV on Android Java:
// if (mJavaDetector != null) mJavaDetector.detectMultiScale(temp, faces, 1.1, 10, 5, new Size(70, 21), new Size(500,150)); // Rect[] facesArray = faces.toArray(); for (int i = 0; i < facesArray.length; i++) { DetectedNum = new Mat(); IsNumDetected=true; // int dW=facesArra[i].width/5; // X 20% int dH=facesArray[i].height*3/10; // Y 30% int left = Math.max(facesArray[i].x-dW/2,0); int top = Math.max(facesArray[i].y-dH/2,0); int right = facesArray[i].x+facesArray[i].width+dW/2; if(right>temp.width())right=temp.width()-1; int bottom = facesArray[i].y+facesArray[i].height+dH/2; if(bottom>temp.height())bottom=temp.height()-1; // DetectedNum = temp.submat(BiggerRect).clone(); } Here, notice an important trifle: after detecting a rectangle, the numbers of its border expand somewhat, since the detector can be mistaken with a scale with a certain probability.
And at the request of workers added http call on the function of search and recognition of the number in the whole frame: 212.116.121.70:10000/uploadimage
In response, you will receive a list of the numbers found and a certain recognition quality criterion for each (more is better):
x000xx99 90%
a111aa197 75%
lines separated by "\ r \ n"
Found 2 rooms, the first higher quality (90%), the second less (75%).
Now you can not select the image with Haar, but send the entire image at once. It is easier to organize automatic testing of algorithms.
On other platforms, the code should not be much more complicated.
A few words about the three days of the flight license plate server
Android program for swearing on cars Recognitor we posted on May 13th. My feelings are mixed: from pride in the fact that it works, to burning shame for the errors in the recognition algorithm, when a clear, clear number comes right before our eyes, but the user is returned with abracadabra.
Number of images sent to server: 1700
Of these, it turned out the numbers of the Russian Federation: 1370
Number of recognized: 830
(up to 10 specified)
Here it is worthwhile to separately explain “of them turned out to be the numbers of the Russian Federation”. We did not take into account that Habr is well read in the territory of the CIS and never indicated that the numbers should be the Russian Federation. Naturally, the errors of the not perfectly trained cascade detector, which was often mistaken in an unusual shooting situation from a monitor, also belong here. And there were several dozens of mirrored numbers, i.e. the user did not select “Flip” in the menu. Also, well, very smeared (not readable eyes), I also took it here.
In the intermediate result, the result is not fantastic, we made conclusions, we have already released 2 Android updates of the program, correcting the jambs and giving the user a new magic function to select the number area with a finger. Changed the algorithms on the server. That we have changed the interesting in the algorithms themselves, in my next article (we used a couple of alternative methods from my previous article ).
But, in spite of the non-ideal work, the users of the application liked it! Ratings on GooglePlay pleased.
And yes, of course, we will encourage the undisputed winners:
P494KE_197 - called 226 times (of course, this is ZlodeiBaal )
X777XX_77 - Called 21 times (in the top of the Yandex request for a “number” request)
They even caught A362MP_97, A231MP_97 and A869MP_97 (possibly also from the Internet).
Good luck
In general, the algorithm was trained on very dirty winter rooms (and, paradoxically, it does not always work steadily on clean ones), so here its advantages are worth looking for. And yes, indeed, often blurry and very dirty numbers could be recognized:
References:
Part 1
Part 2
Updated Android project sources
Update:
1) It turns out that the cascade we trained on Russian license plates was locked up into the main version of OpenCV
2) when pre-allocating the number, we expected a rather large image in the uploadimage, now corrected, everything is reduced to the same scale. Should earn on small pictures from the Internet.
Source: https://habr.com/ru/post/223441/
All Articles