Automatic number image rotation for LPR systems

I decided to share a simple but effective way to rotate the image of the registration number of the car. To implement the idea, I use the .NET cross-platform wrapper over OpenCV - EMGU .

Post processing

When turning the numbers, we will use its horizontal edges. In order for the further algorithm to use them, it is necessary to visualize them. To do this, we apply a series of operations to the image:

public Bitmap SomeMethod(Image<Bgr, byte> img) { using (Image<Gray, byte> gray = img.Convert<Gray, Byte>()) using (Image<Gray, float> sobel = new Image<Gray, float>(img.Size)) { CvInvoke.cvSmooth(gray, gray, Emgu.CV.CvEnum.SMOOTH_TYPE.CV_GAUSSIAN, 5, 5, 25, 25); CvInvoke.cvSobel(gray, sobel, 0, 1, 3); CvInvoke.cvConvert(sobel, gray); // Image<Gray, float> --> Image<Gray, Byte> } return null; } 

')

Feature highlighting

The signs for this task are straight lines, and they must be more than half the width of the image. To select the line we use the method:

 public LineSegment2D[][] HoughLinesBinary( double rhoResolution, double thetaResolution, int threshold, double minLineWidth, double gapBetweenLines ) 

The first two arguments rhoResolution and thetaResolution set the desired resolution for the line in the binary image. Lines can be viewed as 2D histograms with intersection and angle of inclination, thus rhoResolution is assigned in pixels, and thetaResolution in radians. Threshold is the minimum number of pixels in the segments. When the pixel count is greater than the threshold, the segment is recorded in the list of found. The following two parameters, minLineWidth and gapBetweenLines , are the minimum long line and the gap between the lines are assigned in pixels.

Thus, to get the signs, use the following code:

 public Bitmap SomeMethod(Image<Bgr, byte> img) { LineSegment2D[] lines = null; using (Image<Gray, byte> gray = img.Convert<Gray, Byte>()) using (Image<Gray, float> sobel = new Image<Gray, float>(img.Size)) { CvInvoke.cvSmooth(gray, gray, Emgu.CV.CvEnum.SMOOTH_TYPE.CV_GAUSSIAN, 5, 5, 20, 20); CvInvoke.cvSobel(gray, sobel, 0, 1, 3); CvInvoke.cvConvert(sobel, gray); // Image<Gray, float> --> Image<Gray, Byte> lines = gray.HoughLinesBinary(1, Math.PI / 45, 50, img.Width / 2, 0)[0]; } return null; } 

Calculate the angle of inclination

In order to calculate the angle of inclination we will form an average value for the points of the obtained lines (since this is cheaper than calculating the angle for each):

  LineSegment2D avr = new LineSegment2D(); foreach (LineSegment2D seg in lines) { avr.P1 = new Point(avr.P1.X + seg.P1.X, avr.P1.Y + seg.P1.Y); avr.P2 = new Point(avr.P2.X + seg.P2.X, avr.P2.Y + seg.P2.Y); } avr.P1 = new Point(avr.P1.X / lines.Length, avr.P1.Y / lines.Length); avr.P2 = new Point(avr.P2.X / lines.Length, avr.P2.Y / lines.Length); 

And then we will complete the corner using the horizontal line:

  LineSegment2D horizontal = new LineSegment2D(avr.P1, new Point(avr.P2.X, avr.P1.Y)); 

Got the resulting angle:



Where C (horizontal), A - legs, B (avr) - the hypotenuse.
To calculate the sides of the triangle and the angle CB, we use the school formulas:

  double c = horizontal.P2.X - horizontal.P1.X; double a = Math.Abs(horizontal.P2.Y - avr.P2.Y); double b = Math.Sqrt(c * c + a * a); angle = (a / b * (180 / Math.PI)) * (horizontal.P2.Y > avr.P2.Y ? 1 : -1); 

Then just use the Rotate method for the image with the resulting angle.

 public Bitmap SomeMethod(Image<Bgr, byte> img) { LineSegment2D[] lines = null; using (Image<Gray, byte> gray = img.Convert<Gray, Byte>()) using (Image<Gray, float> sobel = new Image<Gray, float>(img.Size)) { CvInvoke.cvSmooth(gray, gray, Emgu.CV.CvEnum.SMOOTH_TYPE.CV_GAUSSIAN, 5, 5, 20, 20); CvInvoke.cvSobel(gray, sobel, 0, 1, 3); CvInvoke.cvConvert(sobel, gray); // Image<Gray, float> --> Image<Gray, Byte> lines = gray.HoughLinesBinary(1, Math.PI / 45, 50, img.Width / 2, 0)[0]; } if (lines != null && lines.Length > 0) { double angle = 0; LineSegment2D avr = new LineSegment2D(); foreach (LineSegment2D seg in lines) { avr.P1 = new Point(avr.P1.X + seg.P1.X, avr.P1.Y + seg.P1.Y); avr.P2 = new Point(avr.P2.X + seg.P2.X, avr.P2.Y + seg.P2.Y); img.Draw(seg, new Bgr(255, 0, 0), 1); } avr.P1 = new Point(avr.P1.X / lines.Length, avr.P1.Y / lines.Length); avr.P2 = new Point(avr.P2.X / lines.Length, avr.P2.Y / lines.Length); LineSegment2D horizontal = new LineSegment2D(avr.P1, new Point(avr.P2.X, avr.P1.Y)); img.Draw(new LineSegment2D(avr.P1, new Point(avr.P2.X, avr.P1.Y)), new Bgr(0, 255, 0), 2); img.Draw(avr, new Bgr(0, 255, 0), 2); double c = horizontal.P2.X - horizontal.P1.X; double a = Math.Abs(horizontal.P2.Y - avr.P2.Y); double b = Math.Sqrt(c * c + a * a); angle = (a / b * (180 / Math.PI)) * (horizontal.P2.Y > avr.P2.Y ? 1 : -1); img = img.Rotate(angle, new Bgr(0, 0, 0)); } return img.ToBitmap(); }