OpenCV. Search for road signs using contour analysis in Android

Hi Habr!

I want to share my own implementation of the road sign search algorithm.

Why contour analysis?
')
Contour analysis has a rather weak resistance to interference, but the simplicity and speed allowed us to apply this approach quite successfully.

However, in practice, it turned out to be quite difficult to implement the search for the necessary coefficients on the Android platform (I did not try to use OpenCV widgets, instead divided the screen into frames, where the settings are on the left, the video from the back camera is on the right). The specific implementation of the UI and project logic is available at the link below.

The sequence of road sign search operations is as follows:

Noise reduction

For example, through a nonlinear filter
```
cv::medianBlur(original, original, 3); 
```
Convert image from RGBA to HSV

When using the HSV color model, the base red color of the road sign is more accurately distinguished based on the color tone, saturation and brightness.

At the same time, speaking of the color tone, they usually mean color. Saturation indicates how much the described color is diluted with white (pink, for example, is a mixture of red and white). The concept of brightness is most difficult to describe, and with some assumptions brightness can be understood as the intensity of light.
```
 cv::Mat hsv; cv::cvtColor(original, hsv, cv::COLOR_RGB2HSV); if (layerType == LAYER_HSV) { *(cv::Mat*)matAddress = hsv; } 
```

Highlight road sign by color

During processing, the image is divided into separate channels H, S and V, which are subsequently binarized along a certain threshold and combined by logical addition into the final matrix of the image.

 std::vector<cv::Mat> channels; cv::split(hsv, channels); cv::Mat minHueThreshold = channels[0] < lowerHue; if (layerType == LAYER_HUE_LOWER) { *(cv::Mat*)matAddress = minHueThreshold; } cv::Mat maxHueThreshold = channels[0] > upperHue; if (layerType == LAYER_HUE_UPPER) { *(cv::Mat*)matAddress = maxHueThreshold; } if (layerType == LAYER_HUE) { *(cv::Mat*)matAddress = minHueThreshold | maxHueThreshold; } cv::Mat saturationThreshold = channels[1] > minSaturation; if (layerType == LAYER_SATURATION) { *(cv::Mat*)matAddress = saturationThreshold; } cv::Mat valueThreshold = channels[2] > minValue; if (layerType == LAYER_VALUE) { *(cv::Mat*)matAddress = valueThreshold; } cv::Mat colorFiltered = (minHueThreshold | maxHueThreshold) & saturationThreshold & valueThreshold; if (layerType == LAYER_RED_FILTERED) { *(cv::Mat*)matAddress = colorFiltered; }

Detection of the road sign on the outer contour

First of all, the operation of stretching is carried out, which eliminates noise and contributes to the unification of areas of the image that were separated by objects, shadows.

 cv::Mat colorDilated; cv::dilate(colorFiltered, colorDilated, cv::Mat()); if (layerType == LAYER_DILATED) { *(cv::Mat*)matAddress = colorDilated; }

To search for contours, the SimpleBlobDetector class is used, which is nothing more than a special case of the implementation of the findContours () function for round objects. It can find external and nested contours and determine their nesting hierarchy.

 cv::SimpleBlobDetector::Params params; params.filterByColor = false; params.filterByConvexity = false; params.filterByInertia = false; params.filterByArea = true; // A = 254.46900494077 px^2      9 px params.minArea = 255; // A = 723822.94738709 px^2      480 px params.maxArea = 723823; params.filterByCircularity = true; params.minCircularity = 0.85f; cv::Ptr<cv::SimpleBlobDetector> detector = cv::SimpleBlobDetector::create(params); std::vector<cv::KeyPoint> keyPoints; detector->detect(colorDilated, keyPoints);

In the keyPoints vector, the coordinates and the radius of the contour of the road sign are written, which are then highlighted on the screen.

Not to say that it works well in real-world conditions (perhaps, the detection algorithm is not the best, but it turned out to be the easiest to use). And there are difficulties in finding the optimal coefficients (there was plenty of "hunting" for road signs in my city; I even met signs that had faded from the sun, there was no red on them at all).

But simplicity is pleasing to the eye, so it is quite possible to find an application to contour analysis, for example, in robotics.

From the additional buns on the experience of using OpenCV in Android:

Rotate camera image if it is flipped

 extern "C" JNIEXPORT void JNICALL Java_ru_dksta_prohibitingsigndetector_ActivityMain_rotation(JNIEnv /* *env */, jclass /* activity */, jlong matAddress, jint angle) { CV_Assert(angle % 90 == 0 && angle <= 360 && angle >= -360); cv::Mat* mat = (cv::Mat*) matAddress; if (angle == 180 || angle == -180) { cv::flip(*mat, *mat, -1); } }

Implementing Salt & Pepper Noise

 extern "C" JNIEXPORT void JNICALL Java_ru_dksta_prohibitingsigndetector_ActivityMain_saltPepperNoise(JNIEnv /* *env */, jclass /* activity */, jlong matAddress) { cv::Mat* mat = (cv::Mat*) matAddress; cv::Mat noise = cv::Mat::zeros((*mat).rows, (*mat).cols, CV_8U); cv::randu(noise, 0, 255); cv::Mat black = noise < 30; cv::Mat white = noise > 225; (*mat).setTo(255, white); (*mat).setTo(0, black); }

Displaying video from a camera with processing in the 'picture in picture' mode

 if (secondView) { cv::Mat miniView = colorDilated.clone(); cv::cvtColor(miniView, miniView, cv::COLOR_GRAY2RGB); cv::resize(miniView, miniView, cv::Size(), 0.6, 0.6, cv::INTER_LINEAR); cv::Size miniSize = miniView.size(); cv::Size maxSize = original.size(); int startY = maxSize.height - miniSize.height; for (int y = startY; y < maxSize.height; y++) { for (int x = 0; x < miniSize.width; x++) { (*(cv::Mat*)matAddress).at<cv::Vec3b>(cv::Point(x, y)) = miniView.at<cv::Vec3b>(cv::Point(x, y - startY)); } } }

Writing text in OpenCV

 cv::Mat* mat = (cv::Mat*)matAddress; int textStartY = TEXT_LINE_HEIGHT; std::ostringstream output; output << std::setw(2) << std::setfill('0') << fpsCount << " FPS"; cv::putText(*mat, output.str(), cv::Point(TEXT_START_X, textStartY), FONT_FACE, FONT_SCALE, GREEN, TEXT_THICKNESS); output.seekp(0); textStartY += TEXT_LINE_HEIGHT; cv::putText(*mat, getLayerTypeDesc(layerType), cv::Point(TEXT_START_X, textStartY), FONT_FACE, FONT_SCALE, GREEN, TEXT_THICKNESS);

→ Link to project

Source: https://habr.com/ru/post/339506/

All Articles

OpenCV. Search for road signs using contour analysis in Android

More articles: