Simple motion recognition algorithm

When I started working on my master's thesis on the topic “Analysis of the spatial structure of dynamic images”, I faced the problem that it is very difficult to find some ready-made examples of pattern recognition algorithms and moving objects. Everywhere, both in literature and on the Internet, there is only one bare theory. The purpose of writing this article just to fill this gap.

So, for the experiment, I took a short video out of the apartment window, broke it into frames and saved a couple of frames in the form of pictures:




')
To determine the fact of movement, I decided to go a simple way so far: turn the images into a matrix, subtract the first matrix from the second matrix (elementwise). Here is the C # code that performs this processing:

/// <summary> ///   /// </summary> /// <param name="file_name">  </param> private ImageMatrix crate_matrix(string file_name) { Bitmap picture = new Bitmap(file_name); ImageMatrix res = new ImageMatrix(); using (var wrapper = new ImageWrapper(picture,true)) { res.matrix = new int[wrapper.Width, wrapper.Height]; for (int i = 0; i < wrapper.Width; i++) { for (int j = 0; j < wrapper.Height; j++) { Color color = wrapper[i, j]; res.matrix[i, j] = (color.R + color.G + color.B) / 3; } } res.height = wrapper.Height; res.width = wrapper.Width; res.picture = picture; } return res; } private void tsmiDifference_Click(object sender, EventArgs e) { ImageMatrix matrix1 = crate_matrix("D:\\3\\1.png"); ImageMatrix matrix2 = crate_matrix("D:\\3\\2.png"); Bitmap picture = new Bitmap(matrix1.picture); using (var wrapper = new ImageWrapper(picture, true)) { for (int i = 0; i < wrapper.Width; i++) { for (int j = 1; j < wrapper.Height; j++) { int light1 = matrix1.matrix[i, j]; int light2 = matrix2.matrix[i, j]; int light = Math.Abs(light2-light1); wrapper[i, j] = Color.FromArgb(light, light, light); } } pbImage.Image = picture; } } 

And that's what happened during the execution of this algorithm:



In the resulting matrix, we see a car's printed outline and some other outlines in the background, which most likely arose from hand shake and from the rain at the time of the shooting. The rain is not visible in the photo, but it is clearly visible in the video.

In order to improve the quality of recognition, you can enter a certain threshold value into the program:

if (light <50) light = 0; else light = 255;

The result will then be completely different:



As we see, with the help of simple subtraction of matrices and using the threshold value, we were able to identify a moving object in a dynamic picture. What to do with the received data next? In fact, what we have now is a binary image. There are some spots in it that define a moving object. Noises, as we see, our algorithm is a bay. In the future, you can determine the coordinates of the detected areas of motion. True, we have several moving objects that turned out - each spot when trying to determine its coordinates will be a separate object. But if we thus process several matrices, then we can notice that several objects move synchronously, which allows us to relate them to a single object.

However, the algorithm given here is not the only way to determine motion. If you like my article, I will continue the topic and talk about other motion recognition and pattern recognition algorithms.