A tool to quickly find scenes in video files

Prehistory

Once I argued with my friend about a movie. He argued that the main character says one thing, but I thought that he said the exact opposite. As a result, we argued for the N-th number of alcohol-soft drinks and went to a mutual friend who was the happy owner of a VHS video player and who was supposed to have this very film. I lost the argument, but I remember well that we spent a lot of time searching for this particular scene. Years later, nowadays, another friend of mine rewrote the video from the recorder to a flash drive with the words that they said there was a turn left to the lens of the device was supposed to get our mutual friend (an unpleasant type), who was in a wonderful state and offered me to pin up on this issue. To look for someone's physiognomy on a video of a couple of hours is, of course, not really looking a needle in a haystack, but, too, a pleasure for an amateur. It seems that it was then that an idea began to appear in my head and I wondered how to watch a few hours of video recordings in order to find there what you need and at the same time not to limit yourself in a dream much?

main idea

As a result of reflection, I came to the conclusion that the easiest way to search is to use a tabular presentation of the video: each cell has its own time interval.

Table video presentation: what kind of animal is it and what does it eat?

Examples of video in tabular format

Candies
Parking
Road

Examples of images in tabular format

Candies
Parking
Road
')

Actually, the tool itself

After some torment, I wrote the program Table View Video Player .

Main features of the program

• Playing video in tabular form
• Ability to save video in tabular form (for viewing on a tablet or simply on another computer)
• Customizable number of rows and columns in the table, as well as the size of the resulting frame in pixels
• List of files (for fast switching between videos, and also if the event consists of several files, for example, VOB files on a DVD or recordings from a video recorder)
• Saving an index image
• Batch create video and index images for all files in the list
• Some transformations of the original video during playback (rotation 90, 180, 270 degrees, the reflection of the vertical and horizontal)
• Ability to select a specific area of ​​the original video
• The ability to play the original video in normal mode with a given position in the built-in player or in another program

And what's under the hood?

The program is written in C ++, MS Visual Studio 2015 was used. FFMPEG could be limited to video libraries, but I used OpenCV (I plan to use other features of this library).

Scope of this tool

I will try to show everything by example. I put a video camera in the kitchen, and I hid candy on the kitchen wall cupboard. The duration of the video is 2 hours. And now look at the picture and tell me who ate them: dad, mom or some of the children? Even if there were no highlighting of the necessary areas, everything would become clear after 2 minutes of viewing, and maybe even earlier. If someone wants to conduct an experiment, then here is the link to the original video , and here is the link to the video in tabular form . So, the main purpose of the program is to find out who has stolen the candy from visual video analysis .

More practice

On holidays, I shoot videos in kindergarten, as well as family anniversaries, weddings of friends and relatives. Pinnacle Studio - an excellent program for video editing! They even say that housewives can easily figure it out! I don’t understand the truth, where do housewives have so much perseverance and free time to spend hours on repeated attempts to export the edited video to a file or to a DVD ? I myself have been using this program since version 8 (I am using the licensed version 17 now, and the last version released is version 19), but, despite my experience, it rarely happens that the application does not hang during the editing or export process. But even if the export was completed successfully (according to Pinnacle Studio), then when watching a video in a VLC-player or a Windows-Media player, there are sometimes artifacts that look confusing to users of my products. Well! Table View Video Player can be used here! We start the program, open our file and, without concentrating on any fragment, just try to feel where suddenly a frieze or some sort of byaka appears. A type of this. Five minutes to search for video artifacts is not so much! In this case, re-open the project and try to fix the problem. And a little rejoice that they discovered it themselves, and not “ happy parents ” during family viewing.

Step back

Some time ago I boasted of my program to my friends, and I received a comment that the program was too complex and he would never understand it personally. Mdja, and I thought that ... Having considered the constructive criticism, I realized that, probably, he is right. So it became necessary to create a simpler (Lite) version.

Main features of the program

• Playing video in tabular form
• Resizable table sizes
• List of files (for fast switching between videos, and also if the event consists of several files, for example, VOB files on a DVD or recordings from a video recorder)
• The ability to play the original video in the normal mode with a given position in the built-in player.

Compared with the standard version, many features have been removed, but the program has become as simple as 2 kopecks !

Theory of the search for scenes and objects

Existing automatic solutions for searching through the video archive are resource-intensive, besides it is not always possible to clearly state what you are looking for. In this case, the use of manual search (I mean with the help of the operator), if you speed it up using table playback, can be very effective.

Strengths

• Search for some events in a relatively static picture. For example, record visitors in the lobby. If there are not so many of them, then finding the right person is pretty easy. Here the search can be accelerated dozens of times!
• Search for scenes with a contrasting color gamut or light in a dynamic video. For example, there is a film shot mostly indoors, but there is one or more scenes shot outside on a clear sunny day. It is required to find them. In this case, they are very fast and in person.

Weak sides

• Search for specific people in the crowd. For example, in the metro at rush hour. Here, a table view does not help, it is better to watch the video in normal mode (less likely to skip what you are looking for) or use specially sharpened automatic tools.

Competitors or programs for searching scenes and objects

I cannot name direct competitors, but there are several programs that do something similar.

• Fast Video Indexer
• Vindex
• Video Cataloger

The list is far from complete, some of the links can not find. But I would be grateful for the addition ...

Future plans

• Speed ​​up video playback. Already, video in VGA resolution on a modern computer can be viewed in a 10x10 table with a frequency of 20-30 FPS. However, video with modern photo and video cameras, action cameras and video recorders has a resolution of Full HD or even higher and just around the corner 4K. Therefore, without using a GPU is not enough.
• The ability to save the found information in the form of images (still image), audio or video fragments of interesting moments for later use. For example, to edit a video (we collect funny plots from a party) or to present evidence in court (if the video surveillance camera captured the moment of theft).

findings

• Tools for finding scenes and objects in the video today for the average user almost does not exist.
• Table View Video Player and Table View Video Player Lite try to fill this niche.
• For certain types of searching scenes in video, the table view method is very effective.
• Despite the fact that simultaneous playback of several dozen video streams requires high computer performance, with a low resolution (SD quality) the original video task is successfully solved by modern PCs.