An algorithm is developed that effectively removes all the "boring" fragments from the video.
Did you come across a link to an interesting video on Youtube and find out that for just a few seconds, where something interesting really happens, you just spent a few minutes looking at a completely useless “garbage” just because the author posted the entire file from a DVR or smartphone? The number of video cameras is growing rapidly, and the number of people capable of at least cutting off a couple of unnecessary fragments seems to remain constant. And the problem is not only in a few minutes of time killed on the Internet - there are more serious cases, for example, tens and hundreds of hours of video from surveillance cameras, which sometimes have to be watched to solve a crime.
Scientists at Carnegie Mellon University have developed an efficient algorithm for extracting the most interesting video fragments based on machine learning. The new algorithm, called “LiveLight” by them, significantly exceeds its analogues in speed and quality of work. LiveLight highlights the characteristic fragments of the video and sotavlyaet their "dictionary", and then tries to predict the next frame based on them. If this is possible with a sufficient degree of accuracy, then this means that the frame does not add almost any new information and can be deleted. In contrast to the “mechanical” approaches that react to any movement in the frame or a drastic change in brightness, color or contrast, LiveLight is quite versatile - it works well on video taken with a fixed camera, and on amateur shooting with a shaking smartphone.
For testing the algorithm, 20 videos from Youtube and video surveillance cameras were selected, lasting from 12 minutes to one and a half hours. For each video, three people manually compiled a "summary" of the most interesting fragments. On average, in 72.3% of cases, the results of the algorithm coincided with the choice of people. For some commercials, the match was more than 90%. The result of LiveLight is 8% better than the nearest competitor, based on a similar principle, but 10 times slower.
')
LiveLight is able to process real-time video on ordinary hardware. Scientists tested the implementation of the algorithm in MATLAB 7.12 on a computer with Intel Core i7 processor at 3.4 GHz and 16 GB of memory. Some videos were calculated twice as fast as real time.
PDF with a detailed description of the algorithm can be downloaded from the link on the project page .
Scientists at Carnegie Mellon University have developed an efficient algorithm for extracting the most interesting video fragments based on machine learning. The new algorithm, called “LiveLight” by them, significantly exceeds its analogues in speed and quality of work. LiveLight highlights the characteristic fragments of the video and sotavlyaet their "dictionary", and then tries to predict the next frame based on them. If this is possible with a sufficient degree of accuracy, then this means that the frame does not add almost any new information and can be deleted. In contrast to the “mechanical” approaches that react to any movement in the frame or a drastic change in brightness, color or contrast, LiveLight is quite versatile - it works well on video taken with a fixed camera, and on amateur shooting with a shaking smartphone.
For testing the algorithm, 20 videos from Youtube and video surveillance cameras were selected, lasting from 12 minutes to one and a half hours. For each video, three people manually compiled a "summary" of the most interesting fragments. On average, in 72.3% of cases, the results of the algorithm coincided with the choice of people. For some commercials, the match was more than 90%. The result of LiveLight is 8% better than the nearest competitor, based on a similar principle, but 10 times slower.
')
LiveLight is able to process real-time video on ordinary hardware. Scientists tested the implementation of the algorithm in MATLAB 7.12 on a computer with Intel Core i7 processor at 3.4 GHz and 16 GB of memory. Some videos were calculated twice as fast as real time.
PDF with a detailed description of the algorithm can be downloaded from the link on the project page .
Source: https://habr.com/ru/post/228229/
All Articles