OpenCV: setting a timeout to wait for a frame in the VideoCapture class

Good day to all! There was a problem here somehow: play the RTSP video stream from the camera. Since I am quite familiar with the OpenCV API, it was decided to use it. To capture a video stream in OpenCV, the VideoCapture class is used. Unfortunately, the network quite often breaks with us, and this problem at my level is not solved, therefore, a quick response to the video stream drop became a prerequisite for comfortable work - the standard connection timeout and waiting for the next frame is 30 seconds, and inside the VideoCapture open ( ) and read () blocking, which makes writing around simple, in fact, code different wrappers like calling them in a separate thread and waiting for the result to be received in asynchronous mode. Naturally, I didn’t feel any joy about this - all these were resources that the program had to use for other purposes, not to mention the complexity of the code. It was decided: to change the standard timeout, or add the possibility of its external installation. It turned out quite dirty hack, which, however, may be useful to someone. Perhaps, if there is a better way - if there is one - I would very much like to know it, so please comment. Ideally, maybe among the readers of Habr, there will be OpenCV developers who will still pay attention to this problem. The goal was to make the code "work as it should, under Windows x64".



Who cares - I ask under the cat.



Research



With the help of Google, it was found that this problem rests on ffmpeg - there are callbacks that give information about the disconnection. A timeout of 30 seconds was set in pull-request # 6053. The problem was added in the following form: currently, the cmake-builder downloads the opencv_ffmpeg.dll file instead of assembling it in place, and the assembly instructions from ffmpeg disappeared. The code with timeout constants (which, at least in Windows is not compiled in any way) is in the modules / videoio / src / cap_ffmpeg_impl.hpp file:

')

#define LIBAVFORMAT_INTERRUPT_OPEN_TIMEOUT_MS 30000 #define LIBAVFORMAT_INTERRUPT_READ_TIMEOUT_MS 30000 

The task was thus formed as follows:

1. Make this file be built under Windows;
2. Change constants in it;
3. Make sure there are no problems when working.

Without thinking, I will talk about how it was solved. First you need to download the latest developer version of ffmpeg with header files, dll, and lib, and put it all in the right places in source / 3rdparty - for the sake of convenience, you can actually fold it anywhere. Next, you need to make the following changes to the OpenCV source files:



modules / videoio / src / cap_ffmpeg.cpp:



Comment out lines on potential inclusion cap_ffmpeg_api.hpp

 //#if defined HAVE_FFMPEG && !defined WIN32 #include "cap_ffmpeg_impl.hpp" //#else //#include "cap_ffmpeg_api.hpp" //#endif 

In the icvInitFFMPEG () function

 ... #ifdef WIN32... // ,    #elif defined HAVE_FFMPEG //  ,    #endif ... 

What happened here - we refused to use the ffmpeg functionality loading from opencv_ffmpeg.dll using LoadLibrary, and switched to the internal implementation, which is inside the cap_ffmpeg_impl.hpp file.



modules / videoio / src / cap_ffmpeg_impl.hpp:



Find the above constants, change them to the desired. In my case -

 #define LIBAVFORMAT_INTERRUPT_OPEN_TIMEOUT_MS 2000 #define LIBAVFORMAT_INTERRUPT_READ_TIMEOUT_MS 1000 

This file, in general, is not designed for building under Windows, therefore, some functionality required to compile it is absent in it - this is a snprintf. The code was pulled from somewhere with StackOverflow. Paste in any convenient place.

 #if defined(_MSC_VER) && _MSC_VER < 1900 #define snprintf c99_snprintf #define vsnprintf c99_vsnprintf __inline int c99_vsnprintf(char *outBuf, size_t size, const char *format, va_list ap) { int count = -1; if (size != 0) count = _vsnprintf_s(outBuf, size, _TRUNCATE, format, ap); if (count == -1) count = _vscprintf(format, ap); return count; } __inline int c99_snprintf(char *outBuf, size_t size, const char *format, ...) { int count; va_list ap; va_start(ap, format); count = c99_vsnprintf(outBuf, size, format, ap); va_end(ap); return count; } #endif 

After making changes, you need to force all this stuff to compile. First, you need to generate * .sln for OpenCV using CMake. Attempting to build opencv_videoio currently does not work. Fixable Open a Visual Studio project for the videoio module: build / modules / videoio / opencv_videoio.sln. We include ffmpeg header files in include directories, we add its * .lib to the link.



As soon as opencv_videoio will be assembled, this dll can be safely put in place of the “standard” one. Remember that now for the work will also need all the dll from the delivery of ffmpeg - without them the application will not work.



An important point: now, in order for this to work, you need to specify cv :: CAP_FFMPEG as the VideoCapture backend.



The result for the current moment is normal flight, I have not noticed any bugs in a month. However, taking into account all the above, they are more than possible, therefore, use only at your own peril and risk. If there are other ways to achieve the desired, as I said, I will be very happy to listen.



Thanks for attention.