VLC based video player. Part 2: use Direct3D

Introduction

Hello to the habra people!

In my previous article, I began to talk about the development of a player based on VLC. In this article I will talk about the main aspects and difficulties that I had to face when writing a player. Here is the main window of the program:


Main program window
')

Frame rendering to screen using Direct3D

So, as I have already said, the LibVLC library, which is the main part of the VLC player, is used to decode streaming video. The VMem plugin allows you to decode a video stream by applying various filters to it and ultimately save it to user memory. In order to speed up the output of the picture, I used Direct3D 9. In Direct3D, there is the concept of surfaces (surfaces) and also textures (textures). The surface is a byte array having a width x height size:


Schematic image of the surface

Those. essentially the surface is a bitmap
The texture differs from the surface in that it can be “superimposed” on primitives (polygons). Also, the texture has mip-map levels - reduced copies of the texture (used in 3D graphics)

To display a movie frame on the screen, we need to create an off-screen surface, pass a pointer to the memory area to the VMem plugin, then copy the surface into the texture and draw a rectangle with a texture stretched over it. Schematically it can be represented as follows:


Drawing process

Intermediate copying of a surface into a texture is necessary due to the peculiarity of access to surfaces and textures in Direct3D.
The numbers in brackets when drawing are the texture coordinates, they show how the texture will be superimposed on the primitive.

The problem of multithreading and window stretching

When stretching the back buffer window, the one used when drawing remains the same size, and therefore when resizing the window, the image is stretched and blurred. To prevent this from happening, you need to recreate the buffer. In Direct3D, this is done by calling IDIrect3DDevice9 :: Reset. In this case, a prerequisite for the successful re-creation of the buffer is the removal of all resources from the video memory. And since LibVLC works in a separate stream and writes data to our video surface, it is necessary to synchronize with mutexes or sections.
You also need to solve the issue of rendering. Because besides rendering the video, there were elements in the application that needed to be animated at a frequency of> 25 frames per second. This can be implemented as follows: GUI elements when drawing cause a forced redrawing of the window as well as a video stream. Either the window is constantly redrawn when the application is idle (as is done in games). In the first method, the stream that decodes the video should send messages to the main stream that the application needs to be redrawn, which additionally loads the message queue. With the second method, no need to send messages - the application is always redrawn.

User interface

All buttons and other controls are drawn by outputting translucent primitives with textures stretched over them. Each element has a bounding box, when clicked, in which a “attached” slot is invoked with the desired action.
For channel transmission technology was used "kinetic scrolling":


Scroll with “kinetic scrolling”

The scrolling technology is quite simple, especially on the Internet there are many articles on the implementation of this effect.
There is nothing more to say about the user interface, except that I had to break my head over some fading effects.

Conclusion

No matter how hard I tried, but the presentation of my thoughts turned out to be rather messy - I wanted to talk about many things, but at the same time not to write a whole talmud about how to write applications on DirectX. If you have any questions or suggestions, I will be glad!
PS (edit:) initially the pictures were uploaded to the dropbox, which led to the account being blocked. The source files were not saved, so I redraw the schema.