VLC stereo player

It all started with the fact that I bought a new monitor with support for ASUS VG23AH 3D and at some point I wanted to take the opportunity to watch a 3D movie on it. Podugul free form of players for Mac OS, it turned out they are not so many - all the more often came across the player Bino . I downloaded this Bino, but it didn’t look normal, i.e. There are no complaints to the player, the picture was 3D, but the video was terribly slow, the picture was twitching. I used the advice on improving the performance of this program - it says there - check how this film plays in the VLC player, if it slows down, it means your computer’s capacity is insufficient / replace the computer. In VLC, everything showed up perfectly without brakes but, of course, in mono mode. Since VLC is a opensource project, I decided to tweak it a bit so that it could show stereo movies, it turned out to be quite simple.

For a start, I made a fork of this repository and started reading documents and howto. I didn’t manage to build the project right away, there were conflicts with my mac ports installed, I had to demolish it. Then I had to do an up-merge for the extras / tools / ragel / aapl components, then everything was assembled according to the instructions .
Then the first thing that struck me was that it was possible to add plug-ins to VLC (these are modules that allow post-processing of decoded frames). Well, well, I thought that would be enough to convert a stereo image.

Retreat: the source file that I wanted to play had 2 angles in each video frame (on the left is a picture for the left eye, on the right - for the right) in general, a fairly standard format for a stereo film.

The processing came down to the fact that I formed the output video buffer line by line from the left and right half-frame, respectively. It was expected that the left eye would see odd lines, the right one - even.
It was easy to implement this filter (I read the instructions and took one of the existing ones as a basis). But, of course, it did not lead to success. On the screen, the picture looked striped, but the lines were not synchronized with the pixels of the screen, so the 3D effect in the glasses was not observed.
')
Retreat: I want to explain how 3D mode works on passive 3D monitors. I am not an expert in this technology, so I will describe as an amateur.
In monitors of my type, FPR (Film-type Patterned Retarder) technology is used to separate the angles of the video stream. I imagine it as if every line of pixels on the monitor is closed by a polarizing filter, it is conditionally possible to imagine that the odd lines are closed by the filter with vertical polarization, and even with horizontally, the same filters are wearing glasses, as a result one eye sees only odd lines, second even.

Having understood this, I realized that I need to do post-processing directly in the render so that I can guarantee pixel-by-pixel location of the output video frame.

I looked at how the render is implemented in VLC for Mac. It turned out pretty simple. Each decoded video frame is loaded as a texture, which is subsequently displayed on the screen by means of open GL. Converting a graphic format from YUV to RBG occurs directly in the pixel shader. This immediately gave me an idea - what needs to be done. Since it is known that the pixel shader just forms a color for each pixel of the screen, I just have to correct this shader so that it takes data from the desired part of the texture for even and odd lines when forming the color.

Here is a sample code (written schematically, a working example can be found here ):

void main(void) { vec4 x,y,z,result; vec4 tc = TexCoord0; + float d = mod((floor(height*(tc.y+1.0))),2.0); //odd or even, height - is new uniform to get viewport height + if(d>0.1) tc.x += 0.5; //shift texture x = texture2D(Texture0, tc.st); y = texture2D(Texture1, tc.st); z = texture2D(Texture2, tc.st); result = x * Coefficient[0] + Coefficient[3]; //Coefficient - const coefficients for convert YUV to RGB result = (y * Coefficient[1]) + result; result = (z * Coefficient[2]) + result; + if(TexCoord0.x > 0.5) result = vec4(0,0,0,1); //cut off right side gl_FragColor = result; }; 

If to compare with the source code, then I added only the lines marked +.

After that, I got the correct stereo image on the left side of the screen.
In the future, it remains only to correct the size of the image, and cut off the right side, and that’s all - the stereo player is ready.
Anyone who is interested - you can see the code on github .

In fact, in order to get a stereo player from VLC in total, I had to write about 30 lines of code, it struck me a lot, and was the reason for writing this article.

PS: Later, when my version was made, I found an excellent opensource stereo player - sview.ru , on MacOS it works well.