The elusive problem of frame timing

Technical director of Croteam Alain Ladavach, who participated in the development of Serious Sam and Talos Principle, tells how he managed to find the reason for the inhibition of graphics even on the most powerful machines.

Finally, an explanation has appeared of why some games slow down on your PC (and a ray of hope that they will stop slowing down in the near future).

Tt-brakes

You were looking forward to the next part of your favorite PC video game series and it finally came out. This time you want to enjoy it in its entirety, so you spent money and time on thorough preparation. You replaced the processor, installed a state-of-the-art video card, added more RAM — damn it, even bought a RAID on the SSD. The game should be smooth with the screen saver.
')
Pre-order is finally unlocked and you have just completed the installation. In a nervous anticipation, you start the game for the first time. So far so good - it works at 60 frames per second. Or, at least, this is what the GPU tuner frame counter reports. But something is wrong. You make the mouse sharp, chaotic movement. Straight left-right, and then the game ... begins to slow down! Damn, how is this possible? How can she slow down at 60 frames per second?

If this has never happened to you, it may seem ridiculous. But if you have experienced them, you will most likely hate brakes with all your heart. Brakes in games. This is not the good old "lag". Not a low frame rate. These are just “brakes” occurring at high frame rates on ideal, super-fast machines. What is it, where did they come from and how to get rid of them? Let me tell you a story ...

Brakes, smoothness, speed ... are they the same thing?

Video games have operated at 60 fps since the days of the first arcade machines in the 70s. Usually it is expected that the game works with the same frequency that is used by the display. This was the case before the popularization of 3D-games, in which for the first time reduced frame rates became permissible. In the 90s, when 3D maps (as we called them before they became “ GPUs ”) began to replace software rendering, people played games at 20 fps, and 35 fps was considered a decent value for serious network battles. I'm not joking .

Today we have super-fast cars and "of course, they can work at 60 fps ." However, the number of users disappointed with the speed of games is greater than ever. How is this possible? The problem is not that games can not work fast enough, but that they slow down, even when they can work fast !

If you read different game forums, you will surely find similar messages:


You might think that this is a single problem, but look at the statistics of Google search queries:


Over the past five years, brakes (stutter) have become (relatively) more serious problem than speed!

(Note that these are relative values. They do not mean that, in general, people ask for braking more than about frame rates. They mean that requests for frame rate remain at the same level, and the number of requests for brakes is growing especially lately.)

Ten years of searching for the cause of unexplained brakes

The patient is more alive than dead, just slows down a little more than necessary.

I first encountered this problem around 2003. We worked on Serious Sam 2 , and users began to send us reports that they were testing something at an empty level, and the movements of the mouse were not smooth. This was accompanied by a very characteristic pattern on the frame rate graph, which we called the "cardiogram".

We thought that there was a bug somewhere in the code, but we could not find it. It seemed that the problem appeared and disappeared randomly - after the application was restarted, the machine was restarted ... and then the player changed some speed option and it disappeared. Then the player turned on this option again, but the problem did not return. She looked like a ghost.

Obviously, this problem arose not only in our country. Watching the same problems in other games, we began to think that the drivers were to blame. But this happened on video cards from different manufacturers. Even on different APIs (OpenGL, DirectX 9, DirectX 11 ...) - the only common thing they had was that they appeared on different machines, in some scenes ... sometimes.


Nessie, Bigfoot ... almost as elusive as the problem with the cardiogram.

We released a few more games, but this strange behavior still appeared and disappeared. It irritated some users, and we recommended them to change the speed options - sometimes it helped, sometimes it did not. Such is life, isn't it?

But one day, on a great winter day at the beginning of 2013, my colleague Dean called me to see another example of this problem, which he at that time could reproduce relatively stably. This time the problem arose at the level of Serious Sam 3 . We experimented with options in this scene, until it suddenly came to me. I understood what was the reason! And she was very simple - no wonder she escaped everyone for a decade.

By changing just one very simple game engine option, we were able to make this problem appear and disappear in this particular scene. But it became immediately obvious to us that much better efforts would be required for its high-quality solution. Efforts not only from our side, but also the entire PC gaming ecosystem — programmers for GPU drivers, API developers, OS vendors — everyone.

Let me explain.

What was the reason all this time

I would like to show you on the example of a scene from Serious Sam 3, which Dean and I explored five years ago. Or even better - using the example of a test scene from Serious Sam 2, in which we first saw it. But, unfortunately, after replacing the “iron”, this elusive beast may move to another scene. I have a scene from The Talos Principle , in which I recently managed to reproduce this problem, and I filmed several videos that made it possible to analyze it in more detail.

But before we start, make sure you actually watch the video at 60 fps. To view the examples below, switch to 1080p60, as shown in the picture:


To watch videos at 60 fps, switch to YouTube at 1080p60.

If you do it right, and your computer and web browser are capable of showing video at 60 fps, then the video below should be played smoothly and without any brakes. If this is not the case, then this is why we are talking about this — many other applications also demonstrate this behavior, not just games. For now, I can only recommend that you try to watch the video on some other machine, or just read the text.


Check, check, one, two, three ... You should see this video in smooth 60 fps.

And now let's get down to business. If you encounter brakes, then most likely it looks like this:


« 60 fps». «».

, «», 60 fps. - , , « ». , ( , - «» ). « », , - , . 60 fps, « ». ( , , .)

(«»), YouTube . (), . , . , , …

---

, ? ? , ?..

, , , !

?

. «», 1/20 , :


60 fps, «». 20 .

: -, — , . -, - - — «» , , .

: — — — «»… «» . , ?

(, ( 1/10 000 ), : , …


. — , — «».

… , «» ( ). , , , ( ).

… «», « », ?

, , . ( , , , .)

-, - … , , . NTSC, 60 , 60 fps, PAL/SECAM, 50 , 50 fps. - « ».

, — , " ", ZX Spectrum, C64, Atari ST, Amstrad CPC 464, Amiga .. , , , 100% , .

«». , , . , Sonic The Hedgehog Sega Genesis , , 16 . PAL NTSC, 50 fps 60 fps. , .

— , PC «» — , . , . 3D-, , . , — , … . , .

, . ( , «» - , .)

, , . 1/60 (16,67 ), 10 /, 1/6 . 1/60 , 1/30 (33,33 ), 1/3 ( «») , .

? , . , . , . 90- ( «35 fps » ), . ( , GPU) «» «», . 3D-, . , .

GPU, "". , GPU - , GPU , , GPU . , GPU, « », GPU . - . — - . , , .

, , , … . . , :


. — , — «».

, . 16,67 ( 1/60 ), , , . () , 24,8 ( 1/60 ), , … , , 10,7 , , - . ( .)

: , , , , PC. , 60 fps, - , . - GPU 60 fps .

— , («») .

, , — , .

. ( , — , ?), :


«», , .

«». «» , !

? Serious Engine sim_fSyncRate=60. , « , 60 fps». — , ! , — , .

? ?

?

— . . , 60, 60 — PC : - , GPU/… — .

, , , — - . ?

, / , .

, ? , — !

, . , API. , , , . , , API , : , . , , .

?

, . API. Vulkan API VK_GOOGLE_display_timing, proof of concept. , Android Linux.

API. ? , .

, Croteam , , .

, , The Talos Principle .

. « », . , , , , .

«»

? , . - , ?

« », . , , , , , Skype .. 3D. , , . .

. , — ?

, GPU , , . , GPU . , , . , .

…

. :) GPU . , GPU . , .