Input delay on retro consoles and emulators

TL; DR The article describes the well-known input delay problem (input lag), which manifests itself when trying to play old games on modern hardware: emulators of retro consoles, bluetooth gamepads, etc. Sometimes the delays are so big that it becomes impossible to play. I will describe my journey of finding an acceptable configuration for running my favorite games.

With the wave of popularity of single-board computers RaspberryPi, OrangePi, many friends bought them in batches. Not thinking up what to do with them, they began to sculpt retro-consoles from them based on the RetroArch emulator and the Recallbox distribution. When I tried to play my favorite childhood games on it, I was surprised: " How could I play it? " The physics of games seemed somehow wrong, a disgusting feeling. Some time later, I was told that the whole thing was to delay the input, which at first glance does not feel like a delay, namely, like other physics.
')
It turned out that the problem has long been known among console geeks and it is being actively investigated. I will analyze other people's research and try to measure the delay of my setup.

What is the problem?

The input delay is the time from the moment you press the button on the gamepad to the reaction of the character on the screen. Many factors play a role here:

• Monitor / TV delay - some modern TVs have an output delay of more than 100ms. This is due to post-processing of the image, buffering, filters, etc. This clearly shows the comparison of CRT and LCD TV with high latency. Typically, manufacturers of monitors indicate in the specifications of the speed of the matrix, referring to the speed of switching between colors, but this is not related to the delay in image output.
  
  
  One analog signal from the set-top box on two televisions. There is a significant delay on the LCD TV compared to the CRT.
  
• Computer latency - a game console or a computer with an emulator doesn’t process the signal immediately, it takes time. The computer needs to receive a signal from the input device, process it and display the image on the screen. The delay also introduces vertical synchronization of the image when displayed on the screen. Here RetroArch developers are trying to solve this problem.
  
  
  Demonstration of a big delay on the emulator. Video Denis Major
  
• Controller delay - the input device can also introduce a delay. For example, if connected to a loaded USB bus, via Bluetooth, etc.

Emulators

Everyone wants to play on modern hardware, normal TVs and monitors with HDMI, and still run all possible games on one device. For this, emulators of popular consoles are used, which run on ordinary X86 or ARM computers. Here are the most popular ones:

Retroarch

The most popular and advanced emulator. Able to emulate PlayStation1, SNES, NES, GameBoy, Sega Genesis / CD and other consoles. Works on desktop OS Windows, Linux, MacOS and on modern consoles Xbox, Android, PlayStation2 / 3/4 / Vita, Nintendo Wii / Switch and others . On its basis, the popular distribution kit Recallox.

OpenEMU

Very convenient and simple emulator for macOS. It supports most popular consoles and has a convenient gallery of games, sorted by platform. Out of the box supports gamepads from Playstation 4 via bluetooth. It is inferior in its capabilities to RetroArch, but for my taste it is much more convenient to use. For my measurements, I will use it, as it immediately works on macOS without perdoling with gamepad settings. ⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀

Measurement Technique

The most affordable way to measure the delay is to record on the video screen and when you press the joystick button. Modern smartphones easily shoot video at 240 FPS and more, which is quite enough for measurements. To accurately fix the moment of pressing, you can solder the LED to the contacts of the button, which will light up at the moment of pressing. The LED lights up in nanoseconds, which allows you to accurately record the moment of pressing.

Brunnis from the forum project libretro conducted a large study of the problem of input delay. His research is the most complete and objective.


USB gamepad with a soldered LED for fixing the moment of pressing a button


Button not pressed


Button pressed


Start of animation on the screen after 24 frames (100 milliseconds)

Calculation formula

In a second of 1000 milliseconds.
The phone camera shoots at 240fps.
This means 1 frame of video = 1000/240 = 4.16ms.

It is also important to consider the frame rate at which the monitor and the computer are running. For example, for 30fps mode, one frame of the monitor will be 33ms, and for 60fps - 17ms. This should be taken into account, since the time between frames cannot be less than these values, which means that events occurring during the interval of frame updates on the monitor will be rounded to the last frame.

I will not give here all the results of measurements of the dude, who are interested in offering to get acquainted yourself.

The main thing you need to remember: the original console had an average delay of 50 to 70 ms, depending on the console and the PAL / NTSC mode. A well-tuned emulator on a PC adds 60-80ms of delay, depending on the emulated platform. The emulator on raspberry pi can have up to 150ms delay in total.

My searches

In my childhood I had a Sega Mega Drive, it was her games that caused the most nostalgia. Therefore, I will describe my search for a comfortable setup specifically for this console. Just want to say that my research does not pretend to absolute measurement accuracy.

My requirements:

• Wireless gamepads - maybe console geeks will condemn me, but I want to play comfortably while sitting on the sofa in any position. Wired gamepads are terribly uncomfortable, everyone hurt about them and you will not play for a long time.
• Output via HDMI or DisplayPort - there are no analog inputs on my monitor, so I cannot buy the original Sega Mega Drive. USB adapters for analog video capture make their delay and inconvenient. I don’t want to buy a separate CRT TV for the console either

Macbook + Dualshok 4 + OpenEMU

I really like the Playstation 4 gamepad, I find it the most convenient gamepad available. It can work both via bluetooth and USB. Fully supported in OpenEMU out of the box without settings.


Despite the convenience of OpenEMU, its main problem is the FPS drawdown in full screen mode. I am not sure what the reason is, and perhaps this is being corrected, but I did not succeed. Therefore, I play in windowed mode.


OpenEMU emulator interface. All games are sorted by platform.

Measurements

I did not want to peel the gamepad from the Playstation in order to solder the LED, so I had to do the counting of the moment when the button was pressed by the sound. The error of this method is about 8-10ms. The monitor uses the LG 24UD58 in 4K @ 60fps mode connected to the MacBook via DisplayPort. Contra Hard Corps game.


The delay on the MacBook with the Dualshock4 controller connected via bluetooth is approximately 150ms. It's a lot!

MacBook + Dulashock gives approximately 150ms of delay. This is quite a lot for a comfortable game, this delay is noticeably felt in hardcore platformers like Contra Hard Corps to play uncomfortable.

Despite the fact that the delay of such a bundle is unforgivably large, it still deserves the right to life. Since it is enough to throw a gamepad in a ryuzkak to a laptop, and get a game console anywhere. Even before the measurements, it was clear to me that this is not the best option and I continued the search.

Clone sega genesis

Many different pirate clones of retro consoles with pre-installed games are on sale now. Some of them are made from set-top boxes on Android with console emulators inside, which means they suffer the same problems with delays. But some promise a full console without an emulator . I found on Avito a mower just such a console: retrogenesis.ru model Retro Genesis HD Ultra. On the site they write that this is not an emulator, but inside it is really a hardware console. There are pre-installed games and the ability to insert cartridges. At the same time there is a HDMI output and wireless gamepads!

Wireless gamepads

Most of all I was strained gamepads operating at a frequency of 2.4GHz. Knowing the problems of bluetooth and WiFi on this range, I first tried to measure the delay of the gamepads themselves. The console supports simultaneous connection of wired and wireless gamepads. It turned out that the wireless part was made using a separate module that closes the contacts in the same way as a wired gamepad. That is, in fact, emulates the usual wired, with the same pinout.


The radio module that emulates pressing a wired gamepad

I connected the probe 1 of the oscilloscope to the LED on the gamepad, and the other to the contact corresponding to the Up key on the console. When the signal is processed by the radio module, it will simulate a key press, and I will see it on the probe 2. By comparing the time difference between signals 1 and 2, I was able to very accurately get the delay that the wireless gamepad adds. On an oscilloscope, the scale of a single cell is 5 ms, which means the delay of 25 ms wireless gamepads.


Comparison of signals on the gamepad and console


Wireless module adds 25ms delay

Measurements

Delay on Sega Retro Genesis approximately 70ms

Less 25ms delay of the radio module, it turns out that the console itself has a delay of 50ms, which corresponds to the original Sega. When connecting wired gamepads, the delay will obviously be less by 25ms. In general, the gameplay is felt like on the original console, and I am satisfied. The only thing that makes me sad is the impossibility of downloading my games (ROMs), but I will try to solve this problem with a rewritable cartridge, or try to find an opportunity to upload games on the built-in USB flash drive myself.

Conclusion

My measurements do not take into account the latency of the monitor output, but all measurements are made on the same monitor, so it does not affect the results. Each measurement I repeated 10 times, sometimes getting a variation in the readings of about 4 frames, that is 16ms. I believe that the matter is in the delay of vertical synchronization in the formation of the image on the console.

In general, I believe that my measurements give a general idea of ​​the delays in the two setups and may be useful to those who choose their own way.

Runahead option on RetroArch

A year ago, the developers of RetroArch stated that they had achieved less delays on their emulator than on the original consoles. I myself have not tried this option, but it must be borne in mind.

Links

Thanks to Denis Major for his research. Thanks to this video, I learned for the first time that the problem is not in the modified physics of games, but in the input delays.

Large study of delays on different platforms .

Another study .

About Hackspace Neuron

In our Hackspace Neuron, we regularly play retro consoles, reverse iron and do many other things. Here you can use a soldering iron, oscilloscope, frequency analyzer and chat with fun zadroty. Come to visit.

All news about the life of Hackspace in the telegram channel: @neuron_news