Porting Quake3

In the Embox operating system (of which I am the developer) some time ago support for OpenGL appeared, but there was no sensible performance check, only drawing scenes with several graphic primitives.

I have never been particularly interested in game devs, although, of course, I like the games, and decided that this is a good way to have fun, but at the same time check OpenGL and see how the games interact with the OS.

In this article I will talk about how to build and run Quake3 on Embox.

More precisely, we will launch not Quake3 itself, but ioquake3 based on it, which also has open source code. For simplicity, we will call ioquake3 just a quake :)

At once I will make a reservation that the article does not analyze the Quake source code itself and its architecture (you can read about it here , there are translations in Habré ), and in this article we will focus on how to ensure the launch of the game on the new operating system.

The code snippets cited in the article are simplified for a better understanding: missing error checks, using pseudo-code, and so on. The original source can be found in our repository .

Dependencies

Oddly enough, not many libraries are needed to build Quake3. We will need:

• POSIX + LibC - malloc() / memcpy() / printf() and so on
• libcurl - networking
• Mesa3D - OpenGL support
• SDL - support for input devices and audio

With the first paragraph, and so everything is clear - it is difficult to do without these functions when developing in C, and the use of these calls is quite expected. Therefore, support for these interfaces is in one way or another practically in all operating systems, and in this case almost no functionality was added. That had to deal with the rest.

libcurl

It was the easiest. Libc is enough to build libcurl (of course, some features will not be available, but they will not be required). Configuring and building this library is statically very simple.

Usually both applications and libraries are linked dynamically, but since in Embox, the main mode is linking into one image, we will link everything statically.

Depending on the build system used, the specific steps will differ, but the meaning is something like this:

 wget https://curl.haxx.se/download/curl-7.61.1.tar.gz tar -xf curl-7.61.1.tar.gz cd curl-7.61.1 ./configure --enable-static --host=i386-unknown-none -disable-shared make ls ./lib/.libs/libcurl.a #       

Mesa / OpenGL

Mesa is an open source framework for working with graphics, a number of interfaces are supported (OpenCL, Vulkan and others), but in this case we are interested in OpenGL. Porting such a large framework is the topic of a separate article. I will confine myself only to the fact that Embox Mesa3D already exists in OS :) Of course, any implementation of OpenGL will work here.

SDL

SDL is a cross-platform framework for working with input devices, audio and graphics.

So far, we are slaughtering everything except graphics, and for frame rendering, we will write stub functions to see when they start to be called.

Backends for working with graphics are specified in SDL2-2.0.8/src/video/SDL_video.c .

It looks like this:

 /* Available video drivers */ static VideoBootStrap *bootstrap[] = { #if SDL_VIDEO_DRIVER_COCOA &COCOA_bootstrap, #endif #if SDL_VIDEO_DRIVER_X11 &X11_bootstrap, #endif ... } 

In order not to bother with the "normal" support of the new platform, just add your VideoBootStrap

For simplicity, you can take something as a basis, for example, src/video/qnx/video.c or src/video/raspberry/SDL_rpivideo.c , but for a start we will make the implementation almost empty at all:

 /* SDL_sysvideo.h */ typedef struct VideoBootStrap { const char *name; const char *desc;``` int (*available) (void); SDL_VideoDevice *(*create) (int devindex); } VideoBootStrap; /* embox_video.c */ static SDL_VideoDevice *createDevice(int devindex) { SDL_VideoDevice *device; device = (SDL_VideoDevice *)SDL_calloc(1, sizeof(SDL_VideoDevice)); if (device == NULL) { return NULL; } return device; } static int available() { return 1; } VideoBootStrap EMBOX_bootstrap = { "embox", "EMBOX Screen", available, createDevice }; 

Add your VideoBootStrap to the array:

 /* Available video drivers */ static VideoBootStrap *bootstrap[] = { &EMBOX_bootstrap, #if SDL_VIDEO_DRIVER_COCOA &COCOA_bootstrap, #endif #if SDL_VIDEO_DRIVER_X11 &X11_bootstrap, #endif ... } 

In principle, at this stage it is already possible to compile SDL. As with libcurl, the compilation details will depend on the particular build system, but somehow you need to do something like this:

 ./configure --host=i386-unknown-none \ --enable-static \ --enable-audio=no \ --enable-video-directfb=no \ --enable-directfb-shared=no \ --enable-video-vulkan=no \ --enable-video-dummy=no \ --with-x=no make ls build/.libs/libSDL2.a #      

Putting yourself Quake

Quake3 assumes the use of dynamic libraries, but we will link it statically, like everything else.

To do this, set some variables in the Makefile.

 CROSS_COMPILING=1 USE_OPENAL=0 USE_OPENAL_DLOPEN=0 USE_RENDERER_DLOPEN=0 SHLIBLDFLAGS=-static 

First start

For simplicity, we will run on qemu / x86. To do this, you need to install it (hereinafter, there will be commands for Debian, for other distributions packages may be called differently).

 sudo apt install qemu-system-i386 

And the launch itself:

 qemu-system-i386 -kernel build/base/bin/embox -m 1024 -vga std -serial stdio 

However, when you start Quake, we immediately get an error

 > quake3 EXCEPTION [0x6]: error = 00000000 EAX=00000001 EBX=00d56370 ECX=80200001 EDX=0781abfd GS=00000010 FS=00000010 ES=00000010 DS=00000010 EDI=007b5740 ESI=007b5740 EBP=338968ec EIP=0081d370 CS=00000008 EFLAGS=00210202 ESP=37895d6d SS=53535353 

The error is not displayed by the game, but by the operating system. Debug showed that this error is caused by incomplete SIMD support for x86 in QEMU: some instructions are not supported and generate an exception for an unknown command (Invalid Opcode). upd: As suggested in the WGH comments, the problem was actually that I forgot to explicitly enable SSE support in cr0 / cr4, so QEMU is fine.

This happens not in Quake itself, but in OpenLibM (this is the library that we use to implement mathematical functions — sin() , expf() and so on). OpenLibm patches so that __test_sse() does not do a real check on SSE, but simply considers that there is no support.

The above steps are enough to start, in the console you can see the following output:

 > quake3 ioq3 1.36 linux-x86_64 Nov 1 2018 SSE instruction set not available ----- FS_Startup ----- We are looking in the current search path: //.q3a/baseq3 ./baseq3 ---------------------- 0 files in pk3 files "pak0.pk3" is missing. Please copy it from your legitimate Q3 CDROM. Point Release files are missing. Please re-install the 1.32 point release. Also check that your ioq3 executable is in the correct place and that every file in the "baseq3 " directory is present and readable ERROR: couldn't open crashlog.txt 

Already well, Quake3 is trying to start and even displays an error message! As you can see, it lacks the files in the baseq3 directory. It contains sounds, textures and all that. Note that pak0.pk3 should be taken from a licensed CD (yes, open source does not imply free use).

Disc preparation

 sudo apt install qemu-utils #  qcow2- qemu-img create -f qcow2 quake.img 1G #   nbd sudo modprobe nbd max_part=63 #  qcow2-      sudo qemu-nbd -c /dev/nbd0 quake.img sudo mkfs.ext4 /dev/nbd0 sudo mount /dev/nbd0 /mnt cp -r path/to/q3/baseq3 /mnt sync sudo umount /mnt sudo qemu-nbd -d /dev/nbd0 

Now you can transfer the block device to qemu

 qemu-system-i386 -kernel build/base/bin/embox -m 1024 -vga std -serial stdio -hda quake.img 

When the system starts, we will bounce the disk on /mnt and run quake3 in this directory, this time it falls later.

 > mount -t ext4 /dev/hda1 /mnt > cd /mnt > quake3 ioq3 1.36 linux-x86_64 Nov 1 2018 SSE instruction set not available ----- FS_Startup ----- We are looking in the current search path: //.q3a/baseq3 ./baseq3 ./baseq3/pak8.pk3 (9 files) ./baseq3/pak7.pk3 (4 files) ./baseq3/pak6.pk3 (64 files) ./baseq3/pak5.pk3 (7 files) ./baseq3/pak4.pk3 (272 files) ./baseq3/pak3.pk3 (4 files) ./baseq3/pak2.pk3 (148 files) ./baseq3/pak1.pk3 (26 files) ./baseq3/pak0.pk3 (3539 files) ---------------------- 4073 files in pk3 files execing default.cfg couldn't exec q3config.cfg couldn't exec autoexec.cfg Hunk_Clear: reset the hunk ok Com_RandomBytes: using weak randomization ----- Client Initialization ----- Couldn't read q3history. ----- Initializing Renderer ---- ------------------------------- QKEY building random string Com_RandomBytes: using weak randomization QKEY generated ----- Client Initialization Complete ----- ----- R_Init ----- tty]EXCEPTION [0xe]: error = 00000000 EAX=00000000 EBX=00d2a2d4 ECX=00000000 EDX=111011e0 GS=00000010 FS=00000010 ES=00000010 DS=00000010 EDI=0366d158 ESI=111011e0 EBP=37869918 EIP=00000000 CS=00000008 EFLAGS=00010212 ESP=006ef6ca SS=111011e0 EXCEPTION [0xe]: error = 00000000 

This error is again with SIMD in Qemu. upd: As suggested in the WGH comments, the problem was actually that I forgot to explicitly enable SSE support in cr0 / cr4, so QEMU is fine. This time, the instructions are used in the Quake3 virtual machine for x86. The problem was solved by replacing the implementation for x86 with an interpreted VM (in more detail about the Quake3 virtual machine and, in principle, about architectural features, you can read everything in the same article ). After that, our functions for the SDL begin to be called, but, of course, nothing happens, because these functions do nothing so far.

Add graphics support

 static SDL_VideoDevice *createDevice(int devindex) { ... device->GL_GetProcAddress = glGetProcAddress; device->GL_CreateContext = glCreateContext; ... } /*   OpenGL- */ SDL_GLContext glCreateContext(_THIS, SDL_Window *window) { OSMesaContext ctx; /*   -  --    .. */ sdl_init_buffers(); /*    Mesa */ ctx = OSMesaCreateContextExt(OSMESA_BGRA, 16, 0, 0, NULL); OSMesaMakeCurrent(ctx, fb_base, GL_UNSIGNED_BYTE, fb_width, fb_height); return ctx; } 

The second handler is needed to tell SDL which functions to call when working with OpenGL.

To do this, we start an array and from launch to launch we check which calls are missing, something like this:

 static struct { char *proc; void *fn; } embox_sdl_tbl[] = { { "glClear", glClear }, { "glClearColor", glClearColor }, { "glColor4f", glColor4f }, { "glColor4ubv", glColor4ubv }, { 0 }, }; void *glGetProcAddress(_THIS, const char *proc) { for (int i = 0; embox_sdl_tbl[i].proc != 0; i++) { if (!strcmp(embox_sdl_tbl[i].proc, proc)) { return embox_sdl_tbl[i].fn; } } printf("embox/sdl: Failed to find %s\n", proc); return 0; } 

After a few restarts, the list becomes full enough to draw a splash screen and a menu. Fortunately, Mesa has all the necessary functions. The only thing is that for some reason there is no glGetString() function, instead it had to use _mesa_GetString() .

Now when you start the application, the splash screen appears, hurray!

Adding Input Devices

Add keyboard and mouse support to the SDL.

To work with events you need to add a handler.

 static SDL_VideoDevice *createDevice(int devindex) { ... device->PumpEvents = pumpEvents; ... } 

Let's start with the keyboard. We hang up the function to interrupt key press / release. This function should memorize the event (in the simplest case, we simply write to a local variable, if desired, you can use queues), for simplicity, we will only store the last event.

 static struct input_event last_event; static int sdl_indev_eventhnd(struct input_dev *indev) { /*    ,   last_event */ while (0 == input_dev_event(indev, &last_event)) { } } 

Then, in pumpEvents() process the event and pass it to the SDL:

 static void pumpEvents(_THIS) { SDL_Scancode scancode; bool pressed; scancode = scancode_from_event(&last_event); pressed = is_press(last_event); if (pressed) { SDL_SendKeyboardKey(SDL_PRESSED, scancode); } else { SDL_SendKeyboardKey(SDL_RELEASED, scancode); } } 

 static int key_to_sdl[] = { [' '] = SDL_SCANCODE_SPACE, ['\r'] = SDL_SCANCODE_RETURN, [27] = SDL_SCANCODE_ESCAPE, ['0'] = SDL_SCANCODE_0, ['1'] = SDL_SCANCODE_1, ... ['8'] = SDL_SCANCODE_8, ['9'] = SDL_SCANCODE_9, ['a'] = SDL_SCANCODE_A, ['b'] = SDL_SCANCODE_B, ['c'] = SDL_SCANCODE_C, ... ['x'] = SDL_SCANCODE_X, ['y'] = SDL_SCANCODE_Y, ['z'] = SDL_SCANCODE_Z, }; 

Everything is on the keyboard, the SDL and Quake itself will do the rest. By the way, it turned out about here that somewhere in the quake key handling, it uses instructions that are not supported by QEMU, you have to switch to an interpretable virtual machine from the virtual machine for x86, to do this, add BASE_CFLAGS += -DNO_VM_COMPILED to the Makefile.

After that, finally, you can solemnly “skip” the screensavers and even start the game (zakostyliv some error :)). It was pleasantly surprised that everything is drawn as it should, albeit with very low fps.

Now you can start to support the mouse. To interrupt the mouse, another handler will be needed, and event handling will require some complication. We confine ourselves only to the left mouse button. It is clear that in the same way you can add the right key, wheel, etc.

 static void pumpEvents(_THIS) { if (from_keyboard(&last_event)) { /*      */ ... } else { /*      */ if (is_left_click(&last_event)) { /*     */ SDL_SendMouseButton(0, 0, SDL_PRESSED, SDL_BUTTON_LEFT); } else if (is_left_release(&last_event)) { /*     */ SDL_SendMouseButton(0, 0, SDL_RELEASED, SDL_BUTTON_LEFT); } else { /*   */ SDL_SendMouseMotion(0, 0, 1, mouse_diff_x(), /*      */ mouse_diff_y()); /*      */ } } } 

After that, it is possible to control the camera and shoot, hooray! In fact, this is enough to play :)

Optimization

Cool, of course, that there is a control and some kind of graphics, but such an FPS is completely worthless. Most likely, most of the time is spent on OpenGL (and it is software, and, moreover, SIMD is not used), and the implementation of hardware support is too long and difficult task.

We will try to speed up the game with a little blood.

Compiler optimization and resolution reduction

We collect the game, all the libraries and the OS itself with -O3 (if, all of a sudden, someone has gotten to that place, but does not know what kind of flag it is - read more about the GCC optimization flags here ).

In addition, we use the minimum resolution - 320x240, to facilitate the work of the processor.

KVM

KVM (Kernel-based Virtual Machine) allows you to use hardware virtualization (Intel VT and AMD-V) to improve performance. Qemu supports this mechanism, you need to do the following to use it.

First, you need to enable virtualization support in the BIOS. My motherboard is Gigabyte B450M DS3H, and AMD-V is enabled via MIT -> Advanced Frequency Settings -> Advanced CPU Core Settings -> SVM Mode -> Enabled (Gigabyte, what's wrong with you?).

Then we put the necessary package and add the appropriate module.

 sudo apt install qemu-kvm sudo modprobe kvm-amd #  kvm-intel 

Everything, now it is possible to pass -enable-kvm flag -enable-kvm (or -no-kvm , in order not to use hardware acceleration).

Total

The game has started, the graphics are displayed as needed, the control is working. Unfortunately, the graphics are drawn on the CPU in one stream, also without SIMD, because of the low fps (2-3 frames per second) it is very inconvenient to manage.

The porting process was interesting. Maybe in the future it will be possible to run quake on a platform with hardware graphic acceleration, but for now I’ll stop on what is.