Screenshot game - the hard way

Well, what could be so complicated about making a screenshot? It would seem - call the function, kindly provided by the operating system and get the finished picture. Surely many of you have done this more than once, and, nevertheless, you can't just take a full-screen directx or opengl application and just take it. Or rather, it is possible, but as a result you will receive not a screenshot of this application, but a rectangle filled in with black.

This is due to the fact that for full-screen games, the frame is rendered by a video card and may not even be received in the usual RAM. As a result, no one, including the OS itself, knows the contents of the frame.

Perhaps the only reliable way to get a frame is to penetrate into the gameplay and, using directx or opengl api, force the process to extract the frame from the video memory and transfer it to the application that makes a screenshot. This technique is used in most programs for recording video from the screen and streaming. The same approach can be used and, if necessary, draw something over the game.

To embed code in someone else's process, traditionally use a method called dll injection. You must write a dll which will contain the executable code. It looks like a dll like this:
')

#include <windows.h> DWORD WINAPI MainLoop(LPVOID) { //    event loop } extern "C" { __declspec (dllexport) BOOL __stdcall DllMain(HMODULE, DWORD ul_reason_for_call, LPVOID) { if (ul_reason_for_call == DLL_PROCESS_ATTACH) { DWORD thrID; CreateThread(0, 0, MainLoop, 0, 0, &thrID); } return TRUE; } } 

To inject a dll, you need to allocate memory inside someone else’s process, write down the address of the dll being implemented, and start the process that loads this dll:

 bool InjectDll(int pid, const std::string& dll) { HANDLE hProcess = OpenProcess(PROCESS_CREATE_THREAD | PROCESS_QUERY_INFORMATION | PROCESS_VM_OPERATION | PROCESS_VM_WRITE | PROCESS_VM_READ, FALSE, pid); HMODULE hKernel32 = ::GetModuleHandle(L"kernel32.dll"); void* remoteMemoryBlock = ::VirtualAllocEx(hProcess, NULL, dll.size() + 1, MEM_COMMIT, PAGE_READWRITE ); if (!remoteMemoryBlock) { return false; } ::WriteProcessMemory(hProcess, remoteMemoryBlock, (void*)dll.c_str(), dll.size() + 1, NULL); HANDLE hThread = ::CreateRemoteThread(hProcess, NULL, 0, (LPTHREAD_START_ROUTINE)::GetProcAddress(hKernel32, "LoadLibraryA"), remoteMemoryBlock, 0, NULL); if (hThread == NULL ) { ::VirtualFreeEx(hProcess, remoteMemoryBlock, dll.size(), MEM_RELEASE); return false; } return true; } 

Now it is necessary to determine the interaction scheme between the embedded code and the main application. On windows there are many different ways of interprocess communication - files, sockets, shared memory, named pipes and others. For development I use Qt - it has a class QLocalSocket and QLocalServer, which in windows work on named pipes - this is just what you need. To begin with - we will start qt-shny event loop inside dll-ki event loop:

 DWORD WINAPI MainLoop(LPVOID) { if (QCoreApplication::instance()) { //        qt  QEventLoop loop; TInjectedApp myApp; return loop.exec(); } else { int argc = 0; char** argv = nullptr; QCoreApplication loop(argc, argv); TInjectedApp myApp; return loop.exec(); } } 

Now we can implement the class TIn DejectedApp in which you can use all the features of qt. On the side of our main application, create a QLocalServer and start waiting for connections, and on the dll side, create a QLocalSocket and connect through it to the main application. I will not dwell on the use of QLocalSocket in detail - there are a large number of examples of its use, also you can see the full source code at the link at the end of the article.

And so - we figured out the implementation of our code in the process and interaction with it. Now you need to actually get a screenshot while inside the process. Consider this on the example of directx9. Using directx api we can get a backbuffer video card. But for this we need to find a pointer to IDirect3DDevice9. The task is complicated by the following factors - firstly, directx does not have api methods that allow you to get a pointer to an existing IDirect3DDevice9 - only to create a new one. Secondly, we don’t have access to the source code of the applications we are embedding in, and we don’t know exactly where this device is created, which variable it is stored in and where to look for it in general.

How can you still find this device? The first option is to go through the entire memory of the application and find an object there that is similar in content to what we are looking for. Most likely, all objects of this class will have many identical members, as well as the same or similar table of virtual functions - this is enough for a search. But this method has several disadvantages. Firstly, it is not reliable (all of a sudden some members of the class for which we are looking for will differ), and secondly, it is slow (a full pass through the entire memory allocated to an application can take a lot of time).

There is another way. We do not know the address of the IDirect3DDevice9 object, but we can easily determine the addresses of the functions that work with this object. For example, all directx applications should call the IDirect3DDevice9 :: Present function to render a frame. And the first argument (this) in it is passed a pointer to IDirect3DDevice9. Knowing the address of this function, we can intercept (hook) the call to this function, and instead perform its own function, which will receive the IDirect3DDevice9 pointer as the first argument and make a screenshot through it.

In windows, the function call interception can be done something like this (for 32-bit applications):

 #include <windows.h> #include <stdint.h> #include <iostream> void Foo() { std::cerr << "Foo()\n"; } void Bar() { std::cerr << "Bar()\n"; } void main() { uint8_t* f = (uint8_t*)Foo; uint8_t* b = (uint8_t*)Bar; DWORD t; VirtualProtect(f, 5, PAGE_EXECUTE_READWRITE, &t); uint32_t distance = b - f - 5; *f = 0xE9; *(uint32_t*)(f + 1) = distance; Foo(); } 

At first, we allow writing 5 bytes to the address of the Foo function. Then we count the number of bytes for which you need to jump (distance). Then - we write to the function address the op-code of the jmp command (1 byte) and the jump distance (4 bytes). Now, when this code is run, the Bar function will be executed instead of the Foo function. For practical use, this method will need to be slightly refined - firstly, to save the old contents of the memory somewhere and restore it after the interception. Secondly - add support for 64-bit applications.

But how do we know the address of the function Present? Present is not a function that dll exports, and therefore its address is also not available to us (at least directly). But we can take advantage of the fact that Present is implemented in the dll itself, and when loading dll it will always be located at the same offset from the dll itself. Therefore, knowing the address of the dll and the offset of the Present function, we get the address of the Present function by adding the first to the second.

And yet - everything is again not as simple as we would like. Depending on the version of the dll in the system - the offsets can be different, so we can not hard-code them in our program - we need to determine the offset again each time the program starts. In c ++ there is no ready-made way to find the address of a virtual function. Normal - please, virtual - no. So you have to do the following: create an IDirect3DDevice9 object in your application, look at the address of the Present function in the virtual functions table of this object and then read the offset between the dll address and the address of the Present function. Knowing this offset and the address of the already loaded dll inside someone else's application, we will find the address of the Present function and be able to hook it up.

 uint64_t GetVtableOffset(uint64_t module, void* cls, uint32_t offset) { uintptr_t* virtualTable = *(uintptr_t**)cls; return (uint64_t)(virtualTable[offset] - module); } 

Here module is the address of the loaded dll (what LoadLibrary returns), cls is a pointer to the previously created IDirect3DDevice9 and offset is the number of the function in the table of virtual functions of the IDirect3DDevice9 class (Present is the 17th). It is best to determine the offset in your process, and then transfer it to the implemented dll. Inside the embedded dll, you can now intercept the Present function and take a screenshot inside it by extracting the contents of the backbuffer.

 void* PresentFun = nullptr; void GetDX9Screenshot(IDirect3DDevice9* device) { IDirect3DSurface9* backbuffer; device->GetRenderTarget(0, &backbuffer); D3DSURFACE_DESC desc; backbuffer->GetDesc(&desc); IDirect3DSurface9* buffer; device->CreateOffscreenPlainSurface(desc.Width, desc.Height, desc.Format, D3DPOOL_SYSTEMMEM, &buffer, nullptr); device->GetRenderTargetData(backbuffer, buffer); D3DLOCKED_RECT rect; buffer->LockRect(&rect, NULL, D3DLOCK_READONLY); QImage img = ConvertToQImage(desc.Format, (char*)rect.pBits, desc.Height, desc.Width); // ... } static HRESULT STDMETHODCALLTYPE HookPresent(IDirect3DDevice9* device, CONST RECT* srcRect, CONST RECT* dstRect, HWND overrideWindow, CONST RGNDATA* dirtyRegion) { UnHook(PresentFun); GetDX9Screenshot(device); return device->Present(srcRect, dstRect, overrideWindow, dirtyRegion); } void MakeDX9Screen(uint64_t presentOffset) { HMODULE dx9module = GetModuleHandleA("d3d9.dll"); PresentFun = (void*)((uintptr_t)dx9module + (uintptr_t)presentOffset); Hook(PresentFun, HookPresent); } 

The extracted backbuffer is converted into the format we need (for example, QImage) - this will be the screenshot we tried to get for so long. Similarly, the process is built for other versions of directx and opengl. For opengl, the general scheme is even simpler, since there is no need to look for offsets from virtual functions — glBegin is exported by dll and its address is known.

You can view the full source code in the library, which I did for one of my projects, LibQtScreen . It implements the method for obtaining screenshots described in the article. It supports mingw and msvc, 32 and 64 bit applications, opengl and directx from 8th to 11th.

The main source of information when writing articles and libraries is the source of the streaming software - obs-studio .