Accelerating the enumeration of processes and threads in Windows

Sometimes it is necessary to list all the processes or threads that are currently running in Windows. This may be necessary for various reasons. Maybe we are writing a system utility like Process Hacker , or maybe we want to somehow react to starting / stopping new processes or threads (write a log, check them, implement our code in them). The most correct way to implement this is, of course, writing a driver. Everything is simple there - we use PsSetCreateProcessNotifyRoutine and PsSetCreateThreadNotifyRoutine to set callback functions that will be called when the processes and threads start / stop. It works very fast and does not eat resources. This is exactly what all serious tools do. But developing drivers is not always the right way. They need to be able to write correctly, for some time now they must be signed with certificates (which is not free) and registered with Microsoft (which is not fast). And yet they are not convenient to distribute - for example, programs with them cannot be uploaded to the Microsoft Store.

Well, let's take advantage of the fact that the public WinAPI offers. And he offers the CreateToolhelp32Snapshot () function, which is proposed to be used like this . Everything seems to be good - there is information about processes and threads. A little frustrating is the fact that instead of elegant callbacks we are forced to do infinite pooling in a loop, but that’s okay.

The biggest problem here is performance. The CreateToolhelp32Snapshot () + Process32First () + Process32Next () bundle works very slowly. Perhaps the problem lies somewhere in the same area as the problem described here in this article with Heap32First () + Heap32Next (). Briefly - for historical reasons, in some places the passage through the linear list takes quadratic time.
')
Is it possible to speed it all up? Can. But you have to get off the bright path of using only the public functions of WinAPI.

For a long time, I believed that WinAPI functions are divided into public (described in MSDN, acceptable for use) and undocumented (found during reverse engineering of system libraries, not described in MSDN, not officially supported). This black-and-white world seemed simple and logical to me: for public products we use public functions, for personal training purposes, utilities, internal tools — you can try to use undocumented ones.

But it turned out that there is a gray zone between these worlds. These are functions that are described in MSDN (this makes them look like public ones), but Microsoft says it can change or delete them at any time (as undocumented). Why do such functions exist? Everything is simple - on the one hand, their benefits are too great to hide, but on the other hand, future OS versions may have internal reasons to change them. Such functions in one of the terminology I have met are called “private”. An example is NtQuerySystemInformation () .

Evaluate the first line in the documentation - “NtQuerySystemInformation may be altered or unavailable in future versions of Windows. This application should not be described for half of the described applications of these “alternative functions”. Can I use these features? Everyone decides for himself. Personally, it seems to me that “fearing wolves — not going to the forest”. As if any other function in MSDN is never guaranteed to become “altered or unavailable in future versions of Windows”. Any code needs to be checked on new versions of the OS. Any code needs support and adaptation. And if there is a function that now works and is beneficial, then why not use it?

And NtQuerySystemInformation brings significant benefits. Here is a graph of the productivity growth that the MHook library got when switching from CreateToolhelp32Snapshot () to NtQuerySystemInformation ():



How to use NtQuerySystemInformation ()? Very simple:

1. We are looking for the “NtQuerySystemInformation” function in the “ntdll.dll” library. Theoretically, it might not be there, but in practice it is exactly on all versions of the OS from Vista to Win10. Not sure about XP (it was not possible or necessary to check)
2. Allocate memory for buffer with results.
3. Call the function with the parameter SystemProcessInformation
4. We go around the results by moving between the records for individual processes using the “NextEntryOffset” value from the description structure of the current process.
5. Do not forget to release the memory allocated in step number 2

Code examples can be found here or here .

Personally, using the transition from CreateToolhelp32Snapshot () to NtQuerySystemInformation (), I managed to win about 2% of the total processor load in one fairly resource-intensive scenario, which is quite a lot.

The moral of this story is that the slow work of the WinAPI function is not always the final verdict. The operating system is a big and complex thing, it may well be another way of obtaining the necessary information.