Process memory card
Have you ever thought about exactly how the memory available to your program is used, and in general, what exactly is located in these two or three gigabytes of virtual memory with which your software works?
Ask why?
Well, for a 32-bit application, 2-3 gigabytes is your limit beyond which you cannot get out without AWE, but it’s still desirable to control your own resources. But even without it, just to figure it out ...
In previous articles, I described the work of the debugger, where modifications were made to the application being debugged. This article is a continuation of this material. And although it will have no relation to the debugger, the debugging process itself is the most direct ...
')
Let's see exactly how a programmer works with memory when debugging (especially when debugging a third-party application, in other words, when reversing):
1. As a rule, the most frequent operation will be to search for a value in the application's memory, and unfortunately, this functionality is for some reason not provided in the Delphi debugger (in fact, as in MS VC ++).
2. Modification of system structures (PEB / TEB / SEHChain / Unwind / directories of PE files, etc ...) will be much easier when the structure fields are mapped to the addresses they occupy and are presented in a readable form.
3. Tracking changes in the process memory (almost no one provides functionality implemented as plug-ins for popular debuggers). Indeed, why trace to blue in the face when it is enough to compare two snapshots of a memory card in order to understand whether the necessary data modification is happening here or not?
Yes, actually, there are many use cases.
However, if without lyrics, there are very few utilities that display more or less imputed information about the process memory card that can be used for debugging.
The most convenient implementation from OllyDebug 2, but unfortunately, it does not display data on 64 bits (still waiting).
VMMap from Mark Russinovich performs purely decorative properties, yes it is beautiful, but signed by Microsoft, but it’s hard to practically use the data it displays.
ProcessHacker is a good tool, but its author did not set himself the task of working with the output of memory data, so the information he displays can be said to be the simplest.
Well, for so many years of working with it, I was not used to the memory card from IDA Pro (I'm not comfortable) :)
However, debugging is not everything where a valid memory card can be useful. In particular, for work, I use a memory card when analyzing the error log sent to us by users along with a dump of critical areas, integrating information about it into EurekaLog.
In this article I will try to tell you in steps how to create a process memory card on your own and place information about the data needed for debugging and analysis in it.
The entire virtual memory of the process is presented in the form of pages.
Pages are small (4096 bytes) and large. ( More details can be found in MSDN )
In most cases, consecutive pages have the same attributes.
What is the region?
Roughly ( based on MSDN ) is a set of all pages with the same attributes that begin with the address passed to the VirtualQuery function.
In the simplest form, you can get a list of regions of our process with this code:
For example, initially we passed the address nil as the first parameter. After calling the function, the MBI variable will take the following values:
The region size is $ 10,000 (64 kb), this corresponds to 16 pages running in a row, starting with the address zero, the state of which (State) is MEM_FREE ($ 10,000) and the security attribute PAGE_NO_ACCESS (1) is set in the Protect parameter.
If you rewrite the code like this:
... then we can clearly see the principle of partitioning into regions by the VirtualAlloc function:
For example, the second and third regions have the same access attributes (read entry), but different AllocationBase. AllocationBase is assigned to pages when allocating memory through VirtualAlloc, thus combining them into a separate region.
The time has come to start filling in the regions we received with information about what they store, and we will start with the flows (the threads - to someone as convenient).
The code for getting the list of threads is simple - via CreateToolhelp32Snapshot.
Steps:
The NT_TIB declaration looks like this:
Well, or so, if you describe a little more:
The remaining fields are not needed.
Well, however, how - not needed?
We need, of course, but for now they are redundant for us.
By the way, here is a link where you can see a slightly outdated description of this structure: Thread Environment Block .
This code will display the following image:
This is how it will be seen in VMMap.
The picture shows that VMMap did not display information about the TEB.
By the way, some of the functions and structures from the above code are not declared in the standard Delphi source code; you can see their declaration in the demo examples included in this article. But this does not mean that they are undocumented in MSDN :)
If we want to work with the TEB of our thread, the code will be greatly simplified due to the fact that you do not need to use the ToolHelp32.dll functions, but rather using the segment register FS (or GS for x64).
For example, this function is very often found to get the address of a TEB:
In this case, the NtTIB.Self parameter of the TEB structure is accessed, which is located at offset 0x18 (or 0x30 in the case of 64-bit TEB) from its beginning.
But let's continue ...
Part of the data received, but this is not all the information available to us.
On the stack of each thread there are SEH frames, which are generated by the machine at the entrance to the try..finally / except block, as well as a stack of procedure calls. It would be nice to have these data on hand and display them in a more visual form - with reference to the region.
The promotion of SEH frames will be dealt with here is such a simple procedure:
Having received the value of TEB.TIB.ExceptionList as a parameter, which points to the first EXCEPTION_REGISTRATION structure, it runs along the data chain of the structures, focusing on the prev value of this structure, which contains the address of the previous EXCEPTION_REGISTRATION structure. And the handler parameter contains the address of the exception handler if it happens.
It looks like this:
Well, CallStack will receive the following procedure:
However, unlike the Delphi debugger, it will output data about the procedures for which the stack frame is generated, it will skip the rest.
The StackWalk (or StackWalk64) function is responsible for listing information about stack frames.
Now a caveat: if we apply this code to ourselves, then it will only be able to ottrassirovat one stack frame, after which the output will occur (you can check on the demo application ).
This will happen for the following reason: to correctly trace the StackWalk function, you must specify the parameters of the current stack frame (EBP and ESP / RBP and RSP for x64) and, in fact, the current code address (EIP or RIP register for x64). If we take this data from ourselves, this will happen at the moment when we call the GetThreadContext function, and we will begin to unwind the stack after exiting this function, where all three parameters become, to put it mildly, not valid. For this reason, tracing yourself by calling this function will not work.
This moment is desirable to take into account ...
I will dwell on getting information about 32-bit process streams under a 64-bit OS including 32 and 64-bit variants a little later, and now ...
By itself, a Delphi application, as a rule, does not use heaps, it is more a prerogative of C ++ applications, but still, heaps are present here. Usually they are created and used by various third-party libraries for their needs.
The nuance in obtaining data about the heaps is that the HeapEntry elements of which each heap consists may be several thousand, and the second nuance is that the Heap32Next function re-rebuilds the entire list with each call, creating a rather sensitive delay (up to tens of seconds).
I already wrote about this unpleasant feature.
True, in that article, the code was rather approximate, just to demonstrate the principle itself, and it will not suit us, but a more combed version would be fine with it:
In short, using the RtlQueryProcessDebugInformation function call, RtlCreateQueryDebugBuffer and RtlQueryProcessDebugInformation creates a buffer containing information about the current process heaps. Then, knowing the structure of the data stored in it, we obtain this data in a cycle.
pDbgBuffer ^ .Heaps - stores lists of heaps (similar to THeapList32), and the records themselves are stored in pDbgBuffer ^ .Heaps ^ .Heaps [N] .Entries (analogue of THeapEntry32).
This code will display the following information:
In principle, I use heaps rarely for debugging, but sometimes this information may be useful.
Now it's time to get information about executable files and libraries loaded into the process address space. There are several ways to do this (for example, by analyzing PEB.LoaderData), but we will proceed more simply.
As a rule, a separate region is allocated for the PE file (well, at least I have not met such that the PE image is loaded without alignment on the top of the region), therefore, using the code from the first chapter and checking the data of the first page of the region to match the PE file, we get a list of all loaded libraries and executable files.
The following code detects the presence of a valid PE file at the specified address:
Well, more precisely, as he simply checks for the presence of ImageDosHeader and ImageNTHeader, focusing on their signatures. In principle, for 99% of cases this is enough.
The third parameter is simply informational, it shows whether the PE file is 64-bit.
You can get the path to the downloaded file by calling the function GetMappedFileName:
And now let's try to see what is being loaded into a regular console application:
It turns out this picture:
64-bit library in a 32-bit application? Yes, easier simple :)
My application is 32-bit, Windows 7 x64 operating system. Judging by what is shown in the picture, four 64-bit libraries live and work in our 32-bit process, however, nothing unusual here is the so-called Wow64 ( Win32 emulation on 64-bit Windows ).
But at once it becomes clear where 64-bit analogs of 32-bit streams and heaps come from.
Now, in a good way, you need to get the addresses of sections of each PE file so that you can show them more clearly. All sections are aligned to the address of the beginning of the page and do not overlap with each other.
Let's do this with this code:
It uses the call to the MapAndLoad function, which, in addition to loading the file and checking its headers, also performs section alignment by calling NtMapViewOfSection.
For its own process, of course, the call to this function is redundant, since The required PE file is already loaded into the address space of the process, but since we need a more universal code to work with other processes, then we use this approach.
MapAndLoad is also good because it allows 64-bit processes to load 32-bit PE files (although this does not work for 32-bit processes), and in the future this feature will come in handy.
The essence of the code is as follows: after running MapAndLoad, we will have in our hands a filled TLoadedImage structure, the Sections parameter of which points to an array of TImageSectionHeader structures. Each of these structures has a VirtualAddress field, which is an offset from the library loading address. By adding the value of this field to the library hInstance, we get the address of the section.
The functions IsExecute and IsWrite check the characteristics of the section and return True if the section contains executable code (IsExecute) or data that can be modified (IsWrite). They look like this:
As a result of this code, we will see the following:
True, with this code there is another small nuance.
As you can see in the previous picture, the GetMappedFileName function returns the path to the downloaded file in the following form: "\ Device \ HarddiskVolume2 \ Windows \ System32 \ wow64cpu.dll", and the MapAndLoad function requires a normalized path like "C: \ Windows \ System32 \ wow64cpu. dll ".
The following code is responsible for bringing the path to the usual view:
This is already quite an old code, constantly used by me to bring it to a normal path. Its essence is that of the paths of the following types:
... get fixed "\ Device \ HarddiskVolume1 \ WINDOWS \ system32 \ ntdll.dll".
This is done by calling ZwOpenFile + NtQueryObject, after which all the disks in the system are simply searched and for each one QueryDosDevice is called, which returns a path in the same format. After that, the paths are compared and (if they match) the corresponding disk label is substituted to the transmitted path.
But this is the lyrics.
To be completely satisfied with yourself, it is advisable to output the same file directory as the PE file so that it is immediately clear where to look, for example, the import table, where UNWIND is sitting, etc.
This is done in fairly simple code:
Having the TLoadedImage structure in hand, we can simply simply call the ImageDirectoryEntryToData function to get its address, although it will be tied to the address where the PE file is displayed. To translate it into real, you need to subtract the address from which the image is displayed from the current address, thus obtaining the offset from the beginning of the file, and add it to the ImageBase library.
In the end, you get this picture:
It is immediately obvious that, for example, the import / export / deferred import directories, etc., are located in the ".text" section of the msctf.dll library.
The resource directory sits in the ".rsrc" section, and relocs are also where they should be, but the "bound_import" directory falls out of the scheme.
Yes, indeed, this directory is not located directly in any of the sections of the library, such is its peculiarity. It usually goes right behind the PE header (although it can sometimes occur between sections). This directory is used to provide the mechanism of "bound imports", which is found mainly only in programs and libraries running in the OS.
Its essence is that all the addresses of imported functions are stitched into an executable file at the compilation stage, thus it is not necessary to perform unnecessary gestures, running around the normal import table in search of a function address.
But the overhead is also relevant, because as soon as any of the libraries declared in the bound import section changes, the application must be recompiled.
Having data about threads, heaps and executable files, you can make a small utility right now that will display information in a readable form, but what else can you add?
At a minimum, it is highly desirable to receive and display information from the process environment block.
It can be accessed by calling the NtQueryInformationProcess function with the ProcessBasicInformation flag (a constant equal to zero). In this case, the hands will have a PROCESS_BASIC_INFORMATION structure, in which the PebBaseAddress field will contain the address PEB.
But this will be relevant only if the bits of the processes (the requester and of which the requested information) match. If we call this function from a 64-bit application as applied to a 32-bit one, we get the address of a 64-bit PEB, rather than a native 32-bit one.
To get access to Wow64PEB from a 64-bit application (let's call it that way), you need to call the NtQueryInformationProcess function with the ProcessWow64Information parameter (constant 26) and the size of the buffer equal to SizeOf (ULONG_PTR). In this case, instead of the PROCESS_BASIC_INFORMATION structure, the function will return a pointer to a 32-bit PEB, from which we will read out the information we need by means of ReadProcessMemory.
What is PEB?
Roughly speaking, this is not a strongly documented structure, most of which is intended for storing data used directly by the system. But this does not mean that it is not interesting to the developer of a conventional application. In particular, this structure contains a number of interesting fields, such as: the BeingDebugged flag, indicating whether the debugger is connected to the process; a pointer to PEB_LDR_DATA, which contains information about the modules loaded in the process; and a lot of the rest is quite useful information for a programmer, especially for someone who knows how to use it for their own purposes :)
This structure looks like this (declaration for Windows7 x86 / 64):
By the way, compare this structure with the one that is officially available in MSDN .
For Window 2000 / XP / 2003 there will be small changes, but not very critical ones.
I will not paint each field, those who work with PEB and so know? what exactly they need, but in some fields I will focus your attention.
So:
Well, and so on - you can continue for a long time.
Most of these fields occupy their pages in the address space of the process. For example, ProcessParameters usually sits in one of the heaps created by the loader, environment variables are also located somewhere in that area.
If we want to visualize all of this (and I am leading to this), we must have this data on hand in order to have something to display in the final application.
Agree, it is much more pleasant to have on your hands, instead of a certain block of binary data, something like this:
But there is also KUSER_SHARED_DATA.
This is also the structure used by the system, and you constantly encounter it by calling the same GetTickCount or IsProcessorFeaturePresent.
For example, NtSystemRoot sits exactly in it, and, again, why list everything, it's easier to see:
However, perhaps, here we will stop for now ...
At the theoretical part ended and it was time to put it all into practice.
First of all, you need to decide how to store information about the regions. Preparing for the article, I wrote a set of classes allocated to the common namespace "MemoryMap", you can find them in the demo examples .
The information for each region will be stored in the TRegionData class, implemented in the “MemoryMap.RegionData.pas” module.
It looks something like this (in the process of developing a project, the class declaration may change).
In order:
Each region, as a rule, stores in itself the data of one type.
Those.for heaps, stream stacks, PE files, its own page region is allocated.
The RegionType property is responsible for storing the region type. This is an enumeration type, declared as follows:
The region parameters obtained by calling VirtualQueryEx are stored in the MBI field.
A brief description of the region is stored in Details. Anything can be stored in it, for example, the path to the displayed PE file, if present, a string description of the stream ID, etc ...
The following three parameters are used to organize the tree structure.
One of the regions is the root node (root), the rest are subsidiaries.
The RegionVisible flag indicates whether a region is a root node.
The HiddenRegionCount property contains the number of subregions (the AllocationBase of which is equal to the BaseAddress of the root).
Well, the parameter Parent stores a link to the root.
The work was not entirely optimal, it would be possible to organize a classic tree, but at the current time there is simply no time to redo it, maybe someday later :)
TotalRegionSize contains the total size of all subregions, including the root one.
In case the region contains a heap, the data on its first element is placed in the Heap parameter, which is the following structure:
The remaining heap items located within the region are placed in the Contains field.
In general, the Contains field may contain data of many types.
Next comes the Thread field, which stores information about the stream that the region uses to store its own data.
If there are a lot of stream data within a region (for example, a list of SEH frames or a CallStack stream), they are also placed in the Contains field.
Data from system structures (fields of PEB / TEB structures, etc.) are placed in the SystemData field, which is an entry from the data address and their descriptions.
Also this data can be placed in the Contains field.
If the region belongs to one of the sections of the PE file, the section data is placed in the Section parameter. Well, the list of file directories is located in the Directory field.
That somehow so briefly. Now, to present the data on the process memory card, we need to obtain a list of regions, create an instance of the TRegionData class for each of them, and initialize the fields of the created object with the required information.
The TMemoryMap class is responsible for this ...
This class is implemented in the "MemoryMap.Core.pas" module.
Its task is literally reduced to three main stages:
In practice, everything looks more complicated.
The basic procedure for collecting information looks like this:
I gave the approximate code of the GetAllRegions / AddThreadsData / AddHeapsData and AddImagesData procedures in the first four chapters and I will not focus on it, but it is advisable to deal with the rest.
The very first step after opening the process is determining the bit depth of the process.
This is necessary for the reason that if the bits of the processes (current and for which we receive information) do not match, some additional actions need to be taken.
The general scheme is as follows:
If everything is clear with the first two points, then the other three will be discussed in more detail.
The reason that the 32-bit process will not be able to get data on the 64-bit is simple: the pointer size will not allow, plus ReadProcessMemory will periodically generate an ERROR_PARTIAL_COPY error.
But getting data from a 32-bit process in a 64-bit OS is much more cunning.
As I said earlier, in a 32-bit application, four 64-bit libraries are loaded that create their heaps / streams.
If we get a list of heaps and streams from a 32-bit application, we will only see data related to 32 bits, data on 64-bit analogues cannot be obtained.
The same will happen in the case of requesting data on a 32-bit process from a 64-bit one, only data related to 64 bits will be returned. Although in this case there is an option to get them in part.
In particular, access to 32-bit PEB is made by calling such a function:
Access to 32-bit TEB can be obtained by reading the address from 64-bit TEB, which is stored in the NtTIB.ExceptionList parameter.
You can get the context of a 32-bit stream for CallStack promotion like this:
Or by calling the Wow64GetThreadContext function.
But I don’t know how to get data about 32-bit heaps from a 64-bit process in a legal way. The only option that I use now is to send a command to a 32-bit process, which collects data on 32-bit heaps and gives them back to the 64-bit one (the handler in the AddWow64HeapsData function does about that).
Now, when we have figured out the process bit definition and why we need it, let's go further, namely, the SuspendProcess function call.
In an amicable way, this is only necessary so that the data in the remote process does not change to irrelevant at the time of their reading. However, I usually use this set of classes in two cases, for my own application or for an application under debugger. In both cases, it is not necessary to freeze the threads, but if some third-party application is being analyzed, then why not?
After freezing the remote process, three auxiliary classes are created.
In fact, TWorkset stores a list of structures of the form:
These structures are stored in the dictionary and each is associated with a specific page address.
Parameters are simple:
This data is obtained in the following way, in which everything comes down to calling the QueryWorkingSet function:
This function returns an array of ULONG_PTR, each element of which stores data as follows: the first five bits store the security attributes of the page; the next three bits are the number of processes by which this page is available; another bit indicates the general availability of the page; well, then comes the address of the page itself.
More details can be found here: PSAPI_WORKING_SET_BLOCK .
In fact, this is just an information class, no more, no less.
However, back to our code.
The next steps are:
As you can see, I have already told all the interesting (well, almost) :)
The remaining steps are not interesting, except for the call to UpdateRegionFilters.
It performs a utilitarian function, namely, it excludes regions from the list that are currently unnecessary (well, for example, removes regions with unallocated memory, etc.).
This procedure will be invoked continuously when the filter is changed through the Filter property.
However, all this you can, if you want to see from the code of the class itself.
Working with him is quite simple:
As they say, he wrote for himself, therefore, working with this class is easy :)
The essence of this class is to obtain more detailed information about the address in the process. Well, for example, in the second chapter we received the CallStack flow (or handlers of SEH frames) and these were just some kind of addresses. But it’s much more interesting to see something like this image instead of dry numbers:
This is done very simply - just call the function SymGetSymFromAddr, but there are a few nuances.
Let's first look at the code:
To correctly obtain the description of the name of the function that owns the address, you need to know the path to the library that owns the function, or the address to which this library is loaded (the code uses both parameters). These parameters are required for the SymLoadModule function.
The second caveat is that the call to the SymGetSymFromAddr function can sometimes fail. The reason is not clear to me, but on the Internet they periodically describe this situation and how to solve it is to call the SymLoadModule function again without calling the SymUnloadModule. In such a strange behavior did not understand - but really helps.
The last of the nuances is that this function will return a valid description of the address only when this information is present (characters are loaded from an external file or they are included in the required module).
This information is not very important when debugging, but it simplifies it a little.
This is, for example, the standard stack of the Chrome browser stream (CallStack + SEH frames):
More useful information that symbols can provide is a list of library functions exported and their current addresses.
In the TSymbols class, this information is obtained by calling the GetExportFuncList procedure and looks like this:
It all comes down to calling SymEnumerateSymbols, which is passed the address of the callback function.
When you call it, the SymbolName parameter will contain the name of the exported function, and SymbolAddress will contain its address.
This is quite enough to display the following label to the user:
In more detail the implementation of this class, including omitted calls SymSetOptions and SymInitialize, you can see in the module "MemoryMap.Symbols.pas".
Well, here we come to the final part of the article.
As I said earlier, I use the MemoryMap class set in two versions:
1. Integrating it into the output of EurekaLog by overlapping its OnAttachedFilesRequest handler, where I add the current process map that is relevant at the time of the exception, and dumps of all Private Regions (pages not associated with certain data with the MEM_PRIVATE flag) and thread stacks, plus some of the information from PEB. Usually this is enough to analyze the causes of the error.
2. I use it as an alternative tool for analyzing the application being debugged.
For the second option, a separate utility was implemented, which works directly with the MemoryMap classes, plus adds some additional functionality.
I will not describe its source code, I’ll just go over a bit of functionality.
From the interface, it is almost one to one reminds VMMap. However, this was originally planned, because such an interface is most convenient for analysis.
At the top is a list with general information by region, grouped by type, it is also a filter.
At the moment, it represents the following functionality:
1. View the contents of the memory at the specified address (Ctrl + Q).
This functionality, in principle, is present in the Delphi debugger in the CPU View window, but the features of this mode are much greater. For example, in the case of viewing the PEB field, data will be displayed in a different form:
This is how the process parameter block will look like:
Well, and so on. In total, at this point, the utility can display mapped data in the following structures:
This list is not final, periodically new structures will be added to it.
2. Search for data in the process memory (Ctrl + F):
This functionality in the Delphi debugger, unfortunately, is missing.
The search can be performed either by Ansi or by Unicode string, or simply by abstract HEX buffer. When searching, you can specify the address of the beginning of the search, as well as a flag indicating the need to search in pages that are accessible only for reading.
The result is displayed as a window with the memory dump shown above.
3. Comparator of two memory cards. Included in the settings.
Allows you to find differences between two memory cards and displays them as text.
Only the maps themselves are compared, not the data. Those.if at some address 4 bytes have changed, this change will not be displayed. But in the event that the size of the region has changed, the heap has been deleted, the file has been downloaded / downloaded, etc. - all this will be displayed in the comparison results.
It is possible to compare both the current map snapshot with the previously saved one, and when updating the snapshot using the F5 hot key.
4. Dump of memory.
Also missing in the Delphi debugger functionality. Allows you to save to disk the contents of the memory of the specified region or data from the specified address.
5. Output of all available exported functions from all libraries loaded in the analyzed process (Ctrl + E).
As well as a quick search for a function by its name or address.
So far, the current functionality is enough for me personally, and I haven’t added a new one, but in the future this utility will be developed.
ProcessMemoryMap is an open source project.
Her latest stable release is always available at the link: http://rouse.drkb.ru/winapi.php#pmm2
GitHub repository with the latest code changes can be found here: https://github.com/AlexanderBagel/ProcessMemoryMap
Direct link to the source code : https://github.com/AlexanderBagel/ProcessMemoryMap/archive/master.zip
Direct link to the latest build: http://rouse.drkb.ru/files/processmm_bin.zip
For self-assembly, you will need the installed package of components Virtual TreeView version 5 and higher: http://www.soft-gems.net/ .
The build is carried out using Delphi XE4 and higher in the “Win32 / Release” mode, and the 64-bit version of this utility will be automatically assembled and connected (as a resource).
Under older versions of Delphi, the health of ProcessMemoryMap has not been tested.
Well, I hope this material will be useful for you. I, of course, passed only at the very top, for if you disclose all the material in more detail, the volume of the article will increase enormously.
Therefore, here are some links where you can find out a little more information.
Information on system structures of TEB / PEB, etc. can be found here:
http://processhacker.sourceforge.net/
http://redplait.blogspot.ru/
http://www.reactos.org/ru
Information about PE files:
http://msdn.microsoft.com/ en-us / magazine / ms809762.aspx
SEH Information:
http://msdn.microsoft.com/en-us/library/ms680657(v=VS.85).aspx
http://www.microsoft.com/msj /0197/exception/exception.aspx
http://qxov.narod.ru/articles/seh/seh.html
The source code of all demo examples can be picked up at this link . Many thanks to the
forum “Masters of Delphi” for their repeated assistance in the preparation of the article.
Personal thanks for reading the material to Dmitry aka "Ptiburdukov's brother", Andrey Vasilyev aka "Inovet", and also to Sergey aka "Cartman".
Good luck.
Ask why?
Well, for a 32-bit application, 2-3 gigabytes is your limit beyond which you cannot get out without AWE, but it’s still desirable to control your own resources. But even without it, just to figure it out ...
In previous articles, I described the work of the debugger, where modifications were made to the application being debugged. This article is a continuation of this material. And although it will have no relation to the debugger, the debugging process itself is the most direct ...
')
Let's see exactly how a programmer works with memory when debugging (especially when debugging a third-party application, in other words, when reversing):
1. As a rule, the most frequent operation will be to search for a value in the application's memory, and unfortunately, this functionality is for some reason not provided in the Delphi debugger (in fact, as in MS VC ++).
2. Modification of system structures (PEB / TEB / SEHChain / Unwind / directories of PE files, etc ...) will be much easier when the structure fields are mapped to the addresses they occupy and are presented in a readable form.
3. Tracking changes in the process memory (almost no one provides functionality implemented as plug-ins for popular debuggers). Indeed, why trace to blue in the face when it is enough to compare two snapshots of a memory card in order to understand whether the necessary data modification is happening here or not?
Yes, actually, there are many use cases.
However, if without lyrics, there are very few utilities that display more or less imputed information about the process memory card that can be used for debugging.
The most convenient implementation from OllyDebug 2, but unfortunately, it does not display data on 64 bits (still waiting).
VMMap from Mark Russinovich performs purely decorative properties, yes it is beautiful, but signed by Microsoft, but it’s hard to practically use the data it displays.
ProcessHacker is a good tool, but its author did not set himself the task of working with the output of memory data, so the information he displays can be said to be the simplest.
Well, for so many years of working with it, I was not used to the memory card from IDA Pro (I'm not comfortable) :)
However, debugging is not everything where a valid memory card can be useful. In particular, for work, I use a memory card when analyzing the error log sent to us by users along with a dump of critical areas, integrating information about it into EurekaLog.
In this article I will try to tell you in steps how to create a process memory card on your own and place information about the data needed for debugging and analysis in it.
1. Get a list of available regions
The entire virtual memory of the process is presented in the form of pages.
Pages are small (4096 bytes) and large. ( More details can be found in MSDN )
In most cases, consecutive pages have the same attributes.
What is the region?
Roughly ( based on MSDN ) is a set of all pages with the same attributes that begin with the address passed to the VirtualQuery function.
In the simplest form, you can get a list of regions of our process with this code:
program Project1; {$APPTYPE CONSOLE} {$R *.res} uses Windows, SysUtils; var MBI: TMemoryBasicInformation; dwLength: NativeUInt; Address: PByte; begin Address := nil; dwLength := SizeOf(TMemoryBasicInformation); while VirtualQuery(Address, MBI, dwLength) <> 0 do begin Writeln( 'AllocationBase: ', IntToHex(NativeUInt(MBI.AllocationBase), 8), ', BaseAddress: ', IntToHex(NativeUInt(MBI.BaseAddress), 8), ', RegionSize: ', MBI.RegionSize); Inc(Address, MBI.RegionSize); end; Readln; end. For example, initially we passed the address nil as the first parameter. After calling the function, the MBI variable will take the following values:
- BaseAddress = nil
- AllocationBase = nil
- AllocationProtect = 0
- RegionSize = $ 10,000
- State = $ 10,000
- Protect = 1
- Type_9 = 0
The region size is $ 10,000 (64 kb), this corresponds to 16 pages running in a row, starting with the address zero, the state of which (State) is MEM_FREE ($ 10,000) and the security attribute PAGE_NO_ACCESS (1) is set in the Protect parameter.
If you rewrite the code like this:
function ExtractAccessString(const Value: DWORD): string; const PAGE_WRITECOMBINE = $400; begin Result := 'Unknown access'; if (Value and PAGE_EXECUTE) = PAGE_EXECUTE then Result := 'E'; if (Value and PAGE_EXECUTE_READ) = PAGE_EXECUTE_READ then Result := 'RE'; if (Value and PAGE_EXECUTE_READWRITE) = PAGE_EXECUTE_READWRITE then Result := 'RWE'; if (Value and PAGE_EXECUTE_WRITECOPY) = PAGE_EXECUTE_WRITECOPY then Result := 'RE, Write copy'; if (Value and PAGE_NOACCESS) = PAGE_NOACCESS then Result := 'No access'; if (Value and PAGE_READONLY) = PAGE_READONLY then Result := 'R'; if (Value and PAGE_READWRITE) = PAGE_READWRITE then Result := 'RW'; if (Value and PAGE_WRITECOPY) = PAGE_WRITECOPY then Result := 'Write copy'; if (Value and PAGE_GUARD) = PAGE_GUARD then Result := Result + ', Guarded'; if (Value and PAGE_NOCACHE) = PAGE_NOCACHE then Result := Result + ', No cache'; if (Value and PAGE_WRITECOMBINE) = PAGE_WRITECOMBINE then Result := Result + ', Write Combine'; end; function ExtractRegionTypeString(Value: TMemoryBasicInformation): string; begin Result := ''; case Value.State of MEM_FREE: Result := 'Free'; MEM_RESERVE: Result := 'Reserved'; MEM_COMMIT: case Value.Type_9 of MEM_IMAGE: Result := 'Image'; MEM_MAPPED: Result := 'Mapped'; MEM_PRIVATE: Result := 'Private'; end; end; Result := Result + ', ' + ExtractAccessString(Value.Protect); end; var MBI: TMemoryBasicInformation; dwLength: NativeUInt; Address: PByte; begin Address := nil; dwLength := SizeOf(TMemoryBasicInformation); while VirtualQuery(Address, MBI, dwLength) <> 0 do begin Writeln( 'AllocationBase: ', IntToHex(NativeUInt(MBI.AllocationBase), 8), ', BaseAddress: ', IntToHex(NativeUInt(MBI.BaseAddress), 8), ' - ', ExtractRegionTypeString(MBI)); Inc(Address, MBI.RegionSize); end; ... then we can clearly see the principle of partitioning into regions by the VirtualAlloc function:
For example, the second and third regions have the same access attributes (read entry), but different AllocationBase. AllocationBase is assigned to pages when allocating memory through VirtualAlloc, thus combining them into a separate region.
2. Collect flow data
The time has come to start filling in the regions we received with information about what they store, and we will start with the flows (the threads - to someone as convenient).
The code for getting the list of threads is simple - via CreateToolhelp32Snapshot.
const THREAD_GET_CONTEXT = 8; THREAD_SUSPEND_RESUME = 2; THREAD_QUERY_INFORMATION = $40; ThreadBasicInformation = 0; ThreadQuerySetWin32StartAddress = 9; STATUS_SUCCESS = 0; var hSnap, hThread: THandle; ThreadEntry: TThreadEntry32; TBI: TThreadBasicInformation; TIB: NT_TIB; lpNumberOfBytesRead: NativeUInt; ThreadStartAddress: Pointer; begin // hSnap := CreateToolhelp32Snapshot(TH32CS_SNAPTHREAD, GetCurrentProcessId); if hSnap <> INVALID_HANDLE_VALUE then try ThreadEntry.dwSize := SizeOf(TThreadEntry32); if Thread32First(hSnap, ThreadEntry) then repeat if ThreadEntry.th32OwnerProcessID <> GetCurrentProcessId then Continue; Writeln('ThreadID: ', ThreadEntry.th32ThreadID); // hThread := OpenThread(THREAD_GET_CONTEXT or THREAD_SUSPEND_RESUME or THREAD_QUERY_INFORMATION, False, ThreadEntry.th32ThreadID); if hThread <> 0 then try // ThreadProc() if NtQueryInformationThread(hThread, ThreadQuerySetWin32StartAddress, @ThreadStartAddress, SizeOf(ThreadStartAddress), nil) = STATUS_SUCCESS then Writeln('ThreadProcAddr: ', IntToHex(NativeUInt(ThreadStartAddress), 1)); // if NtQueryInformationThread(hThread, ThreadBasicInformation, @TBI, SizeOf(TThreadBasicInformation), nil) = STATUS_SUCCESS then begin Writeln('Thread Environment Block (TEB) Addr: ', IntToHex(NativeUInt(TBI.TebBaseAddress), 1)); // // TIB (Thread Information Block) if ReadProcessMemory(GetCurrentProcess, TBI.TebBaseAddress, @TIB, SizeOf(NT_TIB), lpNumberOfBytesRead) then begin Writeln('Thread StackBase Addr: ', IntToHex(NativeUInt(TIB.StackBase), 1)); Writeln('Thread StackLimit Addr: ', IntToHex(NativeUInt(TIB.StackLimit), 1)); end; end; finally CloseHandle(hThread); end; until not Thread32Next(hSnap, ThreadEntry); finally CloseHandle(hSnap); end; Readln; end. Steps:
- Using CreateToolhelp32Snapshot / Thread32First / Thread32Next we get a list of active threads in our application.
- For more detailed information, you will need the handle of the thread, which is obtained by calling OpenThread.
- With the help of NtQueryInformationThread, we get the address of the flow procedure with which it started, and basic information about the flow in the form of a TThreadBasicInformation structure.
- From this structure, we are interested in only one field - TebBaseAddress, which contains the address of the flow environment block, the so-called. TEB (Thread Environment Block).
- By calling ReadProcessMemory (although this is redundant for your application), we read the data at the TEB address, namely, its very first parameter, which is the NT_TIB structure.
The NT_TIB declaration looks like this:
PNT_TIB = ^_NT_TIB; _NT_TIB = record ExceptionList: Pointer; StackBase, StackLimit, SubSystemTib: Pointer; case Integer of 0: ( FiberData: Pointer ); 1: ( Version: ULONG; ArbitraryUserPointer: Pointer; Self: PNT_TIB; ) end; NT_TIB = _NT_TIB; PPNT_TIB = ^PNT_TIB; Well, or so, if you describe a little more:
- ExceptionList - in a 32-bit process, a pointer to the address of the current SEH frame (EXCEPTION_REGISTRATION structure). Based on this information, we will unwind the entire chain of SEH frames.
- If the TEB belongs to a 64-bit stream running in a 32-bit application, then this field will point to the ExceptionList field of its 32-bit counterpart.
- In a 64-bit process, this field is always enclosed, since for 64 bits, instead of the SEH mechanism, a slightly different mechanism works.
- StackBase - base stack. The address from which the stack begins to grow in the direction of StackLimit.
- StackLimit is the current top of the stack.
- ArbitraryUserPointer is something like a free TLS slot. Roughly speaking, a variable belongs to a thread, the value of which can be arbitrarily changed by the programmer himself for his own needs.
- Self - the parameter containing the address of the TEB (i.e. itself)
The remaining fields are not needed.
Well, however, how - not needed?
We need, of course, but for now they are redundant for us.
By the way, here is a link where you can see a slightly outdated description of this structure: Thread Environment Block .
This code will display the following image:
This is how it will be seen in VMMap.
The picture shows that VMMap did not display information about the TEB.
By the way, some of the functions and structures from the above code are not declared in the standard Delphi source code; you can see their declaration in the demo examples included in this article. But this does not mean that they are undocumented in MSDN :)
If we want to work with the TEB of our thread, the code will be greatly simplified due to the fact that you do not need to use the ToolHelp32.dll functions, but rather using the segment register FS (or GS for x64).
For example, this function is very often found to get the address of a TEB:
function GetCurrentTEB: NativeUInt; asm {$IFDEF WIN64} // mov RAX, qword ptr GS:[30h] // , 64- DB $65, $48, $8B, $04, $25, $30, 0, 0, 0 // , mov RAX, qword ptr GS:[abs $30] {$ELSE} mov EAX, FS:[18h] {$ENDIF} end; In this case, the NtTIB.Self parameter of the TEB structure is accessed, which is located at offset 0x18 (or 0x30 in the case of 64-bit TEB) from its beginning.
But let's continue ...
Part of the data received, but this is not all the information available to us.
On the stack of each thread there are SEH frames, which are generated by the machine at the entrance to the try..finally / except block, as well as a stack of procedure calls. It would be nice to have these data on hand and display them in a more visual form - with reference to the region.
The promotion of SEH frames will be dealt with here is such a simple procedure:
procedure GetThreadSEHFrames(InitialAddr: Pointer); type EXCEPTION_REGISTRATION = record prev, handler: Pointer; end; var ER: EXCEPTION_REGISTRATION; lpNumberOfBytesRead: NativeUInt; begin while ReadProcessMemory(GetCurrentProcess, InitialAddr, @ER, SizeOf(EXCEPTION_REGISTRATION), lpNumberOfBytesRead) do begin Writeln('SEH Frame at Addr: ', IntToHex(NativeUInt(InitialAddr), 1), ', handler at addr: ', IntToHex(NativeUInt(ER.handler), 1)); InitialAddr := ER.prev; if DWORD(InitialAddr) <= 0 then Break; end; end; Having received the value of TEB.TIB.ExceptionList as a parameter, which points to the first EXCEPTION_REGISTRATION structure, it runs along the data chain of the structures, focusing on the prev value of this structure, which contains the address of the previous EXCEPTION_REGISTRATION structure. And the handler parameter contains the address of the exception handler if it happens.
It looks like this:
Well, CallStack will receive the following procedure:
procedure GetThreadCallStack(hThread: THandle); var StackFrame: TStackFrame; ThreadContext: PContext; MachineType: DWORD; begin // ThreadContext , VirtualAlloc // // ERROR_NOACCESS (998) ThreadContext := VirtualAlloc(nil, SizeOf(TContext), MEM_COMMIT, PAGE_READWRITE); try ThreadContext^.ContextFlags := CONTEXT_FULL; if not GetThreadContext(hThread, ThreadContext^) then Exit; ZeroMemory(@StackFrame, SizeOf(TStackFrame)); StackFrame.AddrPC.Mode := AddrModeFlat; StackFrame.AddrStack.Mode := AddrModeFlat; StackFrame.AddrFrame.Mode := AddrModeFlat; StackFrame.AddrPC.Offset := ThreadContext.Eip; StackFrame.AddrStack.Offset := ThreadContext.Esp; StackFrame.AddrFrame.Offset := ThreadContext.Ebp; MachineType := IMAGE_FILE_MACHINE_I386; while True do begin if not StackWalk(MachineType, GetCurrentProcess, hThread, @StackFrame, ThreadContext, nil, nil, nil, nil) then Break; if StackFrame.AddrPC.Offset <= 0 then Break; Writeln('CallStack Frame Addr: ', IntToHex(NativeUInt(StackFrame.AddrFrame.Offset), 1)); Writeln('CallStack Handler: ', IntToHex(NativeUInt(StackFrame.AddrPC.Offset), 1)); Writeln('CallStack Stack: ', IntToHex(NativeUInt(StackFrame.AddrStack.Offset), 1)); Writeln('CallStack Return: ', IntToHex(NativeUInt(StackFrame.AddrReturn.Offset), 1)); end; finally VirtualFree(ThreadContext, SizeOf(TContext), MEM_FREE); end; end; However, unlike the Delphi debugger, it will output data about the procedures for which the stack frame is generated, it will skip the rest.
The StackWalk (or StackWalk64) function is responsible for listing information about stack frames.
Now a caveat: if we apply this code to ourselves, then it will only be able to ottrassirovat one stack frame, after which the output will occur (you can check on the demo application ).
This will happen for the following reason: to correctly trace the StackWalk function, you must specify the parameters of the current stack frame (EBP and ESP / RBP and RSP for x64) and, in fact, the current code address (EIP or RIP register for x64). If we take this data from ourselves, this will happen at the moment when we call the GetThreadContext function, and we will begin to unwind the stack after exiting this function, where all three parameters become, to put it mildly, not valid. For this reason, tracing yourself by calling this function will not work.
This moment is desirable to take into account ...
I will dwell on getting information about 32-bit process streams under a 64-bit OS including 32 and 64-bit variants a little later, and now ...
3. Collect Heap Data
By itself, a Delphi application, as a rule, does not use heaps, it is more a prerogative of C ++ applications, but still, heaps are present here. Usually they are created and used by various third-party libraries for their needs.
The nuance in obtaining data about the heaps is that the HeapEntry elements of which each heap consists may be several thousand, and the second nuance is that the Heap32Next function re-rebuilds the entire list with each call, creating a rather sensitive delay (up to tens of seconds).
I already wrote about this unpleasant feature.
True, in that article, the code was rather approximate, just to demonstrate the principle itself, and it will not suit us, but a more combed version would be fine with it:
const RTL_HEAP_BUSY = 1; RTL_HEAP_SEGMENT = 2; RTL_HEAP_SETTABLE_VALUE = $10; RTL_HEAP_SETTABLE_FLAG1 = $20; RTL_HEAP_SETTABLE_FLAG2 = $40; RTL_HEAP_SETTABLE_FLAG3 = $80; RTL_HEAP_SETTABLE_FLAGS = $E0; RTL_HEAP_UNCOMMITTED_RANGE = $100; RTL_HEAP_PROTECTED_ENTRY = $200; RTL_HEAP_FIXED = (RTL_HEAP_BUSY or RTL_HEAP_SETTABLE_VALUE or RTL_HEAP_SETTABLE_FLAG2 or RTL_HEAP_SETTABLE_FLAG3 or RTL_HEAP_SETTABLE_FLAGS or RTL_HEAP_PROTECTED_ENTRY); STATUS_SUCCESS = 0; function CheckSmallBuff(Value: DWORD): Boolean; const STATUS_NO_MEMORY = $C0000017; STATUS_BUFFER_TOO_SMALL = $C0000023; begin Result := (Value = STATUS_NO_MEMORY) or (Value = STATUS_BUFFER_TOO_SMALL); end; function FlagToStr(Value: DWORD): string; begin case Value of LF32_FIXED: Result := 'LF32_FIXED'; LF32_FREE: Result := 'LF32_FREE'; LF32_MOVEABLE: Result := 'LF32_MOVEABLE'; else Result := ''; end; end; var I, A: Integer; pDbgBuffer: PRtlDebugInformation; pHeapInformation: PRtlHeapInformation; pHeapEntry: PRtrHeapEntry; dwAddr, dwLastSize: ULONG_PTR; hit_seg_count: Integer; BuffSize: NativeUInt; begin // .. Heap32ListFirst, Heap32ListNext, Heap32First, Heap32Next // , - // RtlQueryProcessDebugInformation , // // BuffSize := $400000; pDbgBuffer := RtlCreateQueryDebugBuffer(BuffSize, False); // while CheckSmallBuff(RtlQueryProcessDebugInformation(GetCurrentProcessId, RTL_QUERY_PROCESS_HEAP_SUMMARY or RTL_QUERY_PROCESS_HEAP_ENTRIES, pDbgBuffer)) do begin // , ... RtlDestroyQueryDebugBuffer(pDbgBuffer); BuffSize := BuffSize shl 1; pDbgBuffer := RtlCreateQueryDebugBuffer(BuffSize, False); end; if pDbgBuffer <> nil then try // if RtlQueryProcessDebugInformation(GetCurrentProcessId, RTL_QUERY_PROCESS_HEAP_SUMMARY or RTL_QUERY_PROCESS_HEAP_ENTRIES, pDbgBuffer) = STATUS_SUCCESS then begin // pHeapInformation := @pDbgBuffer^.Heaps^.Heaps[0]; // ... for I := 0 to pDbgBuffer^.Heaps^.NumberOfHeaps - 1 do begin // pHeapEntry := pHeapInformation^.Entries; dwAddr := DWORD(pHeapEntry^.u.s2.FirstBlock) + pHeapInformation^.EntryOverhead; dwLastSize := 0; A := 0; while A < Integer(pHeapInformation^.NumberOfEntries) do try hit_seg_count := 0; while (pHeapEntry^.Flags and RTL_HEAP_SEGMENT) = RTL_HEAP_SEGMENT do begin // RTL_HEAP_SEGMENT, // EntryOverhead dwAddr := DWORD(pHeapEntry^.u.s2.FirstBlock) + pHeapInformation^.EntryOverhead; Inc(pHeapEntry); Inc(A); Inc(hit_seg_count); // if A + hit_seg_count >= Integer(pHeapInformation^.NumberOfEntries - 1) then Continue; end; // , , // + if hit_seg_count = 0 then Inc(dwAddr, dwLastSize); // if pHeapEntry^.Flags and RTL_HEAP_FIXED <> 0 then pHeapEntry^.Flags := LF32_FIXED else if pHeapEntry^.Flags and RTL_HEAP_SETTABLE_FLAG1 <> 0 then pHeapEntry^.Flags := LF32_MOVEABLE else if pHeapEntry^.Flags and RTL_HEAP_UNCOMMITTED_RANGE <> 0 then pHeapEntry^.Flags := LF32_FREE; if pHeapEntry^.Flags = 0 then pHeapEntry^.Flags := LF32_FIXED; // Writeln('HeapID: ', I, ', entry addr: ', IntToHex(dwAddr, 8), ', size: ', IntToHex(pHeapEntry^.Size, 8), ' ', FlagToStr(pHeapEntry^.Flags)); // dwLastSize := pHeapEntry^.Size; // Inc(pHeapEntry); finally Inc(A); end; // Inc(pHeapInformation); end; end; finally RtlDestroyQueryDebugBuffer(pDbgBuffer); end; Readln; end. In short, using the RtlQueryProcessDebugInformation function call, RtlCreateQueryDebugBuffer and RtlQueryProcessDebugInformation creates a buffer containing information about the current process heaps. Then, knowing the structure of the data stored in it, we obtain this data in a cycle.
pDbgBuffer ^ .Heaps - stores lists of heaps (similar to THeapList32), and the records themselves are stored in pDbgBuffer ^ .Heaps ^ .Heaps [N] .Entries (analogue of THeapEntry32).
This code will display the following information:
In principle, I use heaps rarely for debugging, but sometimes this information may be useful.
4. Collect data on downloaded PE files
Now it's time to get information about executable files and libraries loaded into the process address space. There are several ways to do this (for example, by analyzing PEB.LoaderData), but we will proceed more simply.
As a rule, a separate region is allocated for the PE file (well, at least I have not met such that the PE image is loaded without alignment on the top of the region), therefore, using the code from the first chapter and checking the data of the first page of the region to match the PE file, we get a list of all loaded libraries and executable files.
The following code detects the presence of a valid PE file at the specified address:
function CheckPEImage(hProcess: THandle; ImageBase: Pointer; var IsPEImage64: Boolean): Boolean; var ReturnLength: NativeUInt; IDH: TImageDosHeader; NT: TImageNtHeaders; begin Result := False; IsPEImage64 := False; if not ReadProcessMemory(hProcess, ImageBase, @IDH, SizeOf(TImageDosHeader), ReturnLength) then Exit; if IDH.e_magic <> IMAGE_DOS_SIGNATURE then Exit; ImageBase := Pointer(NativeInt(ImageBase) + IDH._lfanew); if not ReadProcessMemory(hProcess, ImageBase, @NT, SizeOf(TImageNtHeaders), ReturnLength) then Exit; Result := NT.Signature = IMAGE_NT_SIGNATURE; IsPEImage64 := (NT.FileHeader.Machine = IMAGE_FILE_MACHINE_IA64) or (NT.FileHeader.Machine = IMAGE_FILE_MACHINE_ALPHA64) or (NT.FileHeader.Machine = IMAGE_FILE_MACHINE_AMD64); end; Well, more precisely, as he simply checks for the presence of ImageDosHeader and ImageNTHeader, focusing on their signatures. In principle, for 99% of cases this is enough.
The third parameter is simply informational, it shows whether the PE file is 64-bit.
You can get the path to the downloaded file by calling the function GetMappedFileName:
function GetFileAtAddr(hProcess: THandle; ImageBase: Pointer): string; begin SetLength(Result, MAX_PATH); SetLength(Result, GetMappedFileName(hProcess, ImageBase, @Result[1], MAX_PATH)); end; And now let's try to see what is being loaded into a regular console application:
var MBI: TMemoryBasicInformation; dwLength: NativeUInt; Address: PByte; IsPEImage64: Boolean; begin Address := nil; dwLength := SizeOf(TMemoryBasicInformation); while VirtualQuery(Address, MBI, dwLength) <> 0 do begin if CheckPEImage(GetCurrentProcess, MBI.BaseAddress, IsPEImage64) then begin Write(IntToHex(NativeUInt(MBI.BaseAddress), 8), ': ', GetFileAtAddr(GetCurrentProcess, MBI.BaseAddress)); if IsPEImage64 then Writeln(' (x64)') else Writeln(' (x32)'); end; Inc(Address, MBI.RegionSize); end; Readln; end. It turns out this picture:
64-bit library in a 32-bit application? Yes, easier simple :)
My application is 32-bit, Windows 7 x64 operating system. Judging by what is shown in the picture, four 64-bit libraries live and work in our 32-bit process, however, nothing unusual here is the so-called Wow64 ( Win32 emulation on 64-bit Windows ).
But at once it becomes clear where 64-bit analogs of 32-bit streams and heaps come from.
Now, in a good way, you need to get the addresses of sections of each PE file so that you can show them more clearly. All sections are aligned to the address of the beginning of the page and do not overlap with each other.
Let's do this with this code:
procedure GetInfoFromImage(const FileName: string; ImageBase: Pointer); var ImageInfo: TLoadedImage; ImageSectionHeader: PImageSectionHeader; I: Integer; begin if MapAndLoad(PAnsiChar(AnsiString(FileName)), nil, @ImageInfo, True, True) then try ImageSectionHeader := ImageInfo.Sections; for I := 0 to Integer(ImageInfo.NumberOfSections) - 1 do begin Write( IntToHex((NativeUInt(ImageBase) + ImageSectionHeader^.VirtualAddress), 8), ': ', string(PAnsiChar(@ImageSectionHeader^.Name[0]))); if IsExecute(ImageSectionHeader^.Characteristics) then Write(' Execute'); if IsWrite(ImageSectionHeader^.Characteristics) then Write(' Writable'); Writeln; Inc(ImageSectionHeader); end; finally UnMapAndLoad(@ImageInfo); end; Writeln; end; It uses the call to the MapAndLoad function, which, in addition to loading the file and checking its headers, also performs section alignment by calling NtMapViewOfSection.
For its own process, of course, the call to this function is redundant, since The required PE file is already loaded into the address space of the process, but since we need a more universal code to work with other processes, then we use this approach.
MapAndLoad is also good because it allows 64-bit processes to load 32-bit PE files (although this does not work for 32-bit processes), and in the future this feature will come in handy.
The essence of the code is as follows: after running MapAndLoad, we will have in our hands a filled TLoadedImage structure, the Sections parameter of which points to an array of TImageSectionHeader structures. Each of these structures has a VirtualAddress field, which is an offset from the library loading address. By adding the value of this field to the library hInstance, we get the address of the section.
The functions IsExecute and IsWrite check the characteristics of the section and return True if the section contains executable code (IsExecute) or data that can be modified (IsWrite). They look like this:
function IsExecute(const Value: DWORD): Boolean; begin Result := False; if (Value and IMAGE_SCN_CNT_CODE) = IMAGE_SCN_CNT_CODE then Result := True; if (Value and IMAGE_SCN_MEM_EXECUTE) = IMAGE_SCN_MEM_EXECUTE then Result := True; end; function IsWrite(const Value: DWORD): Boolean; begin Result := False; if (Value and IMAGE_SCN_CNT_UNINITIALIZED_DATA) = IMAGE_SCN_CNT_UNINITIALIZED_DATA then Result := True; if (Value and IMAGE_SCN_MEM_WRITE) = IMAGE_SCN_MEM_WRITE then Result := True; end; As a result of this code, we will see the following:
True, with this code there is another small nuance.
As you can see in the previous picture, the GetMappedFileName function returns the path to the downloaded file in the following form: "\ Device \ HarddiskVolume2 \ Windows \ System32 \ wow64cpu.dll", and the MapAndLoad function requires a normalized path like "C: \ Windows \ System32 \ wow64cpu. dll ".
The following code is responsible for bringing the path to the usual view:
function NormalizePath(const Value: string): string; const OBJ_CASE_INSENSITIVE = $00000040; STATUS_SUCCESS = 0; FILE_SYNCHRONOUS_IO_NONALERT = $00000020; FILE_READ_DATA = 1; ObjectNameInformation = 1; DriveNameSize = 4; VolumeCount = 26; DriveTotalSize = DriveNameSize * VolumeCount; var US: UNICODE_STRING; OA: OBJECT_ATTRIBUTES; IO: IO_STATUS_BLOCK; hFile: THandle; NTSTAT, dwReturn: DWORD; ObjectNameInfo: TOBJECT_NAME_INFORMATION; Buff, Volume: string; I, Count, dwQueryLength: Integer; lpQuery: array [0..MAX_PATH - 1] of Char; AnsiResult: AnsiString; begin Result := Value; // ZwOpenFile RtlInitUnicodeString(@US, StringToOleStr(Value)); // InitializeObjectAttributes FillChar(OA, SizeOf(OBJECT_ATTRIBUTES), #0); OA.Length := SizeOf(OBJECT_ATTRIBUTES); OA.ObjectName := @US; OA.Attributes := OBJ_CASE_INSENSITIVE; // ZwOpenFile , // , : // \SystemRoot\System32\ntdll.dll // \??\C:\Windows\System32\ntdll.dll // \Device\HarddiskVolume1\WINDOWS\system32\ntdll.dll // NTSTAT := ZwOpenFile(@hFile, FILE_READ_DATA or SYNCHRONIZE, @OA, @IO, FILE_SHARE_READ or FILE_SHARE_WRITE or FILE_SHARE_DELETE, FILE_SYNCHRONOUS_IO_NONALERT); if NTSTAT = STATUS_SUCCESS then try // , NTSTAT := NtQueryObject(hFile, ObjectNameInformation, @ObjectNameInfo, MAX_PATH * 2, @dwReturn); if NTSTAT = STATUS_SUCCESS then begin SetLength(AnsiResult, MAX_PATH); WideCharToMultiByte(CP_ACP, 0, @ObjectNameInfo.Name.Buffer[ObjectNameInfo.Name.MaximumLength - ObjectNameInfo.Name.Length {$IFDEF WIN64} + 4{$ENDIF}], ObjectNameInfo.Name.Length, @AnsiResult[1], MAX_PATH, nil, nil); Result := string(PAnsiChar(AnsiResult)); // ZwOpenFile // \Device\HarddiskVolume\- // SetLength(Buff, DriveTotalSize); Count := GetLogicalDriveStrings(DriveTotalSize, @Buff[1]) div DriveNameSize; for I := 0 to Count - 1 do begin Volume := PChar(@Buff[(I * DriveNameSize) + 1]); Volume[3] := #0; // // QueryDosDevice(PChar(Volume), @lpQuery[0], MAX_PATH); dwQueryLength := Length(string(lpQuery)); if Copy(Result, 1, dwQueryLength) = string(lpQuery) then begin Volume[3] := '\'; if lpQuery[dwQueryLength - 1] <> '\' then Inc(dwQueryLength); Delete(Result, 1, dwQueryLength); Result := Volume + Result; Break; end; end; end; finally ZwClose(hFile); end; end; This is already quite an old code, constantly used by me to bring it to a normal path. Its essence is that of the paths of the following types:
- \ SystemRoot \ System32 \ ntdll.dll
- \ ?? \ C: \ Windows \ System32 \ ntdll.dll
- \ Device \ HarddiskVolume1 \ WINDOWS \ system32 \ ntdll.dll
... get fixed "\ Device \ HarddiskVolume1 \ WINDOWS \ system32 \ ntdll.dll".
This is done by calling ZwOpenFile + NtQueryObject, after which all the disks in the system are simply searched and for each one QueryDosDevice is called, which returns a path in the same format. After that, the paths are compared and (if they match) the corresponding disk label is substituted to the transmitted path.
But this is the lyrics.
To be completely satisfied with yourself, it is advisable to output the same file directory as the PE file so that it is immediately clear where to look, for example, the import table, where UNWIND is sitting, etc.
This is done in fairly simple code:
procedure EnumDirectoryes(ImageBase: Pointer; ImageInfo: TLoadedImage; AddrStart, AddrEnd: NativeUInt); const DirectoryStr: array [0..14] of string = ('export', 'import', 'resource', 'exception', 'security', 'basereloc', 'debug', 'copyright', 'globalptr', 'tls', 'load_config', 'bound_import', 'iat', 'delay_import', 'com'); var I: Integer; dwDirSize: DWORD; DirAddr: Pointer; ReadlDirAddr: NativeUInt; begin for I := 0 to 14 do begin DirAddr := ImageDirectoryEntryToData(ImageInfo.MappedAddress, True, I, dwDirSize); if DirAddr = nil then Continue; ReadlDirAddr := NativeUint(ImageBase) + NativeUint(DirAddr) - NativeUint(ImageInfo.MappedAddress); if (ReadlDirAddr >= AddrStart) and (ReadlDirAddr < AddrEnd) then Writeln( IntToHex(ReadlDirAddr, 8), ': directory "', DirectoryStr[I], '"'); end; end; Having the TLoadedImage structure in hand, we can simply simply call the ImageDirectoryEntryToData function to get its address, although it will be tied to the address where the PE file is displayed. To translate it into real, you need to subtract the address from which the image is displayed from the current address, thus obtaining the offset from the beginning of the file, and add it to the ImageBase library.
In the end, you get this picture:
It is immediately obvious that, for example, the import / export / deferred import directories, etc., are located in the ".text" section of the msctf.dll library.
The resource directory sits in the ".rsrc" section, and relocs are also where they should be, but the "bound_import" directory falls out of the scheme.
Yes, indeed, this directory is not located directly in any of the sections of the library, such is its peculiarity. It usually goes right behind the PE header (although it can sometimes occur between sections). This directory is used to provide the mechanism of "bound imports", which is found mainly only in programs and libraries running in the OS.
Its essence is that all the addresses of imported functions are stitched into an executable file at the compilation stage, thus it is not necessary to perform unnecessary gestures, running around the normal import table in search of a function address.
But the overhead is also relevant, because as soon as any of the libraries declared in the bound import section changes, the application must be recompiled.
5. Process Environment Block (PEB) + KUSER_SHARED_DATA
Having data about threads, heaps and executable files, you can make a small utility right now that will display information in a readable form, but what else can you add?
At a minimum, it is highly desirable to receive and display information from the process environment block.
It can be accessed by calling the NtQueryInformationProcess function with the ProcessBasicInformation flag (a constant equal to zero). In this case, the hands will have a PROCESS_BASIC_INFORMATION structure, in which the PebBaseAddress field will contain the address PEB.
But this will be relevant only if the bits of the processes (the requester and of which the requested information) match. If we call this function from a 64-bit application as applied to a 32-bit one, we get the address of a 64-bit PEB, rather than a native 32-bit one.
To get access to Wow64PEB from a 64-bit application (let's call it that way), you need to call the NtQueryInformationProcess function with the ProcessWow64Information parameter (constant 26) and the size of the buffer equal to SizeOf (ULONG_PTR). In this case, instead of the PROCESS_BASIC_INFORMATION structure, the function will return a pointer to a 32-bit PEB, from which we will read out the information we need by means of ReadProcessMemory.
What is PEB?
Roughly speaking, this is not a strongly documented structure, most of which is intended for storing data used directly by the system. But this does not mean that it is not interesting to the developer of a conventional application. In particular, this structure contains a number of interesting fields, such as: the BeingDebugged flag, indicating whether the debugger is connected to the process; a pointer to PEB_LDR_DATA, which contains information about the modules loaded in the process; and a lot of the rest is quite useful information for a programmer, especially for someone who knows how to use it for their own purposes :)
This structure looks like this (declaration for Windows7 x86 / 64):
PPEB = ^TPEB; TPEB = record InheritedAddressSpace: BOOLEAN; ReadImageFileExecOptions: BOOLEAN; BeingDebugged: BOOLEAN; BitField: BOOLEAN; { BOOLEAN ImageUsesLargePages : 1; BOOLEAN IsProtectedProcess : 1; BOOLEAN IsLegacyProcess : 1; BOOLEAN IsImageDynamicallyRelocated : 1; BOOLEAN SkipPatchingUser32Forwarders : 1; BOOLEAN IsPackagedProcess : 1; BOOLEAN IsAppContainer : 1; BOOLEAN SpareBits : 1; } Mutant: HANDLE; ImageBaseAddress: PVOID; LoaderData: PVOID; ProcessParameters: PRTL_USER_PROCESS_PARAMETERS; SubSystemData: PVOID; ProcessHeap: PVOID; FastPebLock: PRTLCriticalSection; AtlThunkSListPtr: PVOID; IFEOKey: PVOID; EnvironmentUpdateCount: ULONG; UserSharedInfoPtr: PVOID; SystemReserved: ULONG; AtlThunkSListPtr32: ULONG; ApiSetMap: PVOID; TlsExpansionCounter: ULONG; TlsBitmap: PVOID; TlsBitmapBits: array[0..1] of ULONG; ReadOnlySharedMemoryBase: PVOID; HotpatchInformation: PVOID; ReadOnlyStaticServerData: PPVOID; AnsiCodePageData: PVOID; OemCodePageData: PVOID; UnicodeCaseTableData: PVOID; KeNumberOfProcessors: ULONG; NtGlobalFlag: ULONG; CriticalSectionTimeout: LARGE_INTEGER; HeapSegmentReserve: SIZE_T; HeapSegmentCommit: SIZE_T; HeapDeCommitTotalFreeThreshold: SIZE_T; HeapDeCommitFreeBlockThreshold: SIZE_T; NumberOfHeaps: ULONG; MaximumNumberOfHeaps: ULONG; ProcessHeaps: PPVOID; GdiSharedHandleTable: PVOID; ProcessStarterHelper: PVOID; GdiDCAttributeList: ULONG; LoaderLock: PRTLCriticalSection; NtMajorVersion: ULONG; NtMinorVersion: ULONG; NtBuildNumber: USHORT; NtCSDVersion: USHORT; PlatformId: ULONG; Subsystem: ULONG; MajorSubsystemVersion: ULONG; MinorSubsystemVersion: ULONG; AffinityMask: ULONG_PTR; {$IFDEF WIN32} GdiHandleBuffer: array [0..33] of ULONG; {$ELSE} GdiHandleBuffer: array [0..59] of ULONG; {$ENDIF} PostProcessInitRoutine: PVOID; TlsExpansionBitmap: PVOID; TlsExpansionBitmapBits: array [0..31] of ULONG; SessionId: ULONG; AppCompatFlags: ULARGE_INTEGER; AppCompatFlagsUser: ULARGE_INTEGER; pShimData: PVOID; AppCompatInfo: PVOID; CSDVersion: UNICODE_STRING; ActivationContextData: PVOID; ProcessAssemblyStorageMap: PVOID; SystemDefaultActivationContextData: PVOID; SystemAssemblyStorageMap: PVOID; MinimumStackCommit: SIZE_T; FlsCallback: PPVOID; FlsListHead: LIST_ENTRY; FlsBitmap: PVOID; FlsBitmapBits: array [1..FLS_MAXIMUM_AVAILABLE div SizeOf(ULONG) * 8] of ULONG; FlsHighIndex: ULONG; WerRegistrationData: PVOID; WerShipAssertPtr: PVOID; pContextData: PVOID; pImageHeaderHash: PVOID; TracingFlags: ULONG; { ULONG HeapTracingEnabled : 1; ULONG CritSecTracingEnabled : 1; ULONG LibLoaderTracingEnabled : 1; ULONG SpareTracingBits : 29; } CsrServerReadOnlySharedMemoryBase: ULONGLONG; end; By the way, compare this structure with the one that is officially available in MSDN .
For Window 2000 / XP / 2003 there will be small changes, but not very critical ones.
I will not paint each field, those who work with PEB and so know? what exactly they need, but in some fields I will focus your attention.
So:
- The BeingDebugged field - in the third part of the article about the debugger, I showed one of the ways to bypass this detection by means of an application memory patch. The essence of the approach was just to determine the address of the PEB and change the value of the BeingDebugged parameter to zero, after which the IsDebuggerPresent function, which is oriented on this field, began to return False, saying that? that she did not find the debugger.
- ImageBaseAddress — hInstance ( ImageBase PE ).
- LoaderData — , , , , , . , :)
- ProcessParameters — ParamStr/GetCurrentDir .. ? . .
- , , CSDVersion. , , NtMajorVersion/NtMinorVersion/NtBuildNumber .
Well, and so on - you can continue for a long time.
Most of these fields occupy their pages in the address space of the process. For example, ProcessParameters usually sits in one of the heaps created by the loader, environment variables are also located somewhere in that area.
If we want to visualize all of this (and I am leading to this), we must have this data on hand in order to have something to display in the final application.
Agree, it is much more pleasant to have on your hands, instead of a certain block of binary data, something like this:
But there is also KUSER_SHARED_DATA.
This is also the structure used by the system, and you constantly encounter it by calling the same GetTickCount or IsProcessorFeaturePresent.
For example, NtSystemRoot sits exactly in it, and, again, why list everything, it's easier to see:
- Want to know what process is active without calling GetForegroundWindow — read ConsoleSessionForegroundProcessId.
- Are you trying to slip the left version of Win to turn off part of the security system that was not designed for the previous OS? Read current values ​​from the NtMajorVersion / NtMinorVersion fields ...
However, perhaps, here we will stop for now ...
6. TRegionData
At the theoretical part ended and it was time to put it all into practice.
First of all, you need to decide how to store information about the regions. Preparing for the article, I wrote a set of classes allocated to the common namespace "MemoryMap", you can find them in the demo examples .
IMPORTANT!!!
This set of classes was developed taking into account the innovations that are present in the Delphi XE4, under older versions of Delphi, its performance has not been tested and is not guaranteed.
The information for each region will be stored in the TRegionData class, implemented in the “MemoryMap.RegionData.pas” module.
It looks something like this (in the process of developing a project, the class declaration may change).
TRegionData = class private FParent: TRegionData; FRegionType: TRegionType; FMBI: TMemoryBasicInformation; FDetails: string; FRegionVisible: Boolean; FHiddenRegionCount: Integer; FTotalRegionSize: NativeUInt; FHeap: THeapData; FThread: TThreadData; FPEBData: TSystemData; FSection: TSection; FContains: TList; FDirectories: TList; FShared: Boolean; FSharedCount: Integer; FFiltered: Boolean; protected ... public constructor Create; destructor Destroy; override; property RegionType: TRegionType read FRegionType; property MBI: TMemoryBasicInformation read FMBI; property Details: string read FDetails; property RegionVisible: Boolean read FRegionVisible; property HiddenRegionCount: Integer read FHiddenRegionCount; property Parent: TRegionData read FParent; property TotalRegionSize: NativeUInt read FTotalRegionSize; property Heap: THeapData read FHeap; property Thread: TThreadData read FThread; property SystemData: TSystemData read FPEBData; property Section: TSection read FSection; property Directory: TList read FDirectories; property Contains: TList read FContains; end; In order:
Each region, as a rule, stores in itself the data of one type.
Those.for heaps, stream stacks, PE files, its own page region is allocated.
The RegionType property is responsible for storing the region type. This is an enumeration type, declared as follows:
// TRegionType = ( rtDefault, rtHeap, // rtThread, // TEB rtSystem, // (PEB/KUSER_SHARED_DATA ..) rtExecutableImage // PE ); The region parameters obtained by calling VirtualQueryEx are stored in the MBI field.
A brief description of the region is stored in Details. Anything can be stored in it, for example, the path to the displayed PE file, if present, a string description of the stream ID, etc ...
The following three parameters are used to organize the tree structure.
One of the regions is the root node (root), the rest are subsidiaries.
The RegionVisible flag indicates whether a region is a root node.
The HiddenRegionCount property contains the number of subregions (the AllocationBase of which is equal to the BaseAddress of the root).
Well, the parameter Parent stores a link to the root.
The work was not entirely optimal, it would be possible to organize a classic tree, but at the current time there is simply no time to redo it, maybe someday later :)
TotalRegionSize contains the total size of all subregions, including the root one.
In case the region contains a heap, the data on its first element is placed in the Heap parameter, which is the following structure:
THeapEntry = record Address: ULONG_PTR; Size: SIZE_T; Flags: ULONG; end; THeapData = record ID: DWORD; Wow64: Boolean; Entry: THeapEntry; end; The remaining heap items located within the region are placed in the Contains field.
In general, the Contains field may contain data of many types.
TContainItemType = (itHeapBlock, itThreadData, itStackFrame, itSEHFrame, itSystem); TContainItem = record ItemType: TContainItemType; function Hash: string; case Integer of 0: (Heap: THeapData); 1: (ThreadData: TThreadData); 2: (StackFrame: TThreadStackEntry); 3: (SEH: TSEHEntry); 4: (System: TSystemData); end; Next comes the Thread field, which stores information about the stream that the region uses to store its own data.
type TThreadInfo = (tiNoData, tiExceptionList, tiStackBase, tiStackLimit, tiTEB, tiThreadProc); type TThreadData = record Flag: TThreadInfo; ThreadID: Integer; Address: Pointer; Wow64: Boolean; end; If there are a lot of stream data within a region (for example, a list of SEH frames or a CallStack stream), they are also placed in the Contains field.
Data from system structures (fields of PEB / TEB structures, etc.) are placed in the SystemData field, which is an entry from the data address and their descriptions.
Also this data can be placed in the Contains field.
If the region belongs to one of the sections of the PE file, the section data is placed in the Section parameter. Well, the list of file directories is located in the Directory field.
That somehow so briefly. Now, to present the data on the process memory card, we need to obtain a list of regions, create an instance of the TRegionData class for each of them, and initialize the fields of the created object with the required information.
The TMemoryMap class is responsible for this ...
7. TMemoryMap
This class is implemented in the "MemoryMap.Core.pas" module.
Its task is literally reduced to three main stages:
- Retrieving a list of all selected regions in the memory of the specified application, data on threads / heaps / loaded images, etc.
- Creating a TRegionData list and filling its fields with the received information.
- Saving / loading data, filtering data.
In practice, everything looks more complicated.
The basic procedure for collecting information looks like this:
function TMemoryMap.InitFromProcess(PID: Cardinal; const ProcessName: string): Boolean; var ProcessLock: TProcessLockHandleList; begin Result := False; FRegions.Clear; FModules.Clear; FFilter := fiNone; ProcessLock := nil; // FProcess := OpenProcess( PROCESS_QUERY_INFORMATION or PROCESS_VM_READ, False, PID); if FProcess = 0 then RaiseLastOSError; try FPID := PID; FProcessName := ProcessName; // FProcess64 := False; {$IFDEF WIN64} if not IsWow64(FProcess) then FProcess64 := True; {$ELSE} // 32 , 64- // if Is64OS and not IsWow64(FProcess) then raise Exception.Create('Can''t scan process.'); {$ENDIF} // if SuspendProcessBeforeScan then ProcessLock := SuspendProcess(PID); try FSymbols := TSymbols.Create(FProcess); try FPEImage := TPEImage.Create; try FWorkset := TWorkset.Create(FProcess);; try // GetAllRegions; finally FWorkset.Free; end; {$IFDEF WIN64} // 32 AddWow64HeapsData; {$ENDIF} // AddThreadsData; // AddHeapsData; // Process Environment Block AddPEBData; // PE AddImagesData; finally FPEImage.Free; end; finally FSymbols.Free; end; finally if SuspendProcessBeforeScan then ResumeProcess(ProcessLock); end; // SortAllContainsBlocks; // CalcTotal; // UpdateRegionFilters; finally CloseHandle(FProcess); end; end; I gave the approximate code of the GetAllRegions / AddThreadsData / AddHeapsData and AddImagesData procedures in the first four chapters and I will not focus on it, but it is advisable to deal with the rest.
The very first step after opening the process is determining the bit depth of the process.
This is necessary for the reason that if the bits of the processes (current and for which we receive information) do not match, some additional actions need to be taken.
The general scheme is as follows:
- A 32-bit process can receive data on a 32-bit under a 32-bit OS in full.
- A 64-bit process can receive data on a 64-bit one in full.
- A 32-bit process MAY NOT get 64-bit data.
- 32- 32- 64- , .
- 64- 32-, .
If everything is clear with the first two points, then the other three will be discussed in more detail.
The reason that the 32-bit process will not be able to get data on the 64-bit is simple: the pointer size will not allow, plus ReadProcessMemory will periodically generate an ERROR_PARTIAL_COPY error.
But getting data from a 32-bit process in a 64-bit OS is much more cunning.
As I said earlier, in a 32-bit application, four 64-bit libraries are loaded that create their heaps / streams.
If we get a list of heaps and streams from a 32-bit application, we will only see data related to 32 bits, data on 64-bit analogues cannot be obtained.
The same will happen in the case of requesting data on a 32-bit process from a 64-bit one, only data related to 64 bits will be returned. Although in this case there is an option to get them in part.
In particular, access to 32-bit PEB is made by calling such a function:
const ProcessWow64Information = 26; ... NtQueryInformationProcess(FProcess, ProcessWow64Information, @FPebWow64BaseAddress, SizeOf(ULONG_PTR), @ReturnLength) Access to 32-bit TEB can be obtained by reading the address from 64-bit TEB, which is stored in the NtTIB.ExceptionList parameter.
// 64 TEB TIB.ExceptionList Wow64TEB if not ReadProcessMemory(hProcess, TIB.ExceptionList, @WOW64_NT_TIB, SizeOf(TWOW64_NT_TIB), lpNumberOfBytesRead) then Exit; You can get the context of a 32-bit stream for CallStack promotion like this:
const ThreadWow64Context = 29; ... ThreadContext^.ContextFlags := CONTEXT_FULL; if NtQueryInformationThread(hThread, ThreadWow64Context, ThreadContext, SizeOf(TWow64Context), nil) <> STATUS_SUCCESS then Exit; Or by calling the Wow64GetThreadContext function.
But I don’t know how to get data about 32-bit heaps from a 64-bit process in a legal way. The only option that I use now is to send a command to a 32-bit process, which collects data on 32-bit heaps and gives them back to the 64-bit one (the handler in the AddWow64HeapsData function does about that).
Now, when we have figured out the process bit definition and why we need it, let's go further, namely, the SuspendProcess function call.
In an amicable way, this is only necessary so that the data in the remote process does not change to irrelevant at the time of their reading. However, I usually use this set of classes in two cases, for my own application or for an application under debugger. In both cases, it is not necessary to freeze the threads, but if some third-party application is being analyzed, then why not?
After freezing the remote process, three auxiliary classes are created.
- TSymbols - I'll talk about him in the next chapter.
- TPEImage - this class contains methods that allow you to get information about the PE file, described in the fourth chapter. Made exclusively for convenience.
- TWorkset is another auxiliary class whose task is to obtain information about shared memory.
In fact, TWorkset stores a list of structures of the form:
TShareInfo = record Shared: Boolean; SharedCount: Byte; end; These structures are stored in the dictionary and each is associated with a specific page address.
Parameters are simple:
- Shared - is the page public?
- SharedCount - how many links are on the page
This data is obtained in the following way, in which everything comes down to calling the QueryWorkingSet function:
procedure TWorkset.InitWorksetData(hProcess: THandle); const {$IFDEF WIN64} AddrMask = $FFFFFFFFFFFFF000; {$ELSE} AddrMask = $FFFFF000; {$ENDIF} SharedBitMask = $100; SharedCountMask = $E0; function GetSharedCount(Value: ULONG_PTR): Byte; inline; begin Result := (Value and SharedCountMask) shr 5; end; var WorksetBuff: array of ULONG_PTR; I: Integer; ShareInfo: TShareInfo; begin SetLength(WorksetBuff, $400000); while not QueryWorkingSet(hProcess, @WorksetBuff[0], Length(WorksetBuff) * SizeOf(ULONG_PTR)) do SetLength(WorksetBuff, WorksetBuff[0] * 2); for I := 0 to WorksetBuff[0] - 1 do begin ShareInfo.Shared := WorksetBuff[I] and SharedBitMask <> 0; ShareInfo.SharedCount := GetSharedCount(WorksetBuff[I]); try FData.Add(Pointer(WorksetBuff[I] and AddrMask), ShareInfo); except on E: EListError do ; else raise; end; end; end; This function returns an array of ULONG_PTR, each element of which stores data as follows: the first five bits store the security attributes of the page; the next three bits are the number of processes by which this page is available; another bit indicates the general availability of the page; well, then comes the address of the page itself.
More details can be found here: PSAPI_WORKING_SET_BLOCK .
In fact, this is just an information class, no more, no less.
However, back to our code.
The next steps are:
- GetAllRegions - analogue of the code from the first chapter.
- AddThreadsData - analogue of the code from the second chapter.
- AddHeapsData - analogue of the code from the third chapter.
- AddPEBData - output data on the structure of the fifth chapter.
- AddImagesData - analogue code from the fourth chapter.
As you can see, I have already told all the interesting (well, almost) :)
The remaining steps are not interesting, except for the call to UpdateRegionFilters.
It performs a utilitarian function, namely, it excludes regions from the list that are currently unnecessary (well, for example, removes regions with unallocated memory, etc.).
This procedure will be invoked continuously when the filter is changed through the Filter property.
However, all this you can, if you want to see from the code of the class itself.
Working with him is quite simple:
var AMemoryMap: TMemoryMap; M: TMemoryStream; I: Integer; begin try M := TMemoryStream.Create; try // AMemoryMap := TMemoryMap.Create; try // AMemoryMap.InitFromProcess(GetCurrentProcessId, ''); // , AMemoryMap.SaveToStream(M); // finally AMemoryMap.Free; end; // -, M.Position := 0; // AMemoryMap := TMemoryMap.Create; try // AMemoryMap.LoadFromStream(M); // AMemoryMap.Filter := fiNone; // // AMemoryMap.ShowEmpty := True; // for I := 0 to AMemoryMap.Count - 1 do Writeln(NativeUInt(AMemoryMap[I].MBI.BaseAddress)); finally AMemoryMap.Free; end; finally M.Free; end; except on E: Exception do Writeln(E.ClassName, ': ', E.Message); end; Readln; end. As they say, he wrote for himself, therefore, working with this class is easy :)
8. TSymbols - working with characters
The essence of this class is to obtain more detailed information about the address in the process. Well, for example, in the second chapter we received the CallStack flow (or handlers of SEH frames) and these were just some kind of addresses. But it’s much more interesting to see something like this image instead of dry numbers:
This is done very simply - just call the function SymGetSymFromAddr, but there are a few nuances.
Let's first look at the code:
function TSymbols.GetDescriptionAtAddr(Address, BaseAddress: ULONG_PTR; const ModuleName: string): string; const BuffSize = $7FF; {$IFDEF WIN64} SizeOfStruct = SizeOf(TImagehlpSymbol64); MaxNameLength = BuffSize - SizeOfStruct; var Symbol: PImagehlpSymbol64; Displacement: DWORD64; {$ELSE} SizeOfStruct = SizeOf(TImagehlpSymbol); MaxNameLength = BuffSize - SizeOfStruct; var Symbol: PImagehlpSymbol; Displacement: DWORD; {$ENDIF} begin Result := ''; if not FInited then Exit; GetMem(Symbol, BuffSize); try Symbol^.SizeOfStruct := SizeOfStruct; Symbol^.MaxNameLength := MaxNameLength; Symbol^.Size := 0; SymLoadModule(FProcess, 0, PAnsiChar(AnsiString(ModuleName)), nil, BaseAddress, 0); try if SymGetSymFromAddr(FProcess, Address, @Displacement, Symbol) then Result := string(PAnsiChar(@(Symbol^).Name[0])) + ' + 0x' + IntToHex(Displacement, 4) else begin // SymLoadModule(FProcess, 0, PAnsiChar(AnsiString(ModuleName)), nil, BaseAddress, 0); if SymGetSymFromAddr(FProcess, Address, @Displacement, Symbol) then Result := string(PAnsiChar(@(Symbol^).Name[0])) + ' + 0x' + IntToHex(Displacement, 4); end; finally SymUnloadModule(FProcess, BaseAddress); end; finally FreeMem(Symbol); end; if Result = '' then Result := ExtractFileName(ModuleName) + ' + 0x' + IntToHex(Address - BaseAddress, 1); end; To correctly obtain the description of the name of the function that owns the address, you need to know the path to the library that owns the function, or the address to which this library is loaded (the code uses both parameters). These parameters are required for the SymLoadModule function.
The second caveat is that the call to the SymGetSymFromAddr function can sometimes fail. The reason is not clear to me, but on the Internet they periodically describe this situation and how to solve it is to call the SymLoadModule function again without calling the SymUnloadModule. In such a strange behavior did not understand - but really helps.
The last of the nuances is that this function will return a valid description of the address only when this information is present (characters are loaded from an external file or they are included in the required module).
This information is not very important when debugging, but it simplifies it a little.
This is, for example, the standard stack of the Chrome browser stream (CallStack + SEH frames):
More useful information that symbols can provide is a list of library functions exported and their current addresses.
In the TSymbols class, this information is obtained by calling the GetExportFuncList procedure and looks like this:
function SymEnumsymbolsCallback(SymbolName: LPSTR; SymbolAddress: ULONG_PTR; SymbolSize: ULONG; UserContext: Pointer): Bool; stdcall; var List: TStringList; begin List := UserContext; List.AddObject(string(SymbolName), Pointer(SymbolAddress)); Result := True; end; procedure TSymbols.GetExportFuncList(const ModuleName: string; BaseAddress: ULONG_PTR; Value: TStringList); begin SymLoadModule(FProcess, 0, PAnsiChar(AnsiString(ModuleName)), nil, BaseAddress, 0); try if not SymEnumerateSymbols(FProcess, BaseAddress, @SymEnumsymbolsCallback, Value) then begin SymLoadModule(FProcess, 0, PAnsiChar(AnsiString(ModuleName)), nil, BaseAddress, 0); SymEnumerateSymbols(FProcess, BaseAddress, @SymEnumsymbolsCallback, Value) end; finally SymUnloadModule(FProcess, BaseAddress); end; end; It all comes down to calling SymEnumerateSymbols, which is passed the address of the callback function.
When you call it, the SymbolName parameter will contain the name of the exported function, and SymbolAddress will contain its address.
This is quite enough to display the following label to the user:
In more detail the implementation of this class, including omitted calls SymSetOptions and SymInitialize, you can see in the module "MemoryMap.Symbols.pas".
9. ProcessMemoryMap
Well, here we come to the final part of the article.
As I said earlier, I use the MemoryMap class set in two versions:
1. Integrating it into the output of EurekaLog by overlapping its OnAttachedFilesRequest handler, where I add the current process map that is relevant at the time of the exception, and dumps of all Private Regions (pages not associated with certain data with the MEM_PRIVATE flag) and thread stacks, plus some of the information from PEB. Usually this is enough to analyze the causes of the error.
2. I use it as an alternative tool for analyzing the application being debugged.
For the second option, a separate utility was implemented, which works directly with the MemoryMap classes, plus adds some additional functionality.
I will not describe its source code, I’ll just go over a bit of functionality.
From the interface, it is almost one to one reminds VMMap. However, this was originally planned, because such an interface is most convenient for analysis.
At the top is a list with general information by region, grouped by type, it is also a filter.
At the moment, it represents the following functionality:
1. View the contents of the memory at the specified address (Ctrl + Q).
This functionality, in principle, is present in the Delphi debugger in the CPU View window, but the features of this mode are much greater. For example, in the case of viewing the PEB field, data will be displayed in a different form:
This is how the process parameter block will look like:
Well, and so on. In total, at this point, the utility can display mapped data in the following structures:
- PEB - Process Environment Block (32/64)
- TEB - Thread Environment Block (32/64)
- KUSER_SHARED_DATA
- PE Header (IMAGE_DOS_HEADER / IMAGE_NT_HEADER / IMAGE_FILE_HEADER / IMAGE_OPTIONAL_HEADER (32/64) / IMAGE_DATA_DIRECTORY / IMAGE_SECTION_HEADERS)
- Process Parameters (32/64)
This list is not final, periodically new structures will be added to it.
2. Search for data in the process memory (Ctrl + F):
This functionality in the Delphi debugger, unfortunately, is missing.
The search can be performed either by Ansi or by Unicode string, or simply by abstract HEX buffer. When searching, you can specify the address of the beginning of the search, as well as a flag indicating the need to search in pages that are accessible only for reading.
The result is displayed as a window with the memory dump shown above.
3. Comparator of two memory cards. Included in the settings.
Allows you to find differences between two memory cards and displays them as text.
Only the maps themselves are compared, not the data. Those.if at some address 4 bytes have changed, this change will not be displayed. But in the event that the size of the region has changed, the heap has been deleted, the file has been downloaded / downloaded, etc. - all this will be displayed in the comparison results.
It is possible to compare both the current map snapshot with the previously saved one, and when updating the snapshot using the F5 hot key.
4. Dump of memory.
Also missing in the Delphi debugger functionality. Allows you to save to disk the contents of the memory of the specified region or data from the specified address.
5. Output of all available exported functions from all libraries loaded in the analyzed process (Ctrl + E).
As well as a quick search for a function by its name or address.
So far, the current functionality is enough for me personally, and I haven’t added a new one, but in the future this utility will be developed.
ProcessMemoryMap is an open source project.
Her latest stable release is always available at the link: http://rouse.drkb.ru/winapi.php#pmm2
GitHub repository with the latest code changes can be found here: https://github.com/AlexanderBagel/ProcessMemoryMap
Direct link to the source code : https://github.com/AlexanderBagel/ProcessMemoryMap/archive/master.zip
Direct link to the latest build: http://rouse.drkb.ru/files/processmm_bin.zip
For self-assembly, you will need the installed package of components Virtual TreeView version 5 and higher: http://www.soft-gems.net/ .
The build is carried out using Delphi XE4 and higher in the “Win32 / Release” mode, and the 64-bit version of this utility will be automatically assembled and connected (as a resource).
Under older versions of Delphi, the health of ProcessMemoryMap has not been tested.
10. As a conclusion
Well, I hope this material will be useful for you. I, of course, passed only at the very top, for if you disclose all the material in more detail, the volume of the article will increase enormously.
Therefore, here are some links where you can find out a little more information.
Information on system structures of TEB / PEB, etc. can be found here:
http://processhacker.sourceforge.net/
http://redplait.blogspot.ru/
http://www.reactos.org/ru
Information about PE files:
http://msdn.microsoft.com/ en-us / magazine / ms809762.aspx
SEH Information:
http://msdn.microsoft.com/en-us/library/ms680657(v=VS.85).aspx
http://www.microsoft.com/msj /0197/exception/exception.aspx
http://qxov.narod.ru/articles/seh/seh.html
The source code of all demo examples can be picked up at this link . Many thanks to the
forum “Masters of Delphi” for their repeated assistance in the preparation of the article.
Personal thanks for reading the material to Dmitry aka "Ptiburdukov's brother", Andrey Vasilyev aka "Inovet", and also to Sergey aka "Cartman".
Good luck.
Source: https://habr.com/ru/post/202242/
All Articles