Using an alternative memory allocator in a C / C ++ project

This article is written primarily for C / C ++ programmers who use Visual Studio 2013 in their work. Since, as they say, totally windows guy, I cannot appreciate the usefulness of this article for programmers who do not use this environment in their work. So.



It is no secret that the standard new / delete / malloc / free allocator in C / C ++ does not shine with speed. Of course, everything depends on the implementation, but if we talk about it from Microsoft, then this is a fact. In addition, the standard implementation of the allocator has another fatal flaw - memory fragmentation. If there are frequent memory allocations / releases in your program, you may find that after several hours of work your program has dropped due to lack of memory, although there is still enough free memory - just as a result of fragmentation in the allocator pool, there is no sufficiently large free area left. (This, by the way, is an absolutely real case that occurred on one of the projects in which I was directly involved.)



Fortunately, there are allocators that are free from both of these drawbacks. At one time I tried dlmalloc and since then I always use it in my projects.



I want to share with you the way to connect dlmalloc to a project in Visual Studio C / C ++.

The method that I use is remarkable in that it allows you to use an alternative allocator absolutely for all allocations that can happen in your program. Yes, the simple way (i.e., replacing malloc calls with dlmalloc) does not achieve this effect. For example, you hooked up a third-party library that allocates memory using malloc. Moreover, some calls to standard functions from stdlib also allocate memory with the malloc function and you have no way to prevent this ... Or is there? There is.

')

The essence of the method

The essence of the method is to force the linker to use your malloc / free / new / delete implementation instead of the standard one. But how to do that? When I first began to investigate this issue, my first attempt was a rather stupid idea: to patch the malloc / free body in runtime in memory, putting an unconditional jmp there on my code. Do I need to explain why this idea is stupid? Although everything worked, but this method did not bring joy. As a result, I came to another solution, namely, to prohibit the linker to generally use the standard library libcmt, in which the standard allocator is located. But this method also had a significant drawback, namely, in this library there were a lot of other useful and not-so-useful functions, to which it was absolutely impossible to write stubs.

Then I began to investigate the possibility of taking the standard library (literally the file libcmt.lib) and throwing out all unnecessary from it. It turned out that this is possible and in the end I use this method.

Technical details

First run the command:

lib.exe /LIST libcmt.lib 

At the output we get a list of obj files that this library contains. For libcmt.lib from Visual Studio 2013, this list looks like this:

 f:\binaries\Intermediate\vctools\crt_bld\SELF_X86\crt\prebuild\INTEL\mt_lib\chandler4.obj f:\binaries\Intermediate\vctools\crt_bld\SELF_X86\crt\prebuild\INTEL\mt_lib\chandler4gs.obj f:\binaries\Intermediate\vctools\crt_bld\SELF_X86\crt\prebuild\INTEL\mt_lib\chkesp.obj f:\binaries\Intermediate\vctools\crt_bld\SELF_X86\crt\prebuild\INTEL\mt_lib\eh3valid.obj f:\binaries\Intermediate\vctools\crt_bld\SELF_X86\crt\prebuild\INTEL\mt_lib\exsup.obj f:\binaries\Intermediate\vctools\crt_bld\SELF_X86\crt\prebuild\INTEL\mt_lib\exsup2.obj ... (  ) 

Fortunately, almost all functions for working with memory are in separate obj files, which, in fact, makes this method possible.

Those. it remains for us to cut out all unnecessary obj files from the library body.

The lib.exe utility with the / remove switch does just what we need.

Implementation

Actually, I put the source code on a githaba .



If you already have Visual Studio 2013 installed, all you have to do is run make_libcmt_nomem.cmd , which will do all the work and create a truncated libcmt_nomem.lib file, which can be connected instead of full-fledged libcmt.



In its work, the script uses the unix utility grep. If you do not have UnixUtils installed, I strongly recommend to do it (for example, from here ).



But that is not all. We got rid of the standard allocator. But the trouble is that at the same time we got rid of some standard functionality, which, alas, is inseparable from the allocator. Therefore, I have written the necessary stubs, which you can find in the include / crtfunc.h file (ibid, on the githaba ).

Mode of application

1. We get a cropped version of the standard library using the make_libcmt_nomem.cmd script and put it in a place accessible to the linker;
2. We disable the use of the standard library libcmt (Ignore Specific Default Libraries "" libcmt "in the options of the linker Configuration Properties-> Linker-> Input) in the project;
3. In any c ++ file in the project we make "#include crtfunc.h" from the sources;
4. We connect dlmalloc to the project.

I do not describe each item in detail, because if you read this article and understand it, you don’t need the details. The only point: it is necessary to connect the crtfunc.h in a C ++ (not C) file. If your project is written in C, you should add an empty .cpp file to the project and include crtfunc.h in it. However, no one forbids you to pick up the file.



Ps. In fact, not a single dlmalloc. There are other, very worthy allocators. Source files are designed for dlmalloc, but this is not critical. Minimal intervention in crtfunc.h can be achieved using any other allocator.



After the release of Visual Studio 2015, all of the above is no longer relevant.