Management and cleaning in D

Good day, Habr!

We all know that D uses the garbage collector. He manages the allocation of memory. It is used by implementations of such built-in types as associative and dynamic arrays, strings (which are also arrays), exceptions, delegates. Its use is also integrated into the syntax of the language (concatenation, new operator). GC removes responsibility and load from the programmer, allows you to write more compact, understandable and secure code. And these are perhaps the most important advantages of a garbage collector. Should I give it up? Payback for the use of the collector will be excessive use of memory, which is unacceptable with very limited resources and pauses of all threads (stop-the-world) to the assembly itself. If these points are critical for you, welcome under cat.

How bad is it?

First you need to find out, is it bad at all?

You can use valgrind , its memcheck tool (by default) will show how many times the program allocated and released the memory, as well as its amount (row total heap usage).
But valgrind will not be able to show GC usage statistics. Fortunately, this is built into runtime D (dmd only). The garbage collector of an already compiled program can be configured and profiled as follows:

app "--DRT-gcopt=profile:1 minPoolSize:16" program args 

The first argument (string) is processed in runtime and does not reach the main function.
Supported parameters:

• disable: 0 | 1 - disabling the collector
• profile: 0 | 1 - profiling with output at completion
• initReserve: N - memory reserved at startup (MB)
• minPoolSize: N - initial and minimum pool size (MB)
• maxPoolSize: N - maximum pool size (MB)
• incPoolSize: N - increment pool (MB)
• heapSizeFactor: N - the ratio of the target heap size to the used memory

When profiling is enabled, the output of the program after its completion will be something like this:

  Number of collections: 101 Total GC prep time: 10 milliseconds Total mark time: 3 milliseconds Total sweep time: 3 milliseconds Total page recovery time: 0 milliseconds Max Pause Time: 0 milliseconds Grand total GC time: 17 milliseconds GC summary: 67 MB, 101 GC 17 ms, Pauses 13 ms < 0 ms 

')

Life without assembly (almost)

If, after all the tests and GC settings, the result remains unsatisfactory, you can resort to some tricks.

Do not need a collector - do not use

Seriously? Is that allowed?
In the critical sections of the program, the collector can simply be disabled:

 import core.memory; ... GC.disable(); ... 

And when "there will be time for cleaning" turn it back on or immediately start:

 ... GC.enable(); GC.collect(); // enable  collect   ,    ... 

During completion, the program once again starts the garbage collector, regardless of the state of its inclusion.
When using this technique, it is important to remember that the memory continues to be allocated and in the case when it is not enough, the program will be terminated by the OS.

Use the right types.

As mentioned in the beginning of the article, arrays, classes, delegates are not the most suitable candidates for use when trying to get away from the GC.
Some classes can be replaced by structures. In D, structures are allocated on the stack and destroyed when leaving the visibility zone. If there are nowhere classes, then you can use it only in the scope:

 import std.typecons; ... auto cls = scoped!MyClass( param, of, my, _class ); ... 

The cls object will behave as an instance of the class MyClass, but will be destroyed when leaving the zone of visibility without GC participation. It is worth replacing that the scope keyword for creating class objects wants to be abstracted in favor of the library implementation, here is a discussion.

Ranges!

A separate coil and, as I understand it, the current trend in the development of a standard library is the transition to the concept of ranges. So now almost all functions from std.algorithm work. The ranges can be different: input, output, infinite, with a length, etc.
Their meaning is that these are objects (structures) containing certain methods, such as front, popFront, and so on. For more information on which structures can act as ranges in the standard library . Their advantages are pending computation and no memory allocation. A simple example:

 import std.stdio; import std.typetuple; import std.range; import std.array; template isIRWL(R) { enum isIRWL = isInputRange!R && hasLength!R; } template FloatHandler(R) { enum FloatHandler = is( ElementType!R == float ); } float avg(R1,R2)( R1 a, R2 b ) if( allSatisfy!(isIRWL,R1,R2) && allSatisfy!(FloatHandler,R1,R2) ) { auto c = chain( a, b ); //     float res = 0.0f; foreach( val; c ) res += val; // foreach ) return res / c.length; //   InputRange   } void main() { float[] a = [1,2,3]; float[] b = [4,5,6,7]; writeln( avg( a, b ) ); // 4 float[] d = chain( a, b ).array; //    writeln( d ); // [1,2,3,4,5,6,7] } 

The chain function returns an object of type Result (local for the function), which itself contains 2 references to the ranges that were specified at the input. When iterating through this object, the front and popFront methods are called using foreach, and this object calls the corresponding methods first on the first range, then on the second, when the first one becomes empty.
A good presentation on the topic of ranges was at DConf2015, by Jonathan M Davis.

If you really want classes

Yes, such that are constantly being created and deleted. In this case, you can slightly re-register the class and use the concept of FreeList

 class Foo { static Foo freelist; //   Foo next; //     static Foo allocate() { Foo f; if( freelist ) //       { f = freelist; //   freelist = f.next; } else f = new Foo(); //    return f; } static void deallocate(Foo f) //       { f.next = freelist; freelist = f; } ...     ... } ... Foo f = Foo.allocate(); ... Foo.deallocate(f); 

In this case, we minimize the allocation of memory for new objects, if these have already been created and are no longer needed. Completely from the collector, this does not enclose us, but if memory is not allocated, then the collector will not run the assembly.

Life without assembly (well, if only slightly)

I did not find a way to fully write on D without using a collector, but this is, in part, in my opinion, good. Manual memory management is fraught with errors, insecure, cumbersome, etc. (old and evil C ++). But if you strongly need, then you can.

For manual memory management, functions from libc malloc and free are used. To work with arrays, this is elementary:

 import core.stdc.stdlib; ... auto arr = (cast(float*)malloc(float.sizeof*count))[0..count]; ... free( arr.ptr ); ... 

To protect yourself from unwanted use of GC, you can use the @nogc attribute. The compiler will give an error when it detects the use of a collector inside blocks with such an attribute.

 void foo() {} void func(int[] arr) @nogc { auto a = new MyClass; //  arr ~= 42; //  foo(); // :  ,    @nogc } 

To maintain flexibility of use, it is not necessary to specify attributes to template functions. If the template function will be called from @nogc code, the compiler will try to make it also @nogc. For this, the condition must be preserved that only @nogc functions are used within this template function. This behavior of the compiler turns out to be useful in case of reuse of template code, when the template function will be needed when using the collector (it will be called from normal code and will use ordinary code inside itself). This also applies to other attributes (nothrow, pure, etc).

When compiling, you can display all the places in the program where the collector is used:

 dmd -vgc source.d ... 

The compiler will only indicate the locations of use, but will not generate an error.

It must be remembered that when creating threads through the standard library, the collector is also used. To create threads without a collector, you must use C-shnye functions, as is the case with malloc and free.

And lastly: creating classes without a collector

A small example with comments

 import std.stdio; import core.exception; import core.stdc.stdlib : malloc, free; import core.stdc.string : memcpy; import core.memory : GC; import std.traits; class A { int x; this( int X ) { x = X; } int foo() { return 2 * x; } } class B : A { int z = 2; this( int x ) { super(x); } override int foo() { return 3 * x * z; } } // std.conv.emplace    @nogc,   T classEmplace(T,Args...)( void[] chunk, auto ref Args args ) if( is(T == class) ) { enum size = __traits(classInstanceSize, T); //     //  ,    if( chunk.length < size ) return null; if( chunk.length % classInstanceAlignment!T != 0 ) return null; //  TypeInfo       init,     //          ,   memcpy( chunk.ptr, typeid(T).init.ptr, size ); auto res = cast(T)chunk.ptr; //   static if( is(typeof(res.__ctor(args))) ) res.__ctor(args); else static assert(args.length == 0 && !is(typeof(&T.__ctor)), "Don't know how to initialize an object of type " ~ T.stringof ~ " with arguments " ~ Args.stringof); return res; } auto heapAlloc(T,Args...)( Args args ) { enum size = __traits(classInstanceSize, T); auto mem = malloc(size)[0..size]; if( !mem ) onOutOfMemoryError(); //GC.addRange( mem.ptr, size ); //    return classEmplace!(T)( mem, args ); } auto heapFree(T)( T obj ) { destroy(obj); //GC.removeRange( cast(void*)obj ); //     free( cast(void*)obj ); } void main() { auto test = heapAlloc!B( 12 ); writeln( "test.foo(): ", test.foo() ); // 72 heapFree(test); } 

About the commented lines GC.addRange () and GC.removeRange (). If you are firmly determined that you will not use the collector, you can leave them zakomementirovannymi. If arrays, delegates, other classes, etc. that are to be cleaned with the help of GC are stored inside the class, then you need to add to the GC a range of memory that it will scan for garbage collection.

If the constructor is @nogc, then you can use heapAlloc in the @nogc functions, with heapFree everything is more complicated: destroy, in addition to calls to destructors (which you can simply implement with mixin), also performs some actions related to the class monitor (of course, if you want, you can and replace them with the @nogc option).

Conclusion

In the development of the language and the standard library, one can observe the tendency to abandon the "forcible" use of the garbage collector. At the moment, work on this is far from complete, but there are some advances.

In this regard, I found interesting reports from Walter Bright and Andrei Alexandrescu from the same DConf2015.

Ps. Why on Habré there is no syntax highlighting D?
Pps. Does anyone know if D conferences are scheduled in Russia?