D2 language and metaprogramming: everything is stranger and stranger
Not long ago, Monnoroch published some excellent introductory articles on the D2 language, and that was good. But, having read the last article devoted to metaprogramming, I wanted to do even better and open the topic in a bit more detail. The devil, as we know, is in the details - and it is precisely the attention to detail that makes the implementation of the meta-paradigm in D2 so convenient. If you have not read the article Monnoroch , I recommend first to get acquainted with it, because In this framework, I would not like to spend time on basic things.
So, if you already know some of the features of templates in D2, I would like to tell you more about what accompanies them - the tools of static introspection, the nuances of CTFE, and even such a forbidden but attractive thing as mixin.
The goal is more visual code examples with comments and fewer words.
is is the main tool for getting a boolean value at compile time. Its name was chosen, perhaps, not too successfully and can be misleading - in fact, it is responsible for all sorts of comparisons and type checking. Take a look at this little program:
The last example is especially interesting - is used simultaneously to
However, much more often you will see is in the context of is (typeof (...)) . Speaking of typeof :
')
typeof is very simple - takes an expression as an argument, returns its type if the expression is semantically correct. If not, it gives a compile-time error. But in combination with is, a slightly non-obvious effect is obtained - the expression inside typeof does not compile as such, since is only checks the validity of the type, and all error messages are suppressed. It turns out a stable idiom D2 to check the semantic correctness of an arbitrary expression at the compilation stage. Something similar was assumed constraints, and not included in the standard C ++ 11.
All that was said above is very, very nice, but if we are going to write good, beautiful libraries using template constraints, then it would be nice to have a more extensive toolkit for understanding what a certain type of data is. And it is in two whole copies:
Traits - a set of directives to the compiler for type integration. Here are some examples:
std.traits is a module of a standard library for similar purposes, written using the previously mentioned tools. Some utilitarian functions are very interesting, for example, mangledName:
D2 can create tuples of both values ​​and types, but we, of course, are primarily interested in the latter. The reason why I mention tuples in the context of instrospecting tools is a special compiler support that allows you to use tuples obtained using the same std.traits for further compilation.
This may not seem very impressive, but you should not forget that you can perform any actions on compile-time over a type tuple, creating different signatures for functions in different conditions, conveniently manipulating functions / templates with a variable number of arguments and indulging in other excesses.
CTFE (Compile-Time Field Evaluation) is a topic that is very easy to describe briefly and immense if you try to consider all the details. The fact that some functions / expressions can be calculated at compile time is not a new concept and is quite familiar to the same C ++ programmers. What is unusual is that there are so few restrictions on the performance of CTFE functions in D2. Here is a description from the official documentation:
Strictly speaking, everything. Loops, dynamic memory allocation, work with strings, associative arrays are all allowed during compilation. Here, for example, is the solution for a typical problem of prediscounting a table of square roots for a previously known range of integers:
Alas, some restrictions are really neat, especially the first. So, once, I was disappointed by the inability to convert double to string with the standard library — it turned out that it was linked to the standard C library somewhere in the depths. But as we work on improving phobos and improving CTFE support in the compiler, the situation improves. - not so long ago associative arrays were included in the forbidden list. And this is one of the areas of development of D2, on which work right now, if you look through commits on github.
But what about CTFE in the context of True Metaprogramming? I'll come back to them after I introduce you to mixin.
I'll start from afar, with a bit more approximate to the real conditions (but still contrived) example. Suppose we have a number of classes that, deep in their private depths, store an associative array and, in addition, remember the order of adding elements. There is a task - to add an interface to all these classes for iterating over the array elements in this order. What can be done? Multiple inheritance? Copy-paste? Interface inheritance and implementation class encapsulation? Ah, no idea about aesthetics.
To come to the aid of D2 come template mixin - a tool for "smart" copy-paste. The substitution patterns are declared through the mixin template keywords (suddenly!) And look similar to any regular D2 pattern. The interesting begins when a similar template is instantiated by the mixin TemplateName! (Parameters) directive - the resulting code is inserted into the context where mixin was applied.
Here is a somewhat longer piece of code, which is already much more similar to what can be written in a real project:
This approach may seem a bit monstrous, but think about it - this code declares a template ready for use through mixin in any of your classes where there are containers with suitable properties. At the same time, there is a check that the conditions required from the class are actually fulfilled - and all this is exclusively on compile-time! Any changes concerning the implementation of the forward range access to your classes will affect only this place, and the hierarchy of the main architecture will remain without unnecessary changes.
And finally, we got to one of the most powerful and ambiguous features of D2 metaprogramming. By entering this territory you lose most of the convenient error messages provided by the type system, and you are a step closer to the macro hell C - but your possibilities are almost endless.
string mixin works in something like eval from scripting languages, its syntax is very simple:
“Some string here” will substitute mixin and will be compiled. In this case, of course, it will not be, since hardly “some string here” will turn out to be a valid program code, but nothing prevents you from using a more useful string.
... for example the one that returns some CTFE-function. In fact, you have the opportunity to define DSL and, having written the corresponding CTFE translator in the code on D2, use this DSL as strings directly in the D2 modules. A good example is the standard library module std.range. In one of the modes, he is able to generate the code of regular expression parsers at the compilation stage, based on the regularity string specified in the usual form. It is for the authors of libraries, in my opinion, that this opportunity of D2 is of the greatest practical interest.
In order not to clutter up an already overgrown article, I will cite a link to the funny trick of a certain Daniel Keep as a vivid example: www.prowiki.org/wiki4d/wiki.cgi?DanielKeep/shfmt
Daniel yearned whenever possible in the style of PHP to do like this:
... and implemented it with the help of string mixin in D :) I warn you - listing by reference is dangerous for the eyes, and this, unfortunately, is for the time being the inevitable price for great features.
If you have ever used the algorithm module from the Phobos standard library, the convenient short lambda format for map, reduce & co also works in the end thanks to the magic string mixin.
Initially, I wanted to stop at every opportunity in more detail and analyze the various successful and undesirable methods of application. But as I wrote, it turned out that the size of the article was growing at an indecent rate for something so richly flavored with code examples. In the end, I tried to just give the broadest possible picture of the whole set of tools that D2 provides for metaprogramming. If you at least once thought, “Wow, and with the help of this you can do <feature of my dream>”, then the goal was accomplished.
Often, newbies in D2 are simply lost from the abundance of opportunities to do anything at the compilation stage and these capabilities require considerable discipline in order to be useful and not turn your code into a mess. So try to observe moderation in architectural temptations. Enjoy your meal :)
On typos and stylistic blunders, please report in private.
All examples were tested on dmd version 2.057
So, if you already know some of the features of templates in D2, I would like to tell you more about what accompanies them - the tools of static introspection, the nuances of CTFE, and even such a forbidden but attractive thing as mixin.
The goal is more visual code examples with comments and fewer words.
Introspection tools
is expression
is is the main tool for getting a boolean value at compile time. Its name was chosen, perhaps, not too successfully and can be misleading - in fact, it is responsible for all sorts of comparisons and type checking. Take a look at this little program:
void main() { // assert( !is(garbage[id]) ); assert( !is(x) ); alias int x; assert( is(x) ); // assert( is(x : long) ); assert( !is( x : string ) ); // assert( is( x == int) ); assert( !is( x == long) ); // + pattern matching + alias declaration alias long[char[]] AA; static if( is( AA T : T[U], U : const char[]) ) alias T key; else alias void key; assert( is(key == long) ); } The last example is especially interesting - is used simultaneously to
- check the type reducibility to an associative container storing const char []
- on pattern to isolate the type of the key from the container type
- create alias for the nested scope
However, much more often you will see is in the context of is (typeof (...)) . Speaking of typeof :
')
typeof expression
typeof is very simple - takes an expression as an argument, returns its type if the expression is semantically correct. If not, it gives a compile-time error. But in combination with is, a slightly non-obvious effect is obtained - the expression inside typeof does not compile as such, since is only checks the validity of the type, and all error messages are suppressed. It turns out a stable idiom D2 to check the semantic correctness of an arbitrary expression at the compilation stage. Something similar was assumed constraints, and not included in the standard C ++ 11.
int func(); void main() { // , alias typeof(func()) ret_type; assert( is( ret_type == int ) ); double func_prim() { // , assert( is( typeof(return) == double ) ); return 0; } // is constraint " <" void template_func(T)( T t ) if ( is(typeof( T.init < T.init )) ) { } template_func(20); // struct S {} // template_func(S.init); // error } traits / std.traits
All that was said above is very, very nice, but if we are going to write good, beautiful libraries using template constraints, then it would be nice to have a more extensive toolkit for understanding what a certain type of data is. And it is in two whole copies:
Traits - a set of directives to the compiler for type integration. Here are some examples:
abstract class C { } class B { int a; } void main() { assert( __traits(isAbstractClass, C ) ); assert(! __traits(isAbstractClass, B ) ); assert(! __traits(isAbstractClass, int ) ); assert( __traits(hasMember, B, "a") ); assert( !__traits(hasMember, C, "a") ); // offtopic: [ ] - allMembers , auto a = [ __traits(allMembers, B) ]; assert( a == ["a", "toString", "toHash", "opCmp", "opEquals", "Monitor", "factory"] ); } std.traits is a module of a standard library for similar purposes, written using the previously mentioned tools. Some utilitarian functions are very interesting, for example, mangledName:
import std.traits; import std.stdio; void func1(); extern ( C ) void func2(); void main() { // _D9stdtraits5func1FZv writeln( mangledName!(func1) ); // func2 writeln( mangledName!(func2) ); } type tuples
D2 can create tuples of both values ​​and types, but we, of course, are primarily interested in the latter. The reason why I mention tuples in the context of instrospecting tools is a special compiler support that allows you to use tuples obtained using the same std.traits for further compilation.
import std.typetuple; import std.traits; import std.stdio; // void func1( int, double, string ) { } // " " alias TypeTuple!( int, double, string ) tp_same; // , , func1! void func2( tp_same tp ) { assert( is( typeof(tp[0]) == int ) ); assert( is( typeof(tp[1]) == double ) ); assert( is( typeof(tp[2]) == string ) ); } // , ? alias ParameterTypeTuple!(func1) tp_func1_copy; // ... . void func3( tp_func1_copy tp ) { foreach( i; tp) { writeln(typeid( typeof(i) ), " ", i); } // : // int 2 // double 2 // immutable(char)[] 2 } void main() { // func1( 2, 2.0, "2"); func2( 2, 2.0, "2"); func3( 2, 2.0, "2"); // ! assert( is( typeof(func1) == typeof(func2) ) ); assert( is( typeof(func2) == typeof(func3) ) ); } This may not seem very impressive, but you should not forget that you can perform any actions on compile-time over a type tuple, creating different signatures for functions in different conditions, conveniently manipulating functions / templates with a variable number of arguments and indulging in other excesses.
CTFE Unleashed
CTFE (Compile-Time Field Evaluation) is a topic that is very easy to describe briefly and immense if you try to consider all the details. The fact that some functions / expressions can be calculated at compile time is not a new concept and is quite familiar to the same C ++ programmers. What is unusual is that there are so few restrictions on the performance of CTFE functions in D2. Here is a description from the official documentation:
- Must have source code fully accessible to the compiler (no extern)
- Should not refer to global or static variables.
- Should not use inline asm
- Non-portable type conversions (for example, int [] to float []), including conversions dependent on byte order, are prohibited. Conversions between signed and unsigned numbers are allowed. Conversion of pointers to data and vice versa is prohibited.
- Actions with pointers are allowed only for pointers to array elements.
- assert and others like him do not create an exception, but simply stop interpreting
Strictly speaking, everything. Loops, dynamic memory allocation, work with strings, associative arrays are all allowed during compilation. Here, for example, is the solution for a typical problem of prediscounting a table of square roots for a previously known range of integers:
import std.math; enum PrecomputedSqrtIndex { Min = 1, Max = 10 } // pure, nothrow in , , , :) pure nothrow double[int] precompute_sqrt_lookup( in int from, in int to ) { double[int] result; foreach( i; from..to+1 ) result[i] = sqrt(i); return result; } enum sqrt_lookup = precompute_sqrt_lookup( PrecomputedSqrtIndex.Min, PrecomputedSqrtIndex.Max ); void main() { } Alas, some restrictions are really neat, especially the first. So, once, I was disappointed by the inability to convert double to string with the standard library — it turned out that it was linked to the standard C library somewhere in the depths. But as we work on improving phobos and improving CTFE support in the compiler, the situation improves. - not so long ago associative arrays were included in the forbidden list. And this is one of the areas of development of D2, on which work right now, if you look through commits on github.
But what about CTFE in the context of True Metaprogramming? I'll come back to them after I introduce you to mixin.
Mixin
Template mixin
I'll start from afar, with a bit more approximate to the real conditions (but still contrived) example. Suppose we have a number of classes that, deep in their private depths, store an associative array and, in addition, remember the order of adding elements. There is a task - to add an interface to all these classes for iterating over the array elements in this order. What can be done? Multiple inheritance? Copy-paste? Interface inheritance and implementation class encapsulation? Ah, no idea about aesthetics.
To come to the aid of D2 come template mixin - a tool for "smart" copy-paste. The substitution patterns are declared through the mixin template keywords (suddenly!) And look similar to any regular D2 pattern. The interesting begins when a similar template is instantiated by the mixin TemplateName! (Parameters) directive - the resulting code is inserted into the context where mixin was applied.
Here is a somewhat longer piece of code, which is already much more similar to what can be written in a real project:
import std.range : isForwardRange; import std.stdio : writeln; import std.typecons : Tuple; // "" mixin, mixin template AddForwardRangeMethods( alias data_container, alias order_container ) // , data_container if ( is(typeof( data_container.length ) : int ) && // ... order_container data_container is(typeof( data_container[ order_container[0] ] )) // - , - :) ) { // .. , // . private struct Result { // private int last_index; // - , // typeof(data_container) ref_data; typeof(order_container) ref_order; alias typeof(order_container[0]) Key_t; static if ( is( typeof(data_container) T : T[U], U : Key_t ) ) { alias T Value_t; } else static assert(0, "Wrong data_container / order_container data_container types" ); this(typeof(data_container) data, typeof(order_container) order) { last_index = 0; ref_data = data; ref_order = order; } // , forward range bool empty() { return last_index >= ref_data.length; } Tuple!(Key_t, Value_t) popFront() { // scope , , // scope (exit) last_index++; return front(); } Tuple!(Key_t, Value_t) front() { return typeof(return)( ref_order[last_index], ref_data[ref_order[last_index]] ); } Result save() { return Result( ref_data, ref_order ); } } public Result fwdRange() { return Result( data_container, order_container ); } } // , mixin class A { private int[int] a; private int[] order; this() { a = [ 2 : 4, 4 : 16, 3 : 9 ]; order = [ 2, 4, 3 ]; } // ! mixin AddForwardRangeMethods!(a, order); } void main() { // - forward range, duck typing assert(isForwardRange!(A.Result)); auto a = new A(); auto r = a.fwdRange; foreach( i; r ) writeln( i ); // Tuple!(int,int)(2, 4) // Tuple!(int,int)(4, 16) // Tuple!(int,int)(3, 9) } This approach may seem a bit monstrous, but think about it - this code declares a template ready for use through mixin in any of your classes where there are containers with suitable properties. At the same time, there is a check that the conditions required from the class are actually fulfilled - and all this is exclusively on compile-time! Any changes concerning the implementation of the forward range access to your classes will affect only this place, and the hierarchy of the main architecture will remain without unnecessary changes.
String mixin
And finally, we got to one of the most powerful and ambiguous features of D2 metaprogramming. By entering this territory you lose most of the convenient error messages provided by the type system, and you are a step closer to the macro hell C - but your possibilities are almost endless.
string mixin works in something like eval from scripting languages, its syntax is very simple:
mixin("some string here") “Some string here” will substitute mixin and will be compiled. In this case, of course, it will not be, since hardly “some string here” will turn out to be a valid program code, but nothing prevents you from using a more useful string.
... for example the one that returns some CTFE-function. In fact, you have the opportunity to define DSL and, having written the corresponding CTFE translator in the code on D2, use this DSL as strings directly in the D2 modules. A good example is the standard library module std.range. In one of the modes, he is able to generate the code of regular expression parsers at the compilation stage, based on the regularity string specified in the usual form. It is for the authors of libraries, in my opinion, that this opportunity of D2 is of the greatest practical interest.
In order not to clutter up an already overgrown article, I will cite a link to the funny trick of a certain Daniel Keep as a vivid example: www.prowiki.org/wiki4d/wiki.cgi?DanielKeep/shfmt
Daniel yearned whenever possible in the style of PHP to do like this:
int a = 3; writeln("a = $a"); ... and implemented it with the help of string mixin in D :) I warn you - listing by reference is dangerous for the eyes, and this, unfortunately, is for the time being the inevitable price for great features.
If you have ever used the algorithm module from the Phobos standard library, the convenient short lambda format for map, reduce & co also works in the end thanks to the magic string mixin.
Instead of an epilogue
Initially, I wanted to stop at every opportunity in more detail and analyze the various successful and undesirable methods of application. But as I wrote, it turned out that the size of the article was growing at an indecent rate for something so richly flavored with code examples. In the end, I tried to just give the broadest possible picture of the whole set of tools that D2 provides for metaprogramming. If you at least once thought, “Wow, and with the help of this you can do <feature of my dream>”, then the goal was accomplished.
Often, newbies in D2 are simply lost from the abundance of opportunities to do anything at the compilation stage and these capabilities require considerable discipline in order to be useful and not turn your code into a mess. So try to observe moderation in architectural temptations. Enjoy your meal :)
On typos and stylistic blunders, please report in private.
All examples were tested on dmd version 2.057
Source: https://habr.com/ru/post/135670/
All Articles