[CppCon 2017] Herb Sutter: Metaprogramming and Code Generation in C ++

I continue the series of Fil publications on CppCon 2017. The report presents early developments on adding reflection and code generation in C ++, as well as on metaclasses that will allow to generate parts of C ++ classes. These innovations will get to the standard not earlier than in C ++ 23.

Preface: Comparison Operators

Suppose we want to write a class whose objects can be compared, for example, a non-case-sensitive string class - CIString (Case-Insensitive String). To do this, at a minimum, we need to write 6 comparison operators: == , < != , > , >= , <= . And all the code, except for the first two, will be absolutely standard. If you need to be able to compare our string with const char* without copying, then add 12 more statements.

The problem of duplication of code in comparison operators was so acute that a sentence was written to the C ++ standard, which adds the so-called "spaceship operator" to the language:

 class CIString { string s; public: // ...   ... std::weak_ordering operator<=>(const CIString& b) const { ... } std::weak_ordering operator<=>(const char* b) const { ... } }; 

We write one function <=> , where we return std::weak_ordering::less , ::equivalent or ::greater , and the compiler generates implementations of all comparison functions. 5 main types of comparison are supported, including std::strong_ordering and generation of functions only == / != .

The “spaceship operator” proposal has value by itself, but, as will be shown later, using metaclasses you can implement it, and many other code generation scenarios, moreover, in pure C ++, without the need to embed them in a compiler or a language standard.

Reflection

For any type, function, or other entity, $T is the constexpr expression that returns the value of the meta type. You can invoke methods on it to get various meta-information related to T

Note: this is a very early version of the standard clause, and $expr can be replaced with reflect(T) , or something like that.

For example, you can get the names of variables of an arbitrary enum using the variables() method:

 template<typename E> void print_strings() { for (auto o : $E.variables()) cout << o.name() << endl; } enum class state { started = 1, waiting, stopped }; print_strings<state>(); //=> started waiting stopped 

Since templates are instantiated only when used, variable names and other meta-information will be in a binary only if we use it in the program.

Coding generation (injection)

constexpr block can be anywhere in a program: in a function, in a class, etc. Here's what it looks like:

 //   constexpr { // ,    } //   

The construction -> { ... } used inside the constexpr block to insert ordinary code in its place:

 constexpr { // ... -> { //   } // ... } 

Example: printing the name of a specific enumeration variable

 template<Enum E> //       ? auto to_string(E value) { switch (value) { constexpr { for (auto o : $E.variables()) -> { case o.value(): return o.name(); } } } } enum class state { started = 1, waiting, stopped }; cout << to_string(state::stopped); //=> stopped 

After preprocessing, the compiler will receive a code equivalent to a hand-painted switch .

Good question - how to debug such code? Without special support from the debugger, which allows you to look at the generated code, it will be difficult to debug such programs. Nevertheless, the debugging function of the code that is executed at the compilation stage would be useful in any case, just like the debugging of the code generated by the compiler (many would like to see the default constructors and copy statements when debugging).

Metaclasses

The definition of a metaclass (not to be confused with a metatype) begins with the keyword $class , inside it is possible to define the generated functions and conditions imposed on the class. When defining a regular class, instead of class you can specify the name of the metaclass to convert the class code with it. It looks like this:

 $class interface { //      //   ,     //   }; interface Shape { int area() const; void scale_by(double factor); }; 

This definition of Shape equivalent to the following:

 class Shape { public: virtual int area() const = 0; virtual void scale_by(double factor) = 0; virtual ~Shape() noexcept { }; }; 

In the definition of interface you can use constexpr -blocks and reflection, as well as receive and modify "your" metatype:

 $class interface { ~interface() noexcept { } constexpr { compiler.require($interface.variables().empty(), "interfaces may not contain data members"); for (auto f : $interface.functions()) { compiler.require(!f.is_copy() && !f.is_move(), "interfaces may not copy or move; consider a virtual clone()"); if (!f.has_access()) f.make_public(); compiler.require(f.is_public(), "interface functions must be public"); f.make_pure_virtual(); } } }; 

It is impossible to change an arbitrary class after the declaration, that is, the ODR remains in effect.

Some thoughts on metaclasses

Metaclass can be understood as a subtype of a class. C ++ classes have so many properties and types of redefinable behavior that it is often convenient to limit their capabilities in order to reduce confusion.

It is important that metaclasses are defined in C ++ code, and not in the compiler. The C ++ Standardization Committee is slow, and it would take years before the approval of specific metaclasses in the C ++ standard could be achieved.

Metaclass examples

value

 value Point { int x = 0; int y = 0; Point(int, int); } 

The fact that Point is a value means that it has a default constructor, copy / move and compare operations; guaranteed no virtual functions.

On the example of value , the composition ("inheritance") of the metaclasses was shown:

 $class basic_value { ... } $class ordered { ... } $class value : basic_value, ordered { } 

It is clear that no operator<=> is needed to use ordered and generate comparison operators based on class fields.

literal_value

It is planned that with the help of metaclasses it will be possible to generate auxiliary classes / functions, like swap and the specialization std::hash . Then, using the hypothetical metaclass literal_value you can write a simple and beautiful definition of std::pair , which this class deserves:

 template<class T1, class T2> literal_value pair { T1 first; T2 second; }; 

enum_class

The fact that in C ++ there is no division into classes, interfaces, etc., as in Java, this is an advantage, so now all user types can be defined using the class keyword ... In addition to enumerations! Fortunately, metaclasses are powerful enough to define the enum class . As a result, you can write:

 enum_class state { auto started = 1, waiting, stopped; }; state s = state::started; while (s != state::waiting) { ... } 

These enum_class can be even better than the existing ones, due to additional automatically generated functions and informative error messages.

flag_enum

flag_enum values ​​can be used as sets of flags, with operations | , & , ^ :

 flag_enum openmode { auto in, out, binary, ate, app, trunc; // 1, 2, 4, 8, 16, 32 }; openmode mode = openmode::in | openmode::out; assert(mode != openmode::none); assert(mode & openmode::out); 

property

 classx MyClass { property<int> value { }; // get/set   // ... }; 

The property<> token is used by the classx metaclass to generate the property:

 class MyClass { int value; public: void set_value(int v) { value = v; } int get_value() const { return value; } }; 

Also, when defining a property you can set your own get / set functions.

QClass

Qt uses the moc utility, which, by class definition, generates data for reflection on the execution time, properties, and so on. The metaclass QClass could handle all this, allowing you to build using Qt only with the help of the C ++ compiler:

 QClass MyClass { property<int> value { }; signal mySignal(); slot mySlot(); }; 

In addition to Qt moc, metaclasses will get rid of the clutter of macros and proprietary language extensions, like WinRT and C ++ / CX.

podio

The particle physics library podio uses special types of YAML files to define data structures:

 ExampleHit: Description: "Example Hit" Author: "B. Heigner" Members: - double x - double y - double z - double energy 

At the same time, 5 classes are generated at once: X , XCollection , XConst , XData , XObj . It is argued that with the help of metaclass it will turn out to do the same.

CRTP

CRTP is a technique in which the derived class is passed as a generic parameter to the base class:

 template<typename Derived> class EqualityComparable { public: friend bool operator !=(const Derived& a, const Derived& b) { return !(a == b); } }; class X : public EqualityComparable<X> { public: friend bool operator==(const X& a, const X& b) { ... } }; 

In fact, we already use some semblance of metaclasses in our projects. Only here in a fairly complex code with error reporting patterns leave much to be desired. With the advent of metaclass, CRTP is gone.

Demonstration

Herb demonstrated a Clang modification that supports everything that was shown above, except for the generation of several classes / functions from one class. The code obtained after applying the metaclass can be viewed and debugged.

Questions

What if N companies define an interface ?

With classes of strings this is already happening, but this does not mean that ordinary classes should be abandoned. Basic metaclasses, like interface , can be added to std .

What is the benefit of being able to define an interface in C ++ code if you still want to standardize some popular metaclasses?

In order to make an interface standard now, it will take an enormous amount of time, and the problem associated with its absence in the language is not so great that anyone becomes involved in it. If the interface can be implemented in 10 C ++ lines, it will fly through the standardization committee "with a whistle".

You said that metaclasses that define C ++ entities in C ++ itself are better than defining them in a standard. You also said that the C ++ standard language is very complex. Isn't it possible that C ++ will become more complicated up to the level of the C ++ standard language?

We are not trying to define meta-C ++. Metaclasses perform a specific function of generating class elements, and this can be implemented without excessively complicating the language.

Metaclasses modify their class "in place". Wouldn't it be better if the metaclass had an "input class" and it would generate code in the "output class"?

Yes, we are going to change the syntax so that everything works that way. The output class will be called $prototype . At first it will be empty, and the metaclass will insert the strings one by one into its definition.

You said that several classes could be generated from one metaclass. Does this mean that all classes will be nested in the main one, or will they be separate classes?

These will be separate classes. The current version of our compiler does not support this function, but we are working on it. Without generating auxiliary external functions, for example, it is not possible to define a literal_value .

If the literal_value is in the standard, will the std :: pair class really be changed as shown?

If it is possible to determine literal_value so that it exactly matches the current behavior of std :: pair, then yes.

Will it be possible to completely get rid of all techniques with templates when generating C ++ code?

Templates will be used with metaclasses. See an example with property<> .

Does it not turn out that each company will have its own "mandatory" basic metaclass from which all classes must inherit, which will lead to the fragmentation of the language?

This is already happening, and it is called "corporate code style". The difference is that it is described in words and, possibly, verified by various tools. Now it will be observed more strictly, predictably and correctly. When we tested the usability of metaclasses on real projects, we found that they found different uses for each project. For example, one of the projects included a robot control library that required all your classes to follow a specific pattern. Now they monitor the observance of these rules manually. From time to time they are forgotten, hard-to-find errors, etc. appear. With just one metaclass of 20-30 lines of code, they could fix this problem once and for all.

CRTP is a great metaclass-like technique that works now. If it is supplemented with reflection and constexpr -blocks, then they will have as many opportunities as the metaclasses. Why then are they needed?

Metaclasses have more options. CRTP is a great hack, but the templates were not originally designed for this, it is not a sufficiently accurate and functional tool. For example, CRTP allows you to mistakenly override the generated methods in a derived class. Using CRTP, it will not be possible to define helper global functions and classes.