Reflection and code generation in C ++
The C ++ language is still one of the most sought-after and flexible programming languages. But sometimes the capacity of the language is not enough, despite the fact that the standard is developed and expanded. I encountered such a problem in the process of developing a 2D game engine. I faced the need to solve several non-trivial tasks, such as serialization, animation, and a link with the editor. For this great reflection. Unfortunately, the ready solutions from the network did not suit me, so I had to design my bike.
The following describes the implementation details and demo project. Who cares - welcome under cat.
There is an opinion that correctly set conditions of a problem practically solve this problem. That is why, before the start of implementation, the necessary requirements and limitations were collected:
Following the requirement of simplicity, a fairly simple syntax for defining an object that supports reflection is developed: it is necessary that the class be inherited from IObject , contain the macro IOBJECT (* class name *) in the body ; and the members of the class were given the necessary attributes through the comments.
')
An example of a class that supports reflection:
Examples of using reflection:
As you probably already guessed, everything works simply: the user declares classes that support reflection, and a separate utility generates code that initializes the types in the system. In this article I will describe the principle of operation, with minimal inclusion of the source code, since its presence will inflate the article. But a demo project with all sources is attached to the article, which can be viewed and tested.
So, more about the structure of reflection.
Each class corresponds to a description of the type, which contains the name of the class, information about inheritance, the members and methods of the class. Using a type you can create an instance of a class. Also in the type of a class there is a rather interesting functional: you can get a pointer to a member of a class along its path and vice versa, get a path to a member of a class using its pointer. To understand what the last two functions are doing, just look at the usage example:
Why do you need it? The closest example is animation. Suppose there is an animation that animates the value parameter from the example. But how to save such an animation to a file? Here we need these functions. We save the animation keys with the path to the “aObject / bObject / value” parameter that is being animated, and when loading from this file, we find the variable we need. This approach allows you to animate absolutely any members of any objects and save / load to a file.
However, there is a small flaw that needs to be considered. The search for a pointer to a class member along the path is fast and linear, but the reverse process is completely non-linear and can take a long time. The algorithm has to “comb” all members of a class, their members, and so on, until it encounters the pointer we need.
Each class member is defined using the FieldInfo type, which contains the member type, its name, and a list of attributes. Also, the class member type allows you to get the value of a class member for a specific instance and vice versa, assign a value. Type, name and attributes are filled with the generated code. The assignment and retrieval of a value works through address arithmetic. At the type initialization stage, the offset in bytes is calculated relative to the beginning of the memory for the object, then, when assigning or receiving a value, this offset is added to the address of the object.
Each function of the class corresponds to an object with the FunctionInfo interface, which stores the type of the return value, the name of the function and the list of parameters. Why interface and how to call a function from this interface? To call a function, we need a pointer to it. Moreover, the pointer to the class function should be of the following form:
For storing such pointers, classes are defined; template classes are defined:
Pattern magic is scary, but it works pretty simple. At the type initialization stage, for each function, an object of class SpecFunctionInfo or SpecConstFunctionInfo is created depending on the constancy of the function and placed in the type of the Type object. Then, when we need to call a function from the FunctionInfo interface, we statically convert the interface to ISpecFunctionInfo and call its virtual function Invoke , the redefined version of which performs the function at the stored address.
The attribute function is the addition of meta information for class members. Using attributes, you can specify the members to be serialized, the type of their serialization, display in the editor, animation parameters, and so on. Attributes are inherited from the IAttribute interface and are added to class members during the type initialization stage. The user only needs to identify the necessary attributes through comments to the members of the class.
So, we have all structure of types, it is necessary to store all this somewhere. The singleton class Reflection copes with this simple task. With it, you can get a type by name, create an instance of a type, or get a list of all registered types.
Why code generation? There are solutions to reflection using tricky macros, prescribing the serialization function, and the like. But these approaches have one common drawback - in fact, you have to manually describe the types. In reflection, one cannot get rid of this, so the easiest way is to entrust this routine work to a separate subroutine that runs before compilation and generates a separate source file with type initialization.
In a good way, of course, the compiler itself should do this, because it has all the information about classes, members, and methods. But the solution should be independent of the compiler, in the end it is foolish to impose your specific compiler on the developer, especially for 2D games with a bunch of target platforms.
That is why code generation is a separate utility. Unfortunately, the utility in my project cannot boast of elegance and one hundred percent stability, but on my rather rather big project it works perfectly. In the future, it will most likely be rewritten and supplemented, since it has at least one significant disadvantage - it is written in C #, which requires the Windows platform, or the availability of Mono. This is the path of least resistance, and I chose it because at the stage of the demo version I need as much functionality as possible in the project. Nevertheless, in this article I will describe the stages of her work and what difficulties I encountered.
The work of the utility is divided into two stages:
Here, my approach is different from the work of compilers.
At the output, we get all the namespaces, classes and class descriptions.
This stage includes the generation of the following parts:
In the end, we get the source file with a function that initializes all types of applications. The user only needs to call it when the application starts.
github.com
PS: Comments and suggestions for improvement are welcome!
The following describes the implementation details and demo project. Who cares - welcome under cat.
Requirements
There is an opinion that correctly set conditions of a problem practically solve this problem. That is why, before the start of implementation, the necessary requirements and limitations were collected:
- Minimum functionality. The user must have access to the types of objects and their inheritance. An object type must provide information about members, its attributes, and class methods. Also, a functional is needed to change the values ​​of class members and method calls through reflection.
- Easy to use. The use of reflection should not burden the syntax with any difficulties.
- Speed ​​performance Working with reflection should not significantly affect the performance of the application.
Syntax
Following the requirement of simplicity, a fairly simple syntax for defining an object that supports reflection is developed: it is necessary that the class be inherited from IObject , contain the macro IOBJECT (* class name *) in the body ; and the members of the class were given the necessary attributes through the comments.
')
An example of a class that supports reflection:
class MyClass: public IObject { IOBJECT(MyClass); float mSomeFloat; // @SERIALIZABLE int mSomeInteger; // @SOME_ATTRIBUTE MyClass* mSomeObject; void Func(int abc) { ... } }; Examples of using reflection:
// MyClass sample; // Type& myClassType = MyClass::type; // Type& myClassTypeToo = sampe.GetType(); // MyClass IObject* newSample = myClassType.CreateSample(); // FieldInfo* fieldInfo = myClassType.Field("mSomeFloat"); // float fieldValue = fieldInfo->GetValue<float>(&sampe); // fieldInfo->SetValue<float>(&sample, 36.6f); // FunctionInfo* funcInfo = myClassType.GetFunction("Func"); // funcInfo->Invoke<void, int>(&sample, 123); Implementation
As you probably already guessed, everything works simply: the user declares classes that support reflection, and a separate utility generates code that initializes the types in the system. In this article I will describe the principle of operation, with minimal inclusion of the source code, since its presence will inflate the article. But a demo project with all sources is attached to the article, which can be viewed and tested.
So, more about the structure of reflection.
Object type
Each class corresponds to a description of the type, which contains the name of the class, information about inheritance, the members and methods of the class. Using a type you can create an instance of a class. Also in the type of a class there is a rather interesting functional: you can get a pointer to a member of a class along its path and vice versa, get a path to a member of a class using its pointer. To understand what the last two functions are doing, just look at the usage example:
class A; class B; class MyClass: public IObject { IOBJECT(MyClass); A* aObject = new A(); }; class A: public IObject { IOBJECT(A); B* bObject = new B(); }; class B: public IObject { IOBJECT(A); int value = 777; }; MyClass sample; // : string path = sample.GetType().GetFieldPath(&sample->aObject->bObject->value); // : int* ptr = sample.GetType().GetFieldPtr<int>("aObject/bObject/value"); Why do you need it? The closest example is animation. Suppose there is an animation that animates the value parameter from the example. But how to save such an animation to a file? Here we need these functions. We save the animation keys with the path to the “aObject / bObject / value” parameter that is being animated, and when loading from this file, we find the variable we need. This approach allows you to animate absolutely any members of any objects and save / load to a file.
However, there is a small flaw that needs to be considered. The search for a pointer to a class member along the path is fast and linear, but the reverse process is completely non-linear and can take a long time. The algorithm has to “comb” all members of a class, their members, and so on, until it encounters the pointer we need.
Class members
Each class member is defined using the FieldInfo type, which contains the member type, its name, and a list of attributes. Also, the class member type allows you to get the value of a class member for a specific instance and vice versa, assign a value. Type, name and attributes are filled with the generated code. The assignment and retrieval of a value works through address arithmetic. At the type initialization stage, the offset in bytes is calculated relative to the beginning of the memory for the object, then, when assigning or receiving a value, this offset is added to the address of the object.
Class functions
Each function of the class corresponds to an object with the FunctionInfo interface, which stores the type of the return value, the name of the function and the list of parameters. Why interface and how to call a function from this interface? To call a function, we need a pointer to it. Moreover, the pointer to the class function should be of the following form:
_res_type(_class_type::*mFunctionPtr)(_args ... args); For storing such pointers, classes are defined; template classes are defined:
template<typename _res_type, typename ... _args> class ISpecFunctionInfo: public FunctionInfo { public: virtual _res_type Invoke(void* object, _args ... args) const = 0; }; template<typename _class_type, typename _res_type, typename ... _args> class SpecFunctionInfo: public ISpecFunctionInfo<_res_type, _args ...> { ... }; template<typename _class_type, typename _res_type, typename ... _args> class SpecConstFunctionInfo: public ISpecFunctionInfo<_res_type, _args ...> { ... }; Pattern magic is scary, but it works pretty simple. At the type initialization stage, for each function, an object of class SpecFunctionInfo or SpecConstFunctionInfo is created depending on the constancy of the function and placed in the type of the Type object. Then, when we need to call a function from the FunctionInfo interface, we statically convert the interface to ISpecFunctionInfo and call its virtual function Invoke , the redefined version of which performs the function at the stored address.
Attributes
The attribute function is the addition of meta information for class members. Using attributes, you can specify the members to be serialized, the type of their serialization, display in the editor, animation parameters, and so on. Attributes are inherited from the IAttribute interface and are added to class members during the type initialization stage. The user only needs to identify the necessary attributes through comments to the members of the class.
Reflection module
So, we have all structure of types, it is necessary to store all this somewhere. The singleton class Reflection copes with this simple task. With it, you can get a type by name, create an instance of a type, or get a list of all registered types.
Code Generation
Why code generation? There are solutions to reflection using tricky macros, prescribing the serialization function, and the like. But these approaches have one common drawback - in fact, you have to manually describe the types. In reflection, one cannot get rid of this, so the easiest way is to entrust this routine work to a separate subroutine that runs before compilation and generates a separate source file with type initialization.
In a good way, of course, the compiler itself should do this, because it has all the information about classes, members, and methods. But the solution should be independent of the compiler, in the end it is foolish to impose your specific compiler on the developer, especially for 2D games with a bunch of target platforms.
That is why code generation is a separate utility. Unfortunately, the utility in my project cannot boast of elegance and one hundred percent stability, but on my rather rather big project it works perfectly. In the future, it will most likely be rewritten and supplemented, since it has at least one significant disadvantage - it is written in C #, which requires the Windows platform, or the availability of Mono. This is the path of least resistance, and I chose it because at the stage of the demo version I need as much functionality as possible in the project. Nevertheless, in this article I will describe the stages of her work and what difficulties I encountered.
The work of the utility is divided into two stages:
- Parsing project sources
- Generating the final source with type initialization
Parsing project sources
Here, my approach is different from the work of compilers.
- Each header file is parsed separately, without processing the inclusion of other header files. Simply put, all #include is ignored.
- It builds a list of header files that have a list of namespaces, a list of classes, and so on.
- The namespaces of all header files are merged and inheritance is bound.
- Specialization of template classes: if somewhere in the code there is a template specialization, a class with such a specialization is created and registered.
At the output, we get all the namespaces, classes and class descriptions.
Generating the final source with type initialization
This stage includes the generation of the following parts:
- Declaring static class types
- Declaring type initialization functions: registering members, adding attributes based on comments, registering functions and their parameters
- All types initialization function and inheritance binding
In the end, we get the source file with a function that initializes all types of applications. The user only needs to call it when the application starts.
Shortcomings and future improvements
- Of course, the code generation utility is far from perfect and will most likely be rewritten in C ++, which will give a reduction in runtime and multiplatform.
- Although the utility runs for a short time (about seven seconds on my laptop), it is still noticeable.
- You still need to manually write the IOBJECT () macro in the class body. It would be nice to have it added automatically.
- The static member of the type class is public and can be changed by carelessness. You can follow this, but no one is immune from mistakes.
- Attributing attributes through comments is fraught with typos and is not as convenient as in languages ​​where reflection is supported. So far, I have not found an acceptable solution.
Link to the demo project
github.com
PS: Comments and suggestions for improvement are welcome!
Source: https://habr.com/ru/post/277329/
All Articles