Combining C ++ and Python. Subtleties Boost.Python. Part two
This article is a continuation of the first part .
We continue to torment Boost.Python. This time it is the turn of the class, which can neither be created nor copied.
Let's wrap an almost ordinary sishny structure with an unusual designer.
And we’ll work with returning a reference to a C ++ object field so that the Python garbage collector doesn’t delete it inadvertently. Well and on the contrary, we will make an alternative option so that Python can clear up the garbage after removing what has been given to it for storage.
Go…
For our purposes, it would be enough for us to supplement the draft example remaining from the previous part.
Let's add to it a couple more files to work with the singleton class:
single.h
single.cpp
And we will render the declarations of auxiliary functions for wrapping in Python into a separate file:
wrap.h
From the previous project there should have been a file that we will actively change:
wrap.cpp
And the wonderful files with the miracle class, which helped us so much in the first part, they will remain as they are:
some.h
some.cpp
Let's start with the fact that we get in single.h a small C-style structure, just with a description of the fields.
Let's for interest, this will be not just a structure, but a kind of cryptic configuration description type:
It is easy to make a wrapper for such a structure, you just need to specifically describe the constructor with parameters using the constructor template boost :: python :: init <...> (...) of the wrapper template parameter boost :: python :: class_:
As you can see, here you don’t even have to use return_value_policy <copy_const_reference> for a string field. Just because the field here is taken essentially by value, and therefore it will automatically be converted to the standard string of the Python language.
The make_setter functions still do a very useful job of checking the type of the incoming value, for example, try setting the coef field in Python to a string type or setting max_size with a value of type float, get an exception.
The fields of the config structure are essentially converted into properties of the object of the full-fledged Python class Config. Well, almost complete ... Let's, by analogy with the Some class from the last chapter, add the __str__ and __repr__ methods to the wrapper, and at the same time add the as_dict property to convert the structure fields to standard dict python and back.
Declaring new functions, as well as old ones, will be transferred to our new wrap.h file:
In the wrap.cpp file, nothing extra will be left and the example module will immediately be declared, which will obviously add readability.
At the end of wrap.cpp we will write the implementation of our new functions, by analogy with the way we wrote them in the first part:
This I, of course, am already struggling with fat, but let's call it a repetition of the past.
In the structure wrapper, of course, we add new announcements:
With the structure of nothing complicated, it turned out to be a wonderful Python class that mirrors the properties of the Config structure in C ++ and at the same time the class is completely Pythonist. The only problem with this class will be that you need to specify something in the constructor when creating it.
To populate the configuration parameters and access them, let's set up a singleton, and at the same time provide it with a “useful” counter.
')
So singleton. Let it contain the aforementioned configuration parameters of the current application and some magic counter for getting the current identifier.
As you have probably noticed, I don’t really like to use the useless Instance () method in the public section and prefer working with the singleton functionality as a set of static methods. From this singleton does not cease to be a singleton, and the user of the class will thank you for hiding the Instance () call into the implementation.
This is actually the implementation in single.cpp :
Only three static methods, the wrapper should not be complicated, if you do not take into account one thing ... but no, not exactly one thing:
1. You cannot create an instance of a class.
2. You cannot copy an instance of a class.
3. We haven't wrapped static methods yet.
As you can see, all the difficulties associated with the 1st and 2nd items are reduced to specifying the template parameter boost :: noncopyable and passing the parameter boost :: python :: no_init to the template class constructor boost :: python :: class_.
If you want the class to support copying or contain a default constructor, you can erase the corresponding muffler of the property generator of the wrapper class.
Generally speaking, the default constructor can be declared below in .def (init <> ()) , some do so, for consistency with other constructors with parameters described separately, also passing no_init to the constructor of the wrapper pattern. There is also the option of replacing the default constructor with a constructor with parameters right when declaring a wrapper class, as we have already done for the Config structure.
With the third point, everything is generally simple, declaring that the static method deals with .staticmethod () after declaring all the overloads of this method through .def () .
In general, the rest no longer raises any questions and is familiar to us from the first part, except for one funny little thing - the policy of returning the value of return_value_policy <reference_existing_object>, about it further.
The greatest difficulty in wrapping the methods of our singleton was caused by the return of an object reference from the method
Just to ensure that the Python interpreter's Garbage Collector (GC) does not remove the contents of the class field returned by reference from the method, we use return_value_policy <reference_existing_object> .
The magic of Boost.Python is so severe that when executing Python code, changing the result fields of AppConfig () will lead to changes in the Singleton field as if it were happening in C ++! By running the following code from the Python command line:
We get the output:
Probably all have already noticed that I like to overload the method or the other in the example, so that the ad is as furious as possible. For the convenience of using the Single class in Python, we now add properties for reading the counter and for getting and setting configuration parameters, since all the methods for this are already there.
The Single :: CurrentID method is wrapped in the current_id property one-two times, but see what a “beautiful” wrapper is for the two Single :: AppConfig overloads, respectively, the get and set methods of the app_config property. And pay attention, for the get-method, we had to use the special function make_function in order to hang the return value policy return_value_policy <reference_existing_object> .
Be very careful, you cannot use the function make_getter for methods, it is used only for C ++ class fields, methods need to be used as is. If you need to set the return value policy for one of the property methods in one of the methods, you need to use make_function . You no longer have an auxiliary additional argument for return_value_policy, as in .def , so you have to pass both the function and the return value policy with one argument.
So, we have already figured out how not to let GC Python delete an object returned by reference. However, it is sometimes required to transfer a new object to the python for storage. GC will correctly remove the object as soonas the last variable dies in torment , referring to your result. For this there is a policy return_value_policy <manage_new_object> .
Let's get a method cloning configuration parameters to a new object. Add a declaration to wrap.h:
And in wrap.cpp we add its implementation:
In the wrapper of the class Single, respectively, a new method will appear with the manage_new_object policy:
To check that Config is really deleted when necessary, we will declare the destructor in a completely not C-style Config structure. In the destructor, we simply output the fields of the Config instance to delete in STDOUT via std :: cout:
We try!
In the test script in Python 3.x, we clone the configs, change them differently and reset all links to the object created via CloneAppConfig () :
Destructors are called exactly when it is expected, when the last link to the object disappears.
This is what should appear on the screen:
As a home task, try to add the __del__ method to the Config wrapper - an analogue of the destructor in Python, you will see how the wrappers behave and the objects they refer to.
So, we got acquainted in practice with two new return value policies by reference: reference_existing_object and manage_new_object . That is, they learned how to use the wrapper object of the return value as a link to an existing C ++ object, and also to transfer to GC Python the new objects created in C ++.
Understood in brief how to act if the default class constructors in C ++ are subject to restrictions. This is relevant not only in the case of a singleton or abstract class, but also for many specific classes, examples of which are probably before you now.
In the third part, we are waiting for a simple enum wrapper, we will write our converter for the byte array from C ++ to Python and back, and also learn how to use the inheritance of C ++ classes at the level of their wrappers.
Next, we are waiting for the magical world of converting exceptions from C ++ to Python and back.
What will happen next while I won’t guess, the topic is unwinding like a tangle: so small and compact, until you start unwinding it ...
Link to the project of the 2nd part can be found here . The MSVS v11 project is configured to build with Python 3.3 x64.
Boost.Python Documentation
Constructor wrapper class boost :: python :: class_
Policies return values ​​by reference in Boost.Python
Getting Started with Boost for Windows
Getting started with Boost for * nix
Boost.Python build subtleties
We continue to torment Boost.Python. This time it is the turn of the class, which can neither be created nor copied.
Let's wrap an almost ordinary sishny structure with an unusual designer.
And we’ll work with returning a reference to a C ++ object field so that the Python garbage collector doesn’t delete it inadvertently. Well and on the contrary, we will make an alternative option so that Python can clear up the garbage after removing what has been given to it for storage.
Go…
Table of contents
- Combining C ++ and Python. Subtleties Boost.Python. Part one
- Combining C ++ and Python. Subtleties Boost.Python. Part two
- Type conversion to Boost.Python. We do the conversion between the usual types of C + + and Python
- Exception travel between C ++ and Python, or "Back and forth"
Preparing a project
For our purposes, it would be enough for us to supplement the draft example remaining from the previous part.
Let's add to it a couple more files to work with the singleton class:
single.h
single.cpp
And we will render the declarations of auxiliary functions for wrapping in Python into a separate file:
wrap.h
From the previous project there should have been a file that we will actively change:
wrap.cpp
And the wonderful files with the miracle class, which helped us so much in the first part, they will remain as they are:
some.h
some.cpp
Wrapping a simple structure
Let's start with the fact that we get in single.h a small C-style structure, just with a description of the fields.
Let's for interest, this will be not just a structure, but a kind of cryptic configuration description type:
struct Config { double coef; string path; int max_size; Config( double def_coef, string const& def_path, int def_max_size ); }; It is easy to make a wrapper for such a structure, you just need to specifically describe the constructor with parameters using the constructor template boost :: python :: init <...> (...) of the wrapper template parameter boost :: python :: class_:
class_<Config>( "Config", init<double,string,int>( args( "coef", "path", "max_size" ) ) .add_property( "coef", make_getter( &Config::coef ), make_setter( &Config::coef ) ) .add_property( "path", make_getter( &Config::path ), make_setter( &Config::path ) ) .add_property( "max_size", make_getter( &Config::max_size ), make_setter( &Config::max_size ) ) ; As you can see, here you don’t even have to use return_value_policy <copy_const_reference> for a string field. Just because the field here is taken essentially by value, and therefore it will automatically be converted to the standard string of the Python language.
The make_setter functions still do a very useful job of checking the type of the incoming value, for example, try setting the coef field in Python to a string type or setting max_size with a value of type float, get an exception.
The fields of the config structure are essentially converted into properties of the object of the full-fledged Python class Config. Well, almost complete ... Let's, by analogy with the Some class from the last chapter, add the __str__ and __repr__ methods to the wrapper, and at the same time add the as_dict property to convert the structure fields to standard dict python and back.
Declaring new functions, as well as old ones, will be transferred to our new wrap.h file:
#pragma once #include <boost/python.hpp> #include "some.h" #include "single.h" using namespace boost::python; string Some_Str( Some const& ); string Some_Repr( Some const& ); dict Some_ToDict( Some const& ); void Some_FromDict( Some&, dict const& ); string Config_Str( Config const& ); string Config_Repr( Config const& ); dict Config_ToDict( Config const& ); void Config_FromDict( Config&, dict const& ); In the wrap.cpp file, nothing extra will be left and the example module will immediately be declared, which will obviously add readability.
At the end of wrap.cpp we will write the implementation of our new functions, by analogy with the way we wrote them in the first part:
string Config_Str( Config const& config ) { stringstream output; output << "{ coef: " << config.coef << ", path: '" << config.path << "', max_size: " << config.max_size << " }"; return output.str(); } string Config_Repr( Config const& config ) { return "Config: " + Config_Str( config ); } dict Config_ToDict( Config const& config ) { dict res; res["coef"] = config.coef; res["path"] = config.path; res["max_size"] = config.max_size; return res; } void Config_FromDict( Config& config, dict const& src ) { if( src.has_key( "coef" ) ) config.coef = extract<double>( src["coef"] ); if( src.has_key( "path" ) ) config.path = extract<string>( src["path"] ); if( src.has_key( "max_size" ) ) config.max_size = extract<int>( src["max_size"] ); } This I, of course, am already struggling with fat, but let's call it a repetition of the past.
In the structure wrapper, of course, we add new announcements:
class_<Config>( "Config", init<double,string,int>( args( "coef", "path", "max_size" ) ) ) .add_property( "coef", make_getter( &Config::coef ), make_setter( &Config::coef ) ) .add_property( "path", make_getter( &Config::path ), make_setter( &Config::path ) ) .add_property( "max_size", make_getter( &Config::max_size ), make_setter( &Config::max_size ) ) .def( "__str__", Config_Str ) .def( "__repr__", Config_Repr ) .add_property( "as_dict", Config_ToDict, Config_FromDict ) ; With the structure of nothing complicated, it turned out to be a wonderful Python class that mirrors the properties of the Config structure in C ++ and at the same time the class is completely Pythonist. The only problem with this class will be that you need to specify something in the constructor when creating it.
To populate the configuration parameters and access them, let's set up a singleton, and at the same time provide it with a “useful” counter.
')
Wrapper class without the ability to create and copy
So singleton. Let it contain the aforementioned configuration parameters of the current application and some magic counter for getting the current identifier.
class Single { public: static int CurrentID(); static Config& AppConfig(); static void AppConfig( Config const& ); private: int mCurrentID; Config mAppConfig; Single(); Single( Single const& ); static Single& Instance(); int ThisCurrentID(); Config& ThisAppConfig(); void ThisAppConfig( Config const& ); }; As you have probably noticed, I don’t really like to use the useless Instance () method in the public section and prefer working with the singleton functionality as a set of static methods. From this singleton does not cease to be a singleton, and the user of the class will thank you for hiding the Instance () call into the implementation.
This is actually the implementation in single.cpp :
#include "single.h" #include <boost/thread.hpp> using boost::mutex; using boost::unique_lock; const double CONFIG_DEFAULT_COEF = 2.5; const int CONFIG_DEFAULT_MAX_SIZE = 0x1000; const string CONFIG_DEFAULT_PATH = "."; int Single::CurrentID() { return Instance().ThisCurrentID(); } Config& Single::AppConfig() { return Instance().ThisAppConfig(); } void Single::AppConfig( Config const& config ) { Instance().ThisAppConfig( config ); } Single::Single() : mCurrentID( 0 ) { mAppConfig.coef = CONFIG_DEFAULT_COEF; mAppConfig.max_size = CONFIG_DEFAULT_MAX_SIZE; mAppConfig.path = CONFIG_DEFAULT_PATH; } Single& Single::Instance() { static mutex single_mutex; unique_lock<mutex> single_lock( single_mutex ); static Single instance; return instance; } int Single::ThisCurrentID() { static mutex id_mutex; unique_lock<mutex> id_lock( id_mutex ); return ++mCurrentID; } Config& Single::ThisAppConfig() { return mAppConfig; } void Single::ThisAppConfig( Config const& config ) { mAppConfig = config; } Only three static methods, the wrapper should not be complicated, if you do not take into account one thing ... but no, not exactly one thing:
1. You cannot create an instance of a class.
2. You cannot copy an instance of a class.
3. We haven't wrapped static methods yet.
class_<Single, noncopyable>( "Single", no_init ) .def( "CurrentID", &Single::CurrentID ) .staticmethod( "CurrentID" ) .def( "AppConfig", static_cast< Config& (*)() >( &Single::AppConfig ), return_value_policy<reference_existing_object>() ) .def( "AppConfig", static_cast< void (*)( Config const& ) >( &Single::AppConfig ) ) .staticmethod( "AppConfig" ) ; As you can see, all the difficulties associated with the 1st and 2nd items are reduced to specifying the template parameter boost :: noncopyable and passing the parameter boost :: python :: no_init to the template class constructor boost :: python :: class_.
If you want the class to support copying or contain a default constructor, you can erase the corresponding muffler of the property generator of the wrapper class.
Generally speaking, the default constructor can be declared below in .def (init <> ()) , some do so, for consistency with other constructors with parameters described separately, also passing no_init to the constructor of the wrapper pattern. There is also the option of replacing the default constructor with a constructor with parameters right when declaring a wrapper class, as we have already done for the Config structure.
With the third point, everything is generally simple, declaring that the static method deals with .staticmethod () after declaring all the overloads of this method through .def () .
In general, the rest no longer raises any questions and is familiar to us from the first part, except for one funny little thing - the policy of returning the value of return_value_policy <reference_existing_object>, about it further.
The policy "do not hit me, I am a translator"
The greatest difficulty in wrapping the methods of our singleton was caused by the return of an object reference from the method
static Config& AppConfig(); Just to ensure that the Python interpreter's Garbage Collector (GC) does not remove the contents of the class field returned by reference from the method, we use return_value_policy <reference_existing_object> .
The magic of Boost.Python is so severe that when executing Python code, changing the result fields of AppConfig () will lead to changes in the Singleton field as if it were happening in C ++! By running the following code from the Python command line:
from example import * Single.AppConfig().coef = 123.45 Single.AppConfig() We get the output:
Config: { coef: 123.45, path: '.', max_size: 4096 } Add a property with a policy for the get method
Probably all have already noticed that I like to overload the method or the other in the example, so that the ad is as furious as possible. For the convenience of using the Single class in Python, we now add properties for reading the counter and for getting and setting configuration parameters, since all the methods for this are already there.
.add_static_property( "current_id", &Single::CurrentID ) .add_static_property( "app_config", make_function( static_cast< Config& (*)() >( &Single::AppConfig ), return_value_policy<reference_existing_object>() ), static_cast< void (*)( Config const& ) >( &Single::AppConfig ) ) The Single :: CurrentID method is wrapped in the current_id property one-two times, but see what a “beautiful” wrapper is for the two Single :: AppConfig overloads, respectively, the get and set methods of the app_config property. And pay attention, for the get-method, we had to use the special function make_function in order to hang the return value policy return_value_policy <reference_existing_object> .
Be very careful, you cannot use the function make_getter for methods, it is used only for C ++ class fields, methods need to be used as is. If you need to set the return value policy for one of the property methods in one of the methods, you need to use make_function . You no longer have an auxiliary additional argument for return_value_policy, as in .def , so you have to pass both the function and the return value policy with one argument.
Policy "here is a new object - delete it"
So, we have already figured out how not to let GC Python delete an object returned by reference. However, it is sometimes required to transfer a new object to the python for storage. GC will correctly remove the object as soon
Let's get a method cloning configuration parameters to a new object. Add a declaration to wrap.h:
Config* Single_CloneAppConfig(); And in wrap.cpp we add its implementation:
Config* Single_CloneAppConfig() { return new Config( Single::AppConfig() ); } In the wrapper of the class Single, respectively, a new method will appear with the manage_new_object policy:
.def( "CloneAppConfig", Single_CloneAppConfig, return_value_policy<manage_new_object>() ) To check that Config is really deleted when necessary, we will declare the destructor in a completely not C-style Config structure. In the destructor, we simply output the fields of the Config instance to delete in STDOUT via std :: cout:
Config::~Config() { cout << "Config destructor of Config: { coef: " << coef << ", path: '" << path << "', max_size: " << max_size << " }" << endl; } We try!
In the test script in Python 3.x, we clone the configs, change them differently and reset all links to the object created via CloneAppConfig () :
from example import * c = Single.CloneAppConfig() c.coef = 11.11; c.path = 'cloned'; c.max_size = 111111 print( "c.coef = 12.34; c.path = 'cloned'; c.max_size = 100500" ) print( "c:", c ); print( "Single.AppConfig():", Single.AppConfig() ) print( "c = Single.CloneAppConfig()" ); c = Single.CloneAppConfig() c.coef = 22.22; c.path = 'another'; c.max_size = 222222 print( "c.coef = 22.22; c.path = 'another'; c.max_size = 222222" ) print( "c:", c ); print( "Single.app_config:", Single.app_config ) print( "c = None" ); c = None print( "Single.app_config:", Single.app_config ) Destructors are called exactly when it is expected, when the last link to the object disappears.
This is what should appear on the screen:
c.coef = 12.34; c.path = 'cloned'; c.max_size = 100500 c: { coef: 11.11, path: 'cloned', max_size: 111111 } Single.AppConfig(): { coef: 2.5, path: '.', max_size: 4096 } c = Single.CloneAppConfig() Config::~Config() destructor of object: { coef: 11.11, path: 'cloned', max_size: 111111 } c.coef = 22.22; c.path = 'another'; c.max_size = 222222 c: { coef: 22.22, path: 'another', max_size: 222222 } Single.app_config: { coef: 2.5, path: '.', max_size: 4096 } c = None Config::~Config() destructor of object: { coef: 22.22, path: 'another', max_size: 222222 } Single.app_config: { coef: 2.5, path: '.', max_size: 4096 } Config::~Config() destructor of object: { coef: 2.5, path: '.', max_size: 4096 } As a home task, try to add the __del__ method to the Config wrapper - an analogue of the destructor in Python, you will see how the wrappers behave and the objects they refer to.
In conclusion of the second part
So, we got acquainted in practice with two new return value policies by reference: reference_existing_object and manage_new_object . That is, they learned how to use the wrapper object of the return value as a link to an existing C ++ object, and also to transfer to GC Python the new objects created in C ++.
Understood in brief how to act if the default class constructors in C ++ are subject to restrictions. This is relevant not only in the case of a singleton or abstract class, but also for many specific classes, examples of which are probably before you now.
In the third part, we are waiting for a simple enum wrapper, we will write our converter for the byte array from C ++ to Python and back, and also learn how to use the inheritance of C ++ classes at the level of their wrappers.
Next, we are waiting for the magical world of converting exceptions from C ++ to Python and back.
What will happen next while I won’t guess, the topic is unwinding like a tangle: so small and compact, until you start unwinding it ...
Link to the project of the 2nd part can be found here . The MSVS v11 project is configured to build with Python 3.3 x64.
useful links
Boost.Python Documentation
Constructor wrapper class boost :: python :: class_
Policies return values ​​by reference in Boost.Python
Getting Started with Boost for Windows
Getting started with Boost for * nix
Boost.Python build subtleties
Source: https://habr.com/ru/post/168233/
All Articles