Exception travel between C ++ and Python, or "Back and forth"
In this chapter, a tale about friendships between C ++ and Python will have surprisingly little use of Boost.Python. Passing exceptions back and forth is essentially a weak point of this library. We will manage the native API of the Python language, and where it is possible to use Boost.Python.
Nevertheless, Boost.Python creates an environment in which exceptions from C ++ fall into Python as a standard RuntimeError, and back from Python, a C ++ exception of the error_already_set type is generated, which means “something has flown to you, go see what is there”. And here we just need to use the Python C-API to subtract the necessary information about the exception and convert it into the appropriate class according to the logic of the application.
Why such difficulties? - The fact is that in Python, unlike C ++, besides the exception text and its type, traceback comes as well - the stack to the point where the exception occurred. Let's extend the standard std :: exception with an additional parameter for this stacktrace, and at the same time we will write an exception converter back and forth from the C ++ classes to the Python exception classes.
1. Boost.Python. Introduction C ++ wrappers in Python.
2. Boost.Python. A wrapper for specific C ++ classes.
3. Boost.Python. Creating type converters between C ++ and Python.
Suppose there is a certain hierarchy of exceptions that you need to be one-to-one in the form of the corresponding class in C ++ and in Python when handling the exception. This is especially true if you combine application logic in C ++ and complex scripting in Python, or if you are writing a module for Python in C ++ with complex logic. Sooner or later we run into exception handling coming from C ++ in Python or vice versa.
Of course, in most cases, you will only need the standard Boost.Python mechanism to convert exceptions from C ++ to Python in the form of a RuntimeException with the text that came from exception :: what (). On the C ++ side, you need to catch an exception of the error_already_set type and use the native API of the Python language, but you can not only subtract the type and text of the exception, but also the traceback - in fact, the exception history.
But first things first.
')
So, you wrote a module in C ++ using Boost.Python, hooked it up in Python code via a regular import and use one of the wrappers for functions or methods. For example, in C ++ code, the most common exception is thrown through an equally ordinary throw . In Python code, you will get a RuntimeException with text obtained from exception :: what () if this exception is thrown from std :: exception.
If you are satisfied that you get nothing but the exception text, then you can even do nothing more. However, if you need to catch the exception of a strictly defined class of errors, you will need to work a little.
In fact, Boost.Python provides the ability to register your translation of the exception of a module leaving native penats written in C ++. All you need to do: in the declaration function of the module, call the template function boost :: python :: register_exception_translator <T, F> (F), where T is the exception type in C ++, and F is the function that accepts the reference to the exception of this type and This is a way of doing your duty to pass an exception of the desired type to external code already in Python. In general, something like this:
Here we used the standard exception Exception embedded in Python, but you can use absolutely any exception: standard , external connected via import with getting PyObject * via object :: ptr () or even your own one created right there on site via PyErr_NewException .
Let's add for completeness of sensations a couple more classes, which will be given as an analogue of ZeroDivisionError and ValueError and for complete happiness we will inherit them from our error, let's call them respectively zero_division_error and value_error:
We will need the m_details field on the way back from Python to C ++, for example, to save a traceback. And the type method () will be needed for debugging a bit later. Simple and clear hierarchy.
Register for our exceptions function translators in Python:
Great, it remains only to get test functions on the C ++ side, which will throw these exceptions:
Why not, these functions are not worse than any other and throw exactly what we need.
Wrap them up:
Well, we assemble our module, import python_module , call our functions with the necessary parameters, get the necessary exceptions (script in Python 3.x):
Script output:
ZeroDivisionError - OK
ValueError - OK
Exception - OK
So far, so good. Let's go in the opposite direction.
Let's separate into a separate project the types of exceptions and test functions and we will build from them a separate dynamic library, the error_types . We will build the module for Python separately in the python_module project.
And now we will get application on C ++ where we will catch exceptions from Python, we will call it catch_exceptions .
All you need is to connect our module via import ("python_module"), then get access to the module's functions via attr ("divide"), attr ("to_num"), attr ("test"). We will call them, they will raise exceptions at the level of C ++ code, pass into the Python interpreter and forward to the C ++ application, causing the exception error_already_set - the exception of the Boost.Python library prepared just for such cases.
By itself, an object of type error_already_set , it is important to just catch the exception. In general, the handling of such an exception is as follows:
So we will always get the exception of the same type, but at least we can extract the text of the exception. But we get the exception type in the variable exc, the exception object itself in the val variable, and even the object with the exception stack in the tb variable. Let's convert the exception to zero_division_error and value_error if ZeroDivisionError or ValueError, respectively, arrived.
Stop! Not everyone understands what these two functions are, why everything is PyObject *, whence exceptions in the C-API, if they are not in C, let's take a look.
Yes, there are no exceptions in pure C, but in Python they exist and its API provides an opportunity to extract information about the exception that occurred. In Python C-API, all values ​​and types, and in general almost everything, is represented as PyObject *, therefore an exception E of type T is a pair of values ​​of type PyObject *, we add to this also PyObject * for traceback - the saved stack where the exception occurred .
You can extract information about an exception that has occurred with the PyErr_Fetch function, after which the exception information can be normalized (if you don’t want to work in the internal representation in the form of a tuple) with the PyErr_NormalizeException function.
After calling a pair of these functions, we will fill in three values ​​of the PyObject * type, respectively: exception class, exception instance (object), and stack (traceback) saved at the time the exception was generated.
Further, it is much more convenient to work with Boost.Python, wrap PyObject * in boost :: python :: handle <>, which is compatible with any object of the Boost.Python library, we just need boost :: python :: str . After the conversion to the analogue of the python string in Boost.Python, we can extend the standard native string std :: string of the C ++ language. If desired, you can extend the usual const char *.
With the first two parameters, everything is clear, they are perfectly reduced to a string, but with traceback you will still have to convert to a readable form. The easiest way to do this is with the traceback module, passing our three parameters to the format_exception function. The traceback.format_exception (exc, val, tb) function will return an array of strings to us in the form of a standard Python list list , which joins into one large thick string.
On the C ++ side, using Boost.Python, it will look something like this:
You can make an auxiliary function to generate a string from the exception. The problem, however, is that the import () call in the function each time leads to an expensive call, so the object obtained from import ("traceback") .attr ("format_exception") is best to save the result of the function in a separate object, we also need to save import result ("python_module"). Considering that this will need to be done somewhere between Py_Initialize () and Py_Finalize (), nothing better than singleton fields for storing such variables does not occur.
So, let's get a singleton, it will complicate the application, but it will simplify the code a bit and allow us to correctly initialize the work with the interpreter, save all auxiliary objects and complete everything correctly:
The constructor will initialize the work with the interpreter, and the destructor will clean up the saved fields and de-initialize the work with the interpreter. The python_module and format_error methods import the corresponding modules only once:
So we got a ready-made mechanism, applicable for any C ++ application that uses Python as a powerful auxiliary functionality with a bunch of libraries.
It's time to check our exception mechanism!
Let's get an auxiliary function:
Then the exception handling mechanism will be reduced to the following scheme for each method under test, such as for divide:
Here, output_error is the simplest function that outputs information about the exception, for example, like this:
This is where we really need the type () virtual method that we have introduced in the error base class.
We create similar sections for to_num and for test, and also check what comes up if we just execute the “1/0” line in Python via exec:
Running ...
The output should be something like this:
Error type: zero_division_error
Message: Division by zero!
Details: <class 'ZeroDivisionError'>
Error type: value_error
Message: Inappropriate value!
Details: <class 'ValueError'>
Error type: error
Message: Test failure.
Details: <class 'Exception'>
Error type: zero_division_error
Message: division by zero
Details: Traceback (most recent call last):
File "", line 1, in ZeroDivisionError: division by zero
Total: we received the mechanism of one-to-one conversion of exceptions from Python to C ++ and back.
The minus is noticeable immediately - these are different completely unrelated entities. This is due to the fact that the C ++ class cannot be inherited from a class from Python, nor is it vice versa. There is a variant with the “encapsulation” of the required exception class in the wrapper of the C ++ class for Python, but these will still be different classes that are simply converted to the corresponding ones in their own language.
If you have a complicated exception hierarchy in C ++, the easiest way is to get yourself an analogue in Python in a separate .py module, because creating PyErr_NewException and then storing it somewhere is quite expensive and will not add readability to the code.
I don’t know about you, but I’m looking forward to when Boost.Python will get a decent exception translator, or at least an analogue of boost :: python :: bases to inherit the wrapper from the Python class. In general, Boost.Python is an excellent library, but this aspect adds to the hemorrhoid when parsing exceptions from Python on the C ++ side. Translation in Python through registration of the register_exception_translator <E, F> (F) translator function looks quite successful and allows you to convert an A type exception to C ++, to a completely different class B on the Python side, but at least automatically.
In principle, it is not necessary to respond to the error_already_set exactly as described above, you can choose for yourself the recipe for the behavior of your application using the Python API to handle exceptions from Python .
Link to the project is here (~ 223 KB) . The MSVS v11 project is configured to build with Boost 1.52 and Python 3.3 x64.
PyWiki - Extracting C ++ Exceptions
Exception Handling with Python C-API
Register translator exceptions from C ++ to Python
Nevertheless, Boost.Python creates an environment in which exceptions from C ++ fall into Python as a standard RuntimeError, and back from Python, a C ++ exception of the error_already_set type is generated, which means “something has flown to you, go see what is there”. And here we just need to use the Python C-API to subtract the necessary information about the exception and convert it into the appropriate class according to the logic of the application.
Why such difficulties? - The fact is that in Python, unlike C ++, besides the exception text and its type, traceback comes as well - the stack to the point where the exception occurred. Let's extend the standard std :: exception with an additional parameter for this stacktrace, and at the same time we will write an exception converter back and forth from the C ++ classes to the Python exception classes.
In the previous series
1. Boost.Python. Introduction C ++ wrappers in Python.
2. Boost.Python. A wrapper for specific C ++ classes.
3. Boost.Python. Creating type converters between C ++ and Python.
Introduction
Suppose there is a certain hierarchy of exceptions that you need to be one-to-one in the form of the corresponding class in C ++ and in Python when handling the exception. This is especially true if you combine application logic in C ++ and complex scripting in Python, or if you are writing a module for Python in C ++ with complex logic. Sooner or later we run into exception handling coming from C ++ in Python or vice versa.
Of course, in most cases, you will only need the standard Boost.Python mechanism to convert exceptions from C ++ to Python in the form of a RuntimeException with the text that came from exception :: what (). On the C ++ side, you need to catch an exception of the error_already_set type and use the native API of the Python language, but you can not only subtract the type and text of the exception, but also the traceback - in fact, the exception history.
But first things first.
')
Traveling small exceptions from C ++ to Python
So, you wrote a module in C ++ using Boost.Python, hooked it up in Python code via a regular import and use one of the wrappers for functions or methods. For example, in C ++ code, the most common exception is thrown through an equally ordinary throw . In Python code, you will get a RuntimeException with text obtained from exception :: what () if this exception is thrown from std :: exception.
If you are satisfied that you get nothing but the exception text, then you can even do nothing more. However, if you need to catch the exception of a strictly defined class of errors, you will need to work a little.
In fact, Boost.Python provides the ability to register your translation of the exception of a module leaving native penats written in C ++. All you need to do: in the declaration function of the module, call the template function boost :: python :: register_exception_translator <T, F> (F), where T is the exception type in C ++, and F is the function that accepts the reference to the exception of this type and This is a way of doing your duty to pass an exception of the desired type to external code already in Python. In general, something like this:
class error : public exception { ... }; ... void translate_error( error const& ); ... BOOST_PYTHON_MODULE( ... ) { ... register_exception_translator<error>( translate_error ); } ... void translate_error( error const& e ) { PyErr_SetString( PyExc_Exception, e.what() ); } Here we used the standard exception Exception embedded in Python, but you can use absolutely any exception: standard , external connected via import with getting PyObject * via object :: ptr () or even your own one created right there on site via PyErr_NewException .
Let's add for completeness of sensations a couple more classes, which will be given as an analogue of ZeroDivisionError and ValueError and for complete happiness we will inherit them from our error, let's call them respectively zero_division_error and value_error:
class error : public exception { public: error(); error( string const& message ); error( string const& message, string const& details ); virtual const char* what() const; virtual string const& get_message() const; virtual string const& get_details() const; virtual const char* type() const; private: string m_message; string m_details; }; class value_error : public error { public: value_error(); value_error( string const& message ); value_error( string const& message, string const& details ); virtual const char* type() const; }; class zero_division_error : public error { public: zero_division_error(); zero_division_error( string const& message ); zero_division_error( string const& message, string const& details ); virtual const char* type() const; }; We will need the m_details field on the way back from Python to C ++, for example, to save a traceback. And the type method () will be needed for debugging a bit later. Simple and clear hierarchy.
Register for our exceptions function translators in Python:
void translate_error( error const& ); void translate_value_error( value_error const& ); void translate_zero_division_error( zero_division_error const& ); ... BOOST_PYTHON_MODULE( ... ) { ... register_exception_translator<error>( translate_error ); register_exception_translator<value_error>( translate_value_error ); register_exception_translator<zero_division_error>( translate_zero_division_error ); } ... void translate_error( error const& e ) { PyErr_SetString( PyExc_Exception, e.what() ); } void translate_value_error( value_error const& e ) { PyErr_SetString( PyExc_ValueError, e.what() ); } void translate_zero_division_error( zero_division_error const& e ) { PyErr_SetString( PyExc_ZeroDivisionError, e.what() ); } Great, it remains only to get test functions on the C ++ side, which will throw these exceptions:
double divide( double a, double b ) { if( abs( b ) < numeric_limits<double>::epsilon() ) throw zero_division_error(); return a / b; } double to_num( const char* val ) { double res; if( !val || !sscanf( val, "%LG", &res ) ) throw value_error(); return res; } void test( bool val ) { if( !val ) throw error( "Test failure.", "test" ); } Why not, these functions are not worse than any other and throw exactly what we need.
Wrap them up:
... BOOST_PYTHON_MODULE( python_module ) { register_exception_translator<error>( translate_error ); register_exception_translator<value_error>( translate_value_error ); register_exception_translator<zero_division_error>( translate_zero_division_error ); def( "divide", divide, args( "a", "b" ) ); def( "to_num", to_num, args( "val" ) ); def( "test", test, args( "val" ) ); } ... Well, we assemble our module, import python_module , call our functions with the necessary parameters, get the necessary exceptions (script in Python 3.x):
import python_module as pm try: res = pm.divide( 1, 0 ) except ZeroDivisionError: print( "ZeroDivisionError - OK" ) except Exception as e: print( "Expected ZeroDivisionError, but exception of type '{t}' with text: '{e}'".format(t=type(e),e=e) ) else: print( "Expected ZeroDivisionError, but no exception raised! Result: {r}".format(r=res) ) try: res = pm.to_num( 'qwe' ) except ValueError: print( "ValueError - OK" ) except Exception as e: print( "Expected ValueError, but exception of type '{t}' with text: '{e}'".format(t=type(e),e=e) ) else: print( "Expected ValueError, but no exception raised! Result: {r}".format(r=res) ) try: res = pm.test( False ) except Exception as e: if type(e) is Exception: print( "Exception - OK" ) else: print( "Exception of type '{t}', expected type 'Exception', message: '{e}'".format(t=type(e),e=e) ) else: print( "Expected Exception, but no exception raised! Result: {r}".format(r=res) ) Script output:
ZeroDivisionError - OK
ValueError - OK
Exception - OK
So far, so good. Let's go in the opposite direction.
Adventures of our exception on the way from Python to C ++
Let's separate into a separate project the types of exceptions and test functions and we will build from them a separate dynamic library, the error_types . We will build the module for Python separately in the python_module project.
And now we will get application on C ++ where we will catch exceptions from Python, we will call it catch_exceptions .
All you need is to connect our module via import ("python_module"), then get access to the module's functions via attr ("divide"), attr ("to_num"), attr ("test"). We will call them, they will raise exceptions at the level of C ++ code, pass into the Python interpreter and forward to the C ++ application, causing the exception error_already_set - the exception of the Boost.Python library prepared just for such cases.
By itself, an object of type error_already_set , it is important to just catch the exception. In general, the handling of such an exception is as follows:
catch( error_already_set const& ) { PyObject *exc, *val, *tb; PyErr_Fetch( &exc, &val, &tb ); PyErr_NormalizeException( &exc, &val, &tb ); handle<> hexc(exc), hval( allow_null( val ) ), htb( allow_null( tb ) ); throw error( extract<string>( !hval ? str( hexc ) : str( hval ) ) ); } So we will always get the exception of the same type, but at least we can extract the text of the exception. But we get the exception type in the variable exc, the exception object itself in the val variable, and even the object with the exception stack in the tb variable. Let's convert the exception to zero_division_error and value_error if ZeroDivisionError or ValueError, respectively, arrived.
Stop! Not everyone understands what these two functions are, why everything is PyObject *, whence exceptions in the C-API, if they are not in C, let's take a look.
Yes, there are no exceptions in pure C, but in Python they exist and its API provides an opportunity to extract information about the exception that occurred. In Python C-API, all values ​​and types, and in general almost everything, is represented as PyObject *, therefore an exception E of type T is a pair of values ​​of type PyObject *, we add to this also PyObject * for traceback - the saved stack where the exception occurred .
You can extract information about an exception that has occurred with the PyErr_Fetch function, after which the exception information can be normalized (if you don’t want to work in the internal representation in the form of a tuple) with the PyErr_NormalizeException function.
After calling a pair of these functions, we will fill in three values ​​of the PyObject * type, respectively: exception class, exception instance (object), and stack (traceback) saved at the time the exception was generated.
Further, it is much more convenient to work with Boost.Python, wrap PyObject * in boost :: python :: handle <>, which is compatible with any object of the Boost.Python library, we just need boost :: python :: str . After the conversion to the analogue of the python string in Boost.Python, we can extend the standard native string std :: string of the C ++ language. If desired, you can extend the usual const char *.
With the first two parameters, everything is clear, they are perfectly reduced to a string, but with traceback you will still have to convert to a readable form. The easiest way to do this is with the traceback module, passing our three parameters to the format_exception function. The traceback.format_exception (exc, val, tb) function will return an array of strings to us in the form of a standard Python list list , which joins into one large thick string.
On the C ++ side, using Boost.Python, it will look something like this:
... format_exception = import( "traceback" ).attr( "format_exception" ); return extract<string>( str( "" ).join( format_exception( exc, val, tb ) ) ); } You can make an auxiliary function to generate a string from the exception. The problem, however, is that the import () call in the function each time leads to an expensive call, so the object obtained from import ("traceback") .attr ("format_exception") is best to save the result of the function in a separate object, we also need to save import result ("python_module"). Considering that this will need to be done somewhere between Py_Initialize () and Py_Finalize (), nothing better than singleton fields for storing such variables does not occur.
Work with Python API via singleton
So, let's get a singleton, it will complicate the application, but it will simplify the code a bit and allow us to correctly initialize the work with the interpreter, save all auxiliary objects and complete everything correctly:
class python_interpreter { public: static double divide( double, double ); static double to_num( string const& ); static void test( bool ); static string format_error( handle<> const&, handle<> const&, handle<> const& ); private: object m_python_module; object m_format_exception; python_interpreter(); ~python_interpreter(); static python_interpreter& instance(); object& python_module(); string format_error( object const&, object const&, object const& ); }; The constructor will initialize the work with the interpreter, and the destructor will clean up the saved fields and de-initialize the work with the interpreter. The python_module and format_error methods import the corresponding modules only once:
python_interpreter::python_interpreter() { Py_Initialize(); } python_interpreter::~python_interpreter() { m_python_module = object(); m_format_exception = object(); Py_Finalize(); } double python_interpreter::divide( double a, double b ) { return extract<double>( instance().python_module().attr("divide")( a, b ) ); } double python_interpreter::to_num( string const& val ) { return extract<double>( instance().python_module().attr("to_num")( val ) ); } void python_interpreter::test( bool val ) { instance().python_module().attr("test")( val ); } string python_interpreter::format_error( handle<> const& exc, handle<> const& msg, handle<> const& tb ) { return instance().format_error( object(exc), object(msg), object(tb) ); } python_interpreter& python_interpreter::instance() { static python_interpreter single; return single; } object& python_interpreter::python_module() { if( m_python_module.is_none() ) m_python_module = import( "python_module" ); return m_python_module; } string python_interpreter::format_error( object const& exc, object const& val, object const& tb ) { if( m_format_exception.is_none() ) m_format_exception = import( "traceback" ).attr( "format_exception" ); return extract<string>( str( "" ).join( m_format_exception( exc, val, tb ) ) ); } So we got a ready-made mechanism, applicable for any C ++ application that uses Python as a powerful auxiliary functionality with a bunch of libraries.
It's time to check our exception mechanism!
Checking the mechanism for translating exceptions from Python to C ++
Let's get an auxiliary function:
void rethrow_python_exception() { PyObject *exc, *val, *tb; PyErr_Fetch( &exc, &val, &tb ); PyErr_NormalizeException( &exc, &val, &tb ); handle<> hexc(exc), hval( allow_null( val ) ), htb( allow_null( tb ) ); string message, details; message = extract<string>( !hval ? str( hexc ) : str( hval ) ); details = !tb ? extract<string>( str( hexc ) ) : python_interpreter::format_error( hexc, hval, htb ); if( PyObject_IsSubclass( exc, PyExc_ZeroDivisionError ) ) throw zero_division_error( message, details ); else if( PyObject_IsSubclass( exc, PyExc_ValueError ) ) throw value_error( message, details ); else throw error( message, details ); } Then the exception handling mechanism will be reduced to the following scheme for each method under test, such as for divide:
try { try { python_interpreter::divide( 1, 0 ); } catch( error_already_set const& ) { rethrow_python_exception(); } } catch( error const& e ) { output_error( e ); } Here, output_error is the simplest function that outputs information about the exception, for example, like this:
void output_error( error const& e ) { cerr << "\nError type: " << e.type() << "\nMessage: " << e.get_message() << "\nDetails: " << e.get_details() << endl; } This is where we really need the type () virtual method that we have introduced in the error base class.
We create similar sections for to_num and for test, and also check what comes up if we just execute the “1/0” line in Python via exec:
try { try { exec( "1 / 0" ); } catch( error_already_set const& ) { rethrow_python_exception(); } } catch( error const& e ) { output_error( e ); } Running ...
The output should be something like this:
Error type: zero_division_error
Message: Division by zero!
Details: <class 'ZeroDivisionError'>
Error type: value_error
Message: Inappropriate value!
Details: <class 'ValueError'>
Error type: error
Message: Test failure.
Details: <class 'Exception'>
Error type: zero_division_error
Message: division by zero
Details: Traceback (most recent call last):
File "", line 1, in ZeroDivisionError: division by zero
Epilogue
Total: we received the mechanism of one-to-one conversion of exceptions from Python to C ++ and back.
The minus is noticeable immediately - these are different completely unrelated entities. This is due to the fact that the C ++ class cannot be inherited from a class from Python, nor is it vice versa. There is a variant with the “encapsulation” of the required exception class in the wrapper of the C ++ class for Python, but these will still be different classes that are simply converted to the corresponding ones in their own language.
If you have a complicated exception hierarchy in C ++, the easiest way is to get yourself an analogue in Python in a separate .py module, because creating PyErr_NewException and then storing it somewhere is quite expensive and will not add readability to the code.
I don’t know about you, but I’m looking forward to when Boost.Python will get a decent exception translator, or at least an analogue of boost :: python :: bases to inherit the wrapper from the Python class. In general, Boost.Python is an excellent library, but this aspect adds to the hemorrhoid when parsing exceptions from Python on the C ++ side. Translation in Python through registration of the register_exception_translator <E, F> (F) translator function looks quite successful and allows you to convert an A type exception to C ++, to a completely different class B on the Python side, but at least automatically.
In principle, it is not necessary to respond to the error_already_set exactly as described above, you can choose for yourself the recipe for the behavior of your application using the Python API to handle exceptions from Python .
Link to the project is here (~ 223 KB) . The MSVS v11 project is configured to build with Boost 1.52 and Python 3.3 x64.
useful links
PyWiki - Extracting C ++ Exceptions
Exception Handling with Python C-API
Register translator exceptions from C ++ to Python
Source: https://habr.com/ru/post/169639/
All Articles