Unmanaged C ++ library in .NET. Full integration

The article describes the full integration of C ++ libraries in a managed environment using Platform Invoke. Full integration implies the possibility of class inheritance of the library, the implementation of its interfaces (the interfaces will be represented in managed code as abstract classes). Instances of the heirs can be "transferred" to the unmanaged environment.

The issue of integration has been raised more than once in Habré, but, as a rule, it is dedicated to the integration of a couple of methods that cannot be implemented in managed code. We had the task to take a module from C ++ and make it work in .NET. The option to write anew, for several reasons, was not considered, so we started the integration.

This article does not reveal all the integration issues of the unmanaged module in .NET. There are also nuances with the transfer of strings, logical values, etc. ... There are documentation on these issues and several articles on Habré, so here these questions were not considered.

It is worth noting that the .NET wrapper based on Platform Invoke is cross-platform, it can be assembled on Mono + gcc.
')

Sealed class integration

The first thing to realize when integrating with Platform Invoke is that this tool allows you to integrate only individual functions. You can't just take a class and integrate it. The solution to the problem looks simple:

On the Unmanaged side, we write a function:

SomeType ClassName_methodName(ClassName * instance, SomeOtherType someArgument) { instance->methodName(someArgument); } 

Do not forget to add extern "C" to such functions so that their names are not decorated with a C ++ compiler. This would prevent us from integrating these features into .NET.

Then we repeat the procedure for all public methods of the class and integrate the functions obtained into the class written in .NET. The resulting class cannot be inherited, so in .NET such a class is declared as sealed. How to get around this limitation and what it is connected with - see below.
In the meantime, here is a small example:

Unmanaged class:

 class A { int mField; public: A( int someArgument); int someMethod( int someArgument); }; 

Functions for integration:

 A * A_createInstance(int someArgument) { return new A(someArgument); } int A_someMethod(A *instance, int someArgument) { return instance->someMethod( someArgument); } void A_deleteInstance(A *instance) { delete instance; } 

Implementation in .Net:

 public sealed class A { private IntPtr mInstance; private bool mDelete; [ DllImport( "shim.dll", CallingConvention = CallingConvention .Cdecl)] private static extern IntPtr A_createInstance( int someArgument); [ DllImport( "shim.dll", CallingConvention = CallingConvention .Cdecl)] private static extern int A_someMethod( IntPtr instance, int someArgument); [ DllImport( "shim.dll", CallingConvention = CallingConvention .Cdecl)] private static extern void A_deleteInstance( IntPtr instance); internal A( IntPtr instance) { Debug.Assert(instance != IntPtr.Zero); mInstance = instance; mDelete = false; } public A( int someArgument) { mInstance = A_createInstance(someArgument); mDelete = true; } public int someMethod( int someArgument) { return A_someMethod(mInstance, someArgument); } internal IntPtr getUnmanaged() { return mInstance; } ~A() { if (mDelete) A_deleteInstance(mInstance); } } 

The internal constructor and method are needed to get class instances from unmanaged code and pass them back. It is with the transfer of a class instance back to the unmanaged environment that the problem of inheritance is related. If class A is inherited in .NET and overridden a number of its methods (imagine that someMethod is declared with the virtual keyword), we will not be able to call the redefined code from the unmanaged environment.

Interface integration

For the integration of interfaces, we need feedback. Those. To fully use the integrated module, we need the ability to implement its interfaces. The implementation is associated with the definition of methods in a managed environment. These methods will need to be called from unmanaged code. This is where Callback Methods, described in the Platform Invoke documentation, comes to our rescue.

On the unmanaged side, the Callback environment is represented as a function pointer:

 typedef void (*PFN_MYCALLBACK )(); int _MyFunction(PFN_MYCALLBACK callback); 

And in .NET, the delegate will play his role:

 [UnmanagedFunctionPointerAttribute( CallingConvention.Cdecl)] public delegate void MyCallback (); [ DllImport("MYDLL.DLL",CallingConvention.Cdecl)] public static extern void MyFunction( MyCallback callback); 

With a feedback tool, we can easily provide a call to the overridden methods.

But in order to transfer an instance of an interface implementation, in an unmanaged environment, we will also have to present it as an instance of an implementation. So you have to write another implementation in an unmanaged environment. In this implementation, we, by the way, lay calls to Callback functions.

Unfortunately, this approach will not allow us to do without logic in the managed interfaces, so we will have to present them in the form of abstract classes. Let's look at the code:

Unmanaged interface:

 class IB { public: virtual int method( int arg) = 0; virtual ~IB() {}; }; 

Unmanaged implementation

 typedef int (*IB_method_ptr)(int arg); class UnmanagedB : public IB { IB_method_ptr mIB_method_ptr; public: void setMethodHandler( IB_method_ptr ptr); virtual int method( int arg); //... / }; void UnmanagedB ::setMethodHandler(IB_method_ptr ptr) { mIB_method_ptr = ptr; } int UnmanagedB ::method(int arg ) { return mIB_method_ptr( arg); } 

UnmanagedB methods simply call the callbacks that the managed class gives it. Here we are waiting for another trouble. Until someone has a pointer to UnmanagedB in the unmanaged code, we do not have the right to delete an instance of the class in the managed code that reacts to a callback call. The last part of the article will be devoted to solving this problem.

Functions for integration:

 UnmanagedB *UnmanagedB_createInstance() { return new UnmanagedB(); } void UnmanagedB_setMethodHandler(UnmanagedB *instance, IB_method_ptr ptr) { instance->setMethodHandler( ptr); } void UnmanagedB_deleteInstance(UnmanagedB *instance) { delete instance; } 

And here is the interface representation in managed code:

 public abstract class AB { private IntPtr mInstance; [DllImport("shim", CallingConvention = CallingConvention.Cdecl)] private static extern IntPtr UnmanagedB_createInstance(); [DllImport("shim", CallingConvention = CallingConvention.Cdecl)] private static extern IntPtr UnmanagedB_setMethodHandler( IntPtr instance, [MarshalAs(UnmanagedType.FunctionPtr)] MethodHandler ptr); [DllImport("shim", CallingConvention = CallingConvention.Cdecl)] private static extern void UnmanagedB_deleteInstance( IntPtr instance); [UnmanagedFunctionPointerAttribute( CallingConvention.Cdecl)] private delegate int MethodHandler( int arg); private int impl_method( int arg) { return method(arg); } public abstract int method(int arg); public AB() { mInstance = UnmanagedB_createInstance(); UnmanagedB_setMethodHandler(mInstance, impl_method); } ~AB() { UnmanagedB_deleteInstance(mInstance); } internal virtual IntPtr getUnmanaged() { return mInstance; } } 

Each interface method has a pair:

1. The public abstract method that we will override
2. The "caller" abstract method (private method with the prefix impl). It may seem that it does not make sense, but it is not. This method may contain additional transformations of the arguments and execution results. It can also contain additional logic for passing exceptions (as you may have guessed, it is not possible to simply transfer an exception from the environment to the environment, exceptions must also be integrated)

That's all. Now we can inherit the class AB and override its method method. If we need to pass a successor to unmanaged code, we will give mInstance instead, which will call the overridden method via a pointer to a function / delegate. If we get a pointer to the IB interface from the unmanaged environment, it will need to be represented as an AB instance in a managed environment. To do this, we implement an AB "default" heir:

 internal sealed class BImpl : AB { [DllImport("shim", CallingConvention = CallingConvention.Cdecl)] private static extern int BImpl_method( IntPtr instance, int arg); private IntPtr mInstance; internal BImpl( IntPtr instance) { Debug.Assert(instance != IntPtr.Zero); mInstance = instance; } public override int method(int arg) { return BImpl_method(mInstance, arg); } internal override IntPtr getUnmanaged() { return mInstance; } } 

Functions for integration:

 int BImpl_method(IB *instance , int arg ) { instance->method( arg); } 

By and large this is the same class integration without support for inheritance, described above . It's not hard to notice that when creating an instance of BImpl, we also create an instance of UnmanagedB and make unnecessary callback bindings. If desired, this can be avoided, but these are subtleties; we will not describe them here.

Integration of classes with inheritance support

The task is to integrate the class and provide the ability to override its methods. We will give the pointer to the class to unmanaged, so it is necessary to provide the class with callbacks in order to be able to call the overridden methods.

Consider a C class that has an implementation in unmanaged code:

 class C { public: virtual int method(int arg); virtual ~C() {}; }; 

To begin with, we pretend that this is an interface. We integrate it as well, as was done above :

Unmanaged heir for callbacks:

 typedef int (*_method_ptr )(int arg); class UnmanagedC : public cpp::C { _method_ptr m_method_ptr; public: void setMethodHandler( _method_ptr ptr); virtual int method( int arg); }; void UnmanagedC ::setMethodHandler(_method_ptr ptr) { m_method_ptr = ptr; } int UnmanagedC ::method(int arg ) { return m_method_ptr( arg); } 

Functions for integration:

 //...   createInstance  deleteInstance void UnmanagedC_setMethodHandler(UnmanagedC *instance , _method_ptr ptr ) { instance->setMethodHandler( ptr); } 

And the implementation in .Net:

 public class C { private IntPtr mHandlerInstance; [DllImport("shim", CallingConvention = CallingConvention.Cdecl)] private static extern IntPtr UnmanagedC_setMethodHandler( IntPtr instance, [MarshalAs(UnmanagedType.FunctionPtr)] MethodHandler ptr); [UnmanagedFunctionPointerAttribute( CallingConvention.Cdecl)] private delegate int MethodHandler( int arg); //...     /   private int impl_method( int arg) { return method(arg); } public virtual int method(int arg) { throw new NotImplementedException(); } public C() { mHandlerInstance = UnmanagedC_createInstance(); UnmanagedC_setMethodHandler(mHandlerInstance, impl_method); } ~C() { UnmanagedC_deleteInstance(mHandlerInstance); } internal IntPtr getUnmanaged() { return mHandlerInstance; } } 

So, we can override the C.method method and it will be correctly called from the unmanaged environment. But we did not provide a default implementation call. Here the code from the first part of the article will help us:
To call the default implementation, we need to integrate it. Also, for her work, we need a corresponding instance of the class, which will have to be created and deleted. We get the already familiar code:

 //...    createInstance  deleteInstance int C_method(C *instance, int arg) { return instance->method( arg); } 

Finish the .Net implementation:

 public class C { //... [DllImport("shim", CallingConvention = CallingConvention.Cdecl)] private static extern int C_method(IntPtr instance, int arg); public virtual int method(int arg) { return C_method(mInstance, arg); } public C() { mHandlerInstance = UnmanagedC_createInstance(); UnmanagedC_setMethodHandler(mHandlerInstance, impl_method); mInstance = C_createInstance(); } ~C() { UnmanagedC_deleteInstance(mHandlerInstance); C_deleteInstance(mInstance); } //... } 

Such a class can be safely applied in managed code, inherit, redefine its methods, and pass a pointer to it in an unmanaged environment. Even if we did not override any methods, we still pass the pointer to UnmanagedC. This is not very rational, given that unmanaged code will call unmanaged class C methods by translating calls through managed code. But such is the price for the possibility of overriding methods. In the example attached to the article, this case is demonstrated by calling the method method of class D. If you look at the callstack, you can see the following sequence:

Exceptions

Platform Invoke does not allow you to pass exceptions and to work around this problem, we intercept all exceptions before going from environment to environment, wrap the exception information in a special class and pass it. On that side, we generate an exception based on the information received.

We were lucky. Our C ++ module generates only exceptions of type ModuleException or its heirs. So it is enough for us to catch this exception in all methods in which it can be generated. To forward an exception object to the managed environment, we need to integrate the ModuleException class. In theory, the exception should contain a text message, but I don’t want to bother with the topic of marshaling the lines in this article, so the example would be “error codes”:

 public sealed class ModuleException : Exception { IntPtr mInstance; bool mDelete; //...  create/delete instance [DllImport("shim", CallingConvention = CallingConvention.Cdecl)] private static extern int ModuleException_getCode( IntPtr instance); public int Code { get { return ModuleException_getCode(mInstance); } } public ModuleException( int code) { mInstance = ModuleException_createInstance(code); mDelete = true; } internal ModuleException( IntPtr instance) { Debug.Assert(instance != IntPtr.Zero); mInstance = instance; mDelete = false; } ~ModuleException() { if (mDelete) ModuleException_deleteInstance(mInstance); } //...  getUnmanaged } 

Now suppose that the C :: method method can throw a ModuleException exception. Rewrite the class with exception support:

 //    ,     typedef int (*_method_ptr )(int arg, ModuleException **error); int UnmanagedC ::method(int arg ) { ModuleException *error = nullptr; int result = m_method_ptr( arg, &error); if (error != nullptr) { int code = error->getCode(); //...    error      throw ModuleException(code); } return result; } 

 int C_method(C *instance, int arg, ModuleException ** error) { try { return instance->method( arg); } catch ( ModuleException& ex) { *error = new ModuleException(ex.getCode()); return 0; } } 

 public class C { //... [DllImport("shim", CallingConvention = CallingConvention.Cdecl)] private static extern int C_method(IntPtr instance, int arg, ref IntPtr error); [UnmanagedFunctionPointerAttribute( CallingConvention.Cdecl)] private delegate int MethodHandler( int arg, ref IntPtr error); private int impl_method( int arg, ref IntPtr error) { try { return method(arg); } catch (ModuleException ex) { error = ex.getUnmanaged(); return 0; } } public virtual int method(int arg) { IntPtr error = IntPtr.Zero; int result = C_method(mInstance, arg, ref error); if (error != IntPtr.Zero) throw ModuleException(error); return result; } //... } 

Here we are also in trouble with memory management. In the impl_method method, we pass a pointer to an error, but Garbage Collector can remove it before it is processed in unmanaged code. It's time to deal with this problem!

Garbage collector against callbacks

Here I must say that we are more or less lucky. All classes and interfaces of the integrated module inherit from a certain IObject interface containing the addRef and release methods. We knew that everywhere in the module, passing the pointer made the call to addRef. And whenever the need for a pointer disappeared, release was called. Due to this approach, we could easily track whether the pointer needed by the unmanaged module or whether the callbacks can already be deleted.

To avoid deleting the managed objects used in the unmanaged environment, we need a manager of these objects. It will take the addRef and release calls from the unmanaged code and release the managed objects when they are no longer needed.

The addRef and release calls will be forwarded from unmanaged code to managed, so the first thing we need is a class that will provide such a forwarding:

 typedef long (*UnmanagedObjectManager_remove )(void * instance); typedef void (*UnmanagedObjectManager_add )(void * instance); class UnmanagedObjectManager { static UnmanagedObjectManager mInstance; UnmanagedObjectManager_remove mRemove; UnmanagedObjectManager_add mAdd; public: static void add( void *instance); static long remove( void *instance); static void setAdd( UnmanagedObjectManager_add ptr); static void setRemove( UnmanagedObjectManager_remove ptr); }; UnmanagedObjectManager UnmanagedObjectManager ::mInstance; void UnmanagedObjectManager ::add(void * instance ) { if (mInstance.mAdd == nullptr) return; mInstance.mAdd( instance); } long UnmanagedObjectManager ::remove(void * instance ) { if (mInstance.mRemove == nullptr) return 0; return mInstance.mRemove( instance); } void UnmanagedObjectManager ::setAdd(UnmanagedObjectManager_add ptr ) { mInstance.mAdd = ptr; } void UnmanagedObjectManager ::setRemove(UnmanagedObjectManager_remove ptr) { mInstance.mRemove = ptr; } 

The second thing we need to do is redefine the addRef and release of the IObject interface so that they change the values ​​of our manager’s counter stored in the managed code:

 template <typename T > class TObjectManagerObjectImpl : public T { mutable bool mManagedObjectReleased; public: TObjectManagerObjectImpl() : mManagedObjectReleased( false) { } virtual ~TObjectManagerObjectImpl() { UnmanagedObjectManager::remove(getInstance()); } void *getInstance() const { return ( void *) this; } virtual void addRef() const { UnmanagedObjectManager::add(getInstance()); } virtual bool release() const { long result = UnmanagedObjectManager::remove(getInstance()); if (result == 0) if (mManagedObjectReleased) delete this; return result == 0; } void resetManagedObject() const { mManagedObjectReleased = true; } }; 

Now the UnmanagedB and UnmanagedC classes need to be inherited from the TObjectManagerObjectImpl class. Consider the example of UnmanagedC:

 class UnmanagedC : public TObjectManagerObjectImpl <C> { _method_ptr m_method_ptr; public: UnmanagedC(); void setMethodHandler( _method_ptr ptr); virtual int method( int arg); virtual ~UnmanagedC(); }; 

Class C implements the IObject interface, but now the addRef and release methods are redefined by the TObjectManagerObjectImpl class, so the object manager in a managed environment will be responsible for counting the number of pointers.
It's time to take a look at the code of the manager himself:

 internal static class ObjectManager { //...  ,  , .  private static AddHandler mAddHandler; private static RemoveHandler mRemoveHandler; private class Holder { internal int count; internal Object ptr; } private static Dictionary< IntPtr, Holder> mObjectMap; private static long removeImpl( IntPtr instance) { return remove(instance); } private static void addImpl(IntPtr instance) { add(instance); } static ObjectManager() { mAddHandler = new AddHandler(addImpl); UnmanagedObjectManager_setAdd(mAddHandler); mRemoveHandler = new RemoveHandler(removeImpl); UnmanagedObjectManager_setRemove(mRemoveHandler); mObjectMap = new Dictionary<IntPtr , Holder >(); } internal static void add(IntPtr instance, Object ptr = null) { Holder holder; if (!mObjectMap.TryGetValue(instance, out holder)) { holder = new Holder(); holder.count = 1; holder.ptr = ptr; mObjectMap.Add(instance, holder); } else { if (holder.ptr == null && ptr != null) holder.ptr = ptr; holder.count++; } } internal static long remove(IntPtr instance) { long result = 0; Holder holder; if (mObjectMap.TryGetValue(instance, out holder)) { holder.count--; if (holder.count == 0) mObjectMap.Remove(instance); result = holder.count; } return result; } } 

Now we have an object manager. Before transferring the instance of a managed object to the unmanaged environment, we must add it to the manager. So the getUnmanaged method for classes AB and C needs to be changed. I will give the code for class C:

 internal IntPtr getUnmanaged() { ObjectManager.add(mHandlerInstance, this); return mHandlerInstance; } 

Now we can be sure that callbacks will work as long as necessary.

Given the specifics of the module, you will need to rewrite the classes, replacing all calls to ClassName_deleteInstance with calls to IObject :: release, and also remember to do IObject :: addRef where it is needed. In particular, this will avoid the premature removal of ModuleException, even if the garbage collector removes the managed wrapper, the unmanaged instance being an IObject descendant will not be deleted until the unmanaged module handles the error and causes IObject_release for it.

Conclusion

In fact, while we were engaged in the integration of the module, we experienced a huge amount of emotions, learned a lot of obscene words and learned how to sleep while standing. Perhaps we should want this article to be useful to someone, but God forbid. Of course, addressing memory management, inheritance, and passing exceptions was fun. But we did not integrate three classes and it was not in them by one method. It was a test of endurance.

If, nevertheless, you encounter such a task, then here is a tip for you: love Sublime Text, regular expressions and snippets. This small set saved us from alcoholism.

PS A working library integration example is available at github.com/simbirsoft-public/pinvoke_example