Useful code implementation

The article describes how to build a bridge between unmanaged and managed code using the example of the mathematical package Mathcad. The picture shows an example of how the one chipmunk is going to process his image by means of a mathematical package. For this, he “used” a user-defined function written on VB.Net, in which the ability to connect to a webcam and create a snapshot is implemented. The result of the function is immediately available in the working document.

Sources

For impatient people who want to understand everything at once, having run the code diagonally, I specify the storage: NetEFI . You can also find test user libraries in three languages: c #, vb.net and c ++ / cli (VS2012, .Net 2.0, x86-32). So far only 32-bit implementation is available.

Prehistory

In the mathematical program Mathcad there is the possibility of connecting third-party libraries. This interface is called User EFI and was developed more than 10 years ago. Since then, it has not changed at all, although Mathcad himself has changed beyond recognition. There was a time when this interface was thrown out of the package, but old users have requested it back and in new versions of Mathcad Prime this rare interface is once again alive.
')
There is quite a clear guide to creating custom libraries, I brought it at the end of the article. In short, the process looks like this. We create a regular dll, where at the entry point, i.e. when loading it, register our functions. At the same time, in the function descriptor we specify its address for the subsequent call from Mathcad directly. In addition, you can still register one table with error messages. The result returned by the user function in the event of an error can be used to select messages from this table. Here, in general, and the whole kitchen.

The function descriptor looks like this:


The problem is that today it would be much more convenient to write our functions if we did it in .net languages. But the direct way to do this is through the use of C ++ / CLI. The option of “wrapping” each user-defined function through an adapter to C ++ / CLI or marshaling structures, I think, can be dismissed immediately as impractical and requiring nontrivial knowledge from the user of the mathematical program. I want to offer a universal "wrapper", which I called .Net User EFI.

The question arises as to how to create a universal function that could be registered instead of all the functions of all connected assemblies, but at the same time it would have all the necessary information at the entry point to call a specific function from a specific assembly. The reseller library in which such a function is located should automatically work with any number of connected assemblies and functions in them.

To implement such universality there is one significant problem. Mathcad requires you to specify the address of the called function, the prototype itself is declared as having a variable number of parameters. It turns out that at the entry point of the universal function, the stack with parameters will have a different size and there is no possibility to pass this information when the function is called by standard means, since it is determined by the compiled code itself. In the structure above, only the address itself acts as a parameter by which we could distinguish the calling of one function from another.

And here our thought should come to one known solution, which is called code injection. On Habré, more than once they wrote about it, but there are not so many practical useful examples of using such equipment. In a sense, we will also intercept function calls from dll, everything will look a bit more specific, but much easier.

Idea

So, what we will inject, introduce, where and why. Once again, clarify the situation. We want to write a universal function that will process all calls uniformly and distribute them depending on the type of the function being called. Mathcad should not “suspect” anything, and we should get additional information from somewhere at the entry point of the universal function about the call parameters.

The solution will be to dynamically generate the code at the address that we register in Mathcad. We will reserve a lot of space in memory for dynamic code. This code will carry out auxiliary work on the transfer of parameters of the universal function. First, I’ll say that two parameters are enough for us, this is the number of the assembly in the array of loaded assemblies and the number of the function from the assembly. There are two ways to pass parameters: global variables and the stack. I chose the first option, because disrupting the stack balance (in which the parameters are located) is easy, but I think it will be difficult to restore it in our case.

I forgot to mention that there are only three user function types and all of them are passed by pointer: MCSTRING, COMPLEXSCALAR and COMPLEXARRAY. The maximum number is also limited to 10 pieces. This simplifies the implementation of parsing parameters in a universal function.

Implementation

Now we are morally ready to sort out the specific sequence of events that should occur at the stage of implementation and after it.

Step 1 . The user creates a .net class that implements the IFunction interface, which contains the necessary information about the function. Compiles it into an assembly and copies it to the userefi folder. Also in this folder should be an intermediary assembly, we will call it netefi.

Step 2 . When loading Mathcad, the netefi mediation build is perceived as a user library. It searches for all .net assemblies in the current folder and searches the functions in them for the implementation of the IFunction interface.

Step 3 . netefi stores information about assemblies and functions in them in internal arrays, while in order to define a function, you need two numbers: the assembly index and the function index in it.

Step 4 . netefi enumerates all the functions and registers them in Mathcad in a standard way, but in the address field of the FUNCTIONINFO structure we write a link to the dynamic code, the form of which is determined by the two indices from the previous step.

Here is the concrete implementation of the implementation method:


The InjectCode () method is called in a loop when registering functions in Mathcad. The global variables assemblyId and functionId are used to determine the type of the function at the time it is called. It works like this. Mathcad for each function receives a link to such a dynamic code. In this case, the assembly index recorded in the assemblyId, known at the time of loading (parameter k), the function index is written to functionId — the parameter n. Next comes the unconditional transition to CallbackFunction (), in which our universal function is called. This is done so that you can call managed code in UserFunction (). Unmanaged / managed directives will not allow this to be done in CallbackFunction ().

Note that the parameter of the universal function is a link to the CallbackFunction () stack, i.e. on the array of parameters (the return value is in the same place). The dynamic code does not spoil the stack for us, so after the completion of CallbackFunction () control will return to Mathcad. That's all the magic.

Step 5 . After registration is complete, you can call a custom function in a Mathcad document. The universal UserFunction () function can now restore the user function type by the global assemblyId and functionId parameters and disassemble the stack, knowing the number and type of parameters.

Step 6 . Each unmanaged type of the function parameter is replaced with an analog: MCSTRING with String, COMPLEXSCALAR with TComplex (I did not use Complex from .Net 4.0, so that there was no conflict) and COMPLEXARRAY with TComplex [,].

Step 7 . The implementation of the IFunction.NumericEvaluation method for the function is called. The returned result passes the reverse sequence of transformations and is given to Mathcad.

About implementation

I think that I explained this particular implementation method more or less clearly. As for the source of the project itself, it is worth mentioning briefly the environment and some details. Visual Studio 2012, C ++ / CLI, .Net Framework 2.0 is used as the development environment (the corresponding mode is set in the project properties). Since the dynamic code, generally speaking, depends on the digit capacity and I still don’t know exactly how to bring it to the 64-bit representation, all projects are set to be compiled for 32-bit machines. Although I was told that there would be few changes.

Using global variables is not good, but working in Mathcad does not involve the simultaneous calling of several functions. Everything is done there in order, one after the other.

In the mediation assembly, some more ideas are implemented that allow you to fully use the old interface in the new environment. This applies to error handling and you need to write about it separately. All main code is concentrated in a single Manager class (netefi.cpp). Analyzing test examples, you can understand how to work with the interface IFunction. All test examples in different languages ​​do the same thing, and are called almost the same.

Examples are tested in Mathcad 15 and Mathcad Prime 3.0. Since the User EFI interface itself has not changed for more than 10 years (and it is unlikely to change already), the described method can also be used in other versions of Mathcad, starting, probably, from version 11. In Mathcad Prime 3.0, custom functions were given a new name - Custom Functions, although the filling is the same.

Test cases

As mentioned above, you can find them here . But the article would not be complete if you did not show the specific form of .net user-defined functions for Mathcad.

Let's see how the echo function will look like for one string parameter.

Other

Although the main functionality is almost ready, there are some shortcomings. For example, it is desirable that the work of a universal function be performed in a separate thread. This is one of the first things to do. Interruption of work by calling isUserInterrupted is not reflected in any way in the new interface. All hope so far that Mathcad himself can interrupt the function. I will think about it and it echoes the work in the stream.

The current project only works on 32-bit systems. To add 64-bit configurations, you need to test the operation of dynamic code on 64-bit systems. There is no such possibility yet.

Working with COM inside the user function is now also apparently impossible. I faced this when I implemented the function to create a snapshot from a webcam. One of the standard options intended to use the interface to the Clipboard, and so it did not work, saying that the stream should be with the STAThreadAttribute attribute. Solved the problem through Graphics.CopyFromScreen. Also need to understand.

Downloading the missing assemblies is also not done reliably enough, because Assembly :: LoadFile () is used. If you use Assembly :: LoadFrom (), then Mathcad hangs in this place. There is also a problem with debugging mixed code. For some reason, she did not work for me as it should. I practically debugged the code in my mind, only logs were saved.

Maybe someone did something like this and could suggest good ideas to simplify my code. I will listen to all practical options. It would be great if someone made my project work under the studio debugger in mixed mode. While only breakpoints in unmanaged code are working. In the test examples, you can wander through the code, of course.

Links

0. How to generate and run native code dynamically?
1. Source codes and test examples on github .
2. Creating a User DLL (pdf).
3. .Net User EFI interface (a thread on the main PTC forum).
4. Sources and builds of the example with a webcam (in the same thread below).
5. Mathcad EFI plugin (another project of mine that performs the inverse function - calls unmanaged code from a managed one).