Network access to libraries and runtime-formation of function calls

I want to share a story from professional activities that can be deservedly put on a blog named crazydev :) This is a story about unusual decisions (those that I tried to describe in two words in the title), to which I was forced to come even more unusual restrictions and requirements.


And somehow, through the cleverly twisted *** y, it works ©

Formulation of the problem

A few years ago we were working on a system designed for the platform of our valiant Armed Forces . A significant part of the work proceeded in porting (with windows) already existing software. And here it turns out: an important component of the system (let's call it the sepulenium library ) provided by the partner simply does not have a linux version, and its developer is not in a hurry with the port, since no such arrangements. Swearing with his superiors, who made this mistake with the planned works, is meaningless - if, as a result, the sepulkaries stand, then this is an unclosed TK, and we, as performers, will be the first to blame.

Searching of decisions

Initially, there was a wall of text about iterated solutions, which included porting source codes on their own, and using a virtual machine with a Reactive Axis (in order not to use the proprietary software of a potential enemy ), and the inclusion of a sistemnik in the local network in the Mini-ITX form factor with the same ReactOS on board. And the intense search for a more or less stable version of wine (the system libraries are quite old, and it is impossible to update - it threatens to lose OS certification ).
On the version of emulation using Wine (which, as you know, is not an emulator at all), we stopped. It remained to think about how sepulcaria launched inside the server software process will get access to the separation algorithms, forced to be under the jurisdiction of the wine-process. Then an idea comes to my mind - to organize a network translator of accessing libraries .
')

Translator

In general, it looks like this:

(by the way, it seems that one of the variations of the Program as a Service model ( Soft as a service ) is obtained, but this is a topic for another story)
What seems to me interesting in such a scheme is the fact that clients can work both in different instances of the selected library and in the same one .
To implement there are two ways:
1) Teach the translator to work with the set of libraries that are needed now and then, in the case of replenishing this set, add its code each time (or connect via adapters, it does not matter), providing pairing with each new library.
2) To ensure the universality of the translator, making it really just a query translator , shifting the function of generating the necessary requests to new libraries on the client part.
Obviously, if I had gone the first way, I would have nothing to write to crazydev :)

Universality everywhere

A simple example of connecting a library and calling functions from it.

typedef double (*myfunc_type)(long, long); ... void *mylib; myfunc_type myfunc; double res; ... mylib = dlopen("mylib.dll", RTLD_LAZY); myfunc = dlsym(mylib, "my_func_name"); res = (*myfunc)(2, 4); 

As we can see, in order to access any function, we must first describe its type:
typedef double (*myfunc_type)(long, long);

And what to do when the types of functions to which we will refer are unknown at the design stage ?

That's right, we come to the aid of the good old assembler. What is essentially a function call? Putting arguments on the stack, passing control to the address, then getting the result, and, depending on the calling convention, clearing the stack.
Here is a piece of code (in Delphi) that was just performing similar operations in my translator (I made additional comments to eliminate unclear moments):

 //    , //    TDLL_Function //  ,    (  ) function TDLL_Function.Execute(): TByteAr; var i: Integer; //   len : Integer; //      B1 : Byte; //    B2 : Word; //    B4 : Cardinal; //  4-  B8 : Double; //   8-  StackPos : Integer; //     begin //ParamBytes   ,      , //      len:=Length(ParamBytes); asm //    (  esp) mov StackPos, esp end; //ParamType   ,     : //TParamType = (ptOne=1, ptTwo=2, ptFour=4, ptEight=8, ptVoid=0, ptPointer=-1); //  ,       //   , ..         for i := Length(ParamType)-1 downto 0 do begin case ParamType[i] of //      ptOne:begin dec(len,1); //  ParamBytes    ( ),    Move(ParamBytes[len],B1,1); asm //  ,     MOVSX EAX,B1 PUSH EAX end; end; ptTwo:begin //     dec(len,2); Move(ParamBytes[len],B2,2); asm MOVSX EAX,B2 PUSH EAX end; end; ptFour:begin //     dec(len,4); Move(ParamBytes[len],B4,4); asm MOV EAX,B4 PUSH EAX end; end; //        ,    //64-  (8   )    ptPointer:begin dec(len,4); Move(ParamBytes[len],B4,4); asm MOV EAX,B4 PUSH EAX end; end; ptEight:begin dec(len,8); Move(ParamBytes[len],B8,8); asm //        PUSH DWORD PTR [B8]+$04 PUSH DWORD PTR B8 end; end; ptVoid: begin end; end; end; // B4       case fCallingConv of //       ccStdcall: begin TStdCall(Proc)(); //Proc -     asm //    MOV B4,EAX end; end; ccCdecl:begin TCdeclCall(Proc)(); //   Cdecl          asm MOV B4,EAX mov esp, StackPos; end; end; end; //   ,     case ResultType of //     ptOne:begin //   Result      B4 SetLength(Result,1); Move(Byte(B4),Result[0],1); end; ptTwo:begin SetLength(Result,2); Move(Word(B4),Result[0],2); end; ptFour:begin SetLength(Result,4); Move(B4,Result[0],4); end; ptPointer:begin SetLength(Result,4); Move(B4,Result[0],4); end; ptEight:begin //    ,  B8    asm FSTP B8 end; SetLength(Result,8); Move(B8,Result[0],8); end; ptVoid:begin SetLength(Result,0); end; end; end; 

Please do not kick with the code, it definitely requires optimization.
About implementation features:
1) Only made stdcall and cdecl conventions available
2) There are no guarantees that assembly inserts will also work on architectures other than those for which this was done.
1) There is no support for 64-bit code, although in general, some ways to ensure I laid
2) If a pointer is passed to a function, for example, to an array, then the client had to forward the entire array so that the translator would deploy it and send a pointer to it to the function. If this array had to be returned, then its return was organized in the same way.

In general, the protocol for client and translator communication turned out to be quite complicated and confusing, and I don’t know if it makes sense to describe it. I can only say that it was binary :)

The general sequence of actions for the call looked like this:
1) The client connects to the broadcaster
2) The client sends the name of the library to which it wants to connect.
3) The client sends a description of the library function that it wants to call.
4) The client packs and sends the parameters to call the library function.
5) The translator expands the parameters in accordance with the received description of the function.
6) The translator calls the above Execute ()
7) The translator packs the necessary results of work (as described in the function) and sends them to the client.

Here is such a crazydev :) In defense of my crafts, I’ll say that in this form it worked smoothly for a year, and taking advantage of this versatility, we managed to reduce the time for porting some other libraries as well. And a year later, during the planned update, the ported version of the sepulic libraries had already ripened, and everything ended well.

UPD. Added the “crutches” tag so that there are no misunderstandings with the self-identification of the solutions described.