Developing Modules for Limbo C (Part 1)

Limbo modules written in C are also sometimes called OS Inferno drivers. they are built into the OS kernel. The need for such modules is usually caused either by a desire to add to Limbo the functionality missing in Inferno (connect existing 3rd-party C / C ++ libraries, give access to specific host OS-specific syscalls) or desire to squeeze the highest possible performance (according to my observations speeds between Limbo with JIT and C enabled are about 1.3-1.5 times, but sometimes this can be critical).

Content

• We build the module into the kernel
  
  • How it works
• Functions: receiving parameters and returning results
  
  • Numbers
    
    • How it works
  • Strings
    
    • How it works
  • Exceptions
  • We return the link
• Dis blocking

We build the module into the kernel

Unfortunately, while in Inferno there is no possibility to dynamically load modules implemented in C, so you have to build it directly into the OS kernel. (This makes development a bit more difficult, because after each change, Inferno has to be rebuilt. Fortunately, the partial reassembly takes about 10 seconds.)

To embed your module, you need to modify several files: libinterp/mkfile , module/runt.m , emu/Linux/emu and emu/Linux/emu-g . And since each new module tries to be built into the same files in the same places, and the user may want to add several such modules, moreover, in an order not known in advance, the standard patch command will not be able to make the necessary changes. She will add one or two modules, but with the following she will have a problem. the editable location in these files will start to differ too much from what she expected to see.

To solve this problem, I sketched a Perl script - in most cases it is enough to change the name of the added module in the line

 my $MODNAME = 'CJSON'; 

and it will make the necessary changes to all of the above files by embedding your module in the OS Inferno kernel. In more complex cases, for example, when it is necessary to connect additional C / C ++ libraries to Inferno, this script will have to be modified to suit your needs (an example of such a modification for connecting the C ++ re2 library can be seen in the Re2 module ). The script can be run with the -R parameter to rollback the changes made.
')
So, download the script, put it into $INFERNO_ROOT , rename it to $INFERNO_ROOT , change the module name to “Example” in it, and launch it. Now the (non-existent for now) Module Example is connected to the kernel, it remains to create it and rebuild Inferno with it.

First, create two files:

1. module/example.m
   

    Example: module { PATH: con "$Example"; }; 

2. libinterp/example.c
   

    #include <lib9.h> #include <isa.h> #include <interp.h> #include "runt.h" #include "examplemod.h" void examplemodinit(void) { builtinmod("$Example", Examplemodtab, Examplemodlen); } 

And run OS Inferno reassembly:

 $ (cd libinterp/; mk nuke) $ rm Linux/386/bin/emu # work around "text file busy" error $ mk install 

Now we can write a program in Limbo, which successfully loads our, while not doing anything useful, module:

• testexample.b
  

   implement TestExample; include "sys.m"; include "draw.m"; include "example.m"; TestExample: module { init: fn(nil: ref Draw->Context, nil: list of string); }; init(nil: ref Draw->Context, nil: list of string) { sys := load Sys Sys->PATH; example := load Example Example->PATH; if(example == nil) sys->print("fail to load Example: %r\n"); else sys->print("Example module loaded\n"); } 

  
  

And run:

 $ emu ; limbo testexample.b ; testexample Example module loaded ; 

How it works

During the build process, the module/example.m file is analyzed, and the necessary C-shny structures describing this module are generated — in a separate libinterp/examplemod.h — and its entire public interface (constants, adt-shki, functions) are added to the libinterp/runt.h file libinterp/runt.h containing information on all C-modules. These two .h files are already connected to our libinterp/example.c .

Further, during the OS Inferno boot process, the function examplemodinit() will be called once, which should initialize the global data of our module (if any) and connect it (by calling builtinmod(…) ) to the Inferno core. The call to builtinmod() establishes a connection between our module and the pseudo-path to it $ Example specified in the PATH constant used from Limbo when this module is loaded with the load command.

Functions: receiving parameters and returning results

Numbers

Let's start with simple data types in order not to complicate the example of working with links.

• module/example.m
  

   Example: module { ... increment: fn(i: int): int; }; 

• libinterp/example.c
  

   ... void Example_increment(void *fp) { F_Example_increment *f; int i; f = fp; i = f->i; *f->ret = i + 1; } 

Reassemble Inferno.

• testexample.b
  

   ... init(nil: ref Draw->Context, nil: list of string) { ... sys->print("increment(5) = %d\n", example->increment(5)); } 

Do not forget to restart emu before running our example, since
the currently running emu does not contain a modified C-module.

 $ emu ; limbo testexample.b ; testexample Example module loaded increment(5) = 6 ; 

How it works

When building, for the increment() function found in module/example.m , a description of this function, its parameters and return values ​​was automatically added to the libinterp/runt.h file:

 void Example_increment(void*); typedef struct F_Example_increment F_Example_increment; struct F_Example_increment { WORD regs[NREG-1]; WORD* ret; uchar temps[12]; WORD i; }; 

I haven't figured out what regs ; temps are explicitly added for alignment; ret is a pointer to the return value; i is our parameter.

Strings

• module/example.m
  

   Example: module { ... say: fn(s: string); }; 

• libinterp/example.c
  

   ... void Example_say(void *fp) { F_Example_say *f; String *s; char *str; f = fp; s = f->s; str = string2c(s); print("%s\n", str); } 

• testexample.b
  

   ... init(nil: ref Draw->Context, nil: list of string) { ... example->say("Hello!"); } 

We collect, restart, check:

 $ emu ; limbo testexample.b ; testexample Example module loaded increment(5) = 6 Hello! ; 

How it works

This is what we got in libinterp/runt.h :

 void Example_say(void*); typedef struct F_Example_say F_Example_say; struct F_Example_say { WORD regs[NREG-1]; WORD noret; uchar temps[12]; String* s; }; 

With noret instead of ret everything is clear, the say() function returns nothing. The String* type is a C-implementation of Limbo strings. You can find the struct String in include/interp.h , functions for working with strings (such as used in our example, string2c() ) are in libinterp/string.c .

The work with other Limbo data types is implemented in the same way: via Array* , List* , etc. Not all structures have ready auxiliary functions as for working with strings, but you can find enough examples in the implementation of opcodes of the virtual machine libinterp/xec.c (for example, how to work with array slices).

Custom adt declared in module/example.m converted to the usual C-shny struct (and pick adt to union). Tuples are also converted to regular struct.

Most likely after modifying module/example.m you have to start the build (which fails by mistake) to libinterp/runt.h and see exactly which structures were created for your data and understand how to implement working with them in libinterp/example.c .

Exceptions

To generate an exception, just call the error() function. You can connect raise.h to return standard errors described in libinterp/raise.c or declare your own libinterp/example.c in libinterp/example.c in the same way.

Of course, if you allocated memory yourself through malloc() , then before calling error() you need to free this memory, otherwise there will be a leak. Objects allocated in a standard way via heap (like String* and Array* ) need not be released, anyway, they will be found and deleted by the garbage collector a little later. (In more detail about work of heap and the collector of garbage in part 2. )

We return the link

One implicit moment when the result is returned from a function is due to the fact that *f->ret physically points to the memory cell where the result of the function execution should be after its successful completion. Two consequences follow from this:

1. If you first put the result in *f->ret , and then decide that an error occurred and throw an exception, then something impossible will happen in terms of Limbo: the AND function will return the value AND cause an exception.
2. If the variable where the result of your function returns, already contains some value (which is also a reference, of course, because the type of this variable is the same as the value returned by your function), then you should free it from memory before How to rewrite this link with yours.

To demonstrate the first problem, let's modify our function.
increment() like this:

• libinterp/example.c
  

   ... void Example_increment(void *fp) { ... *f->ret = i + 1; error("some error"); } 

• testexample.b
  

   ... init(nil: ref Draw->Context, nil: list of string) { ... i := 0; { i = example->increment(5); } exception e { "*" => sys->print("catched: %s\n", e); } sys->print("i = %d\n", i); } 

 ; testexample ... catched: some error i = 6 ; 

To solve the second problem in C functions, before saving the return value in *f->ret you must release the current value. This is usually done either like this:

 destroy(*f->ret); *f->ret = new_value; 

either way ( H is the C-shny analogue of Limbo-vskogo nil ):

 void *tmp; ... tmp = *f->ret; *f->ret = H; destroy(tmp); ... *f->ret = new_value; 

As far as I understand, at the moment there is no difference between these options, but if Dis is rewritten to work simultaneously on several CPU / Core, then the second option will work correctly, and the first will not.

Dis blocking

Inferno uses the global Dis lock (probably similar to the well-known GIL in python). C-shnye functions are called with the lock set, because it is safe to work with Dis data structures (i.e., any values ​​and variables available from Limbo — including parameters and return values ​​of C-shny functions) is possible only with the lock set.

But if your function has to perform some long operation (for example, read / write or call a “heavy” function from an external library or perform some lengthy calculations), then you need to release() before this operation so that Dis continues to run in another thread in parallel with your function, and then again to acquire() (otherwise it will be impossible to return the result and return to the code that caused this function in Limbo). An example can be found in the sys->read() implementation in the emu/port/inferno.c :

 void Sys_read(void *fp) { ... release(); *f->ret = kread(fdchk(f->fd), f->buf->data, n); acquire(); } 

Part 2.