PHP extension and Kotlin Native. Part three, probably final

The first part describes quite basic things about toolkit customization and general concepts.

The second part is about, so to speak, the first approach to the projectile, ideas, plans, plans.

In this article there will be a little more hardcore about interop C and K / N, a lot of macros, pain, hopelessness and the "rays of good." Of course there will be a chapter with a story about achievements (you cannot praise yourself ... and as a bonus, a story about an epic fakape.

Disclaimer: all of the following is considered in the context of writing a library for PHP.
')

Chapter one. Interop naive

How to use the K / N functions in C is described in the first part of the cycle. Accordingly, here I will explain how to use C functions in K / N.

Official documentation is rather stingy and laconic, however, for simple projects, it is quite enough.

In short, you need to create a special file with the .def extension and specify the necessary header files in it.

headers = php.h 

Then feed it to a program called cinterop .

 # cinterop -def php.def -o php 

At the exit, you will receive a library of libphp.klib , containing llvm bitcode and various meta-information.

Then you can safely use the functions and macros described in the header file ( #define ), without forgetting to connect the library at the compilation stage.

 # kotlinc -opt -produce static ${SOURCES} -l libphp.klib -o myLib 

But there is a nuance. And not one.

In the form as described above, the library will not gather

Why? Because php.h contains the following lines:

 #include "php_version.h" #include "zend.h" #include "zend_sort.h" #include "php_compat.h" #include "zend_API.h" 

Here it should be noted that llvm does the compilation of the library, but it has the -I key, and cinterop has the -copt key. Well, you understand. As a result, to compile php.h is enough of such a command.

 # cinterop -def my.def -o myLib -I${PHP_LIB_ROOT} -copt -I${PHP_LIB_ROOT} \ -copt -I${PHP_LIB_ROOT}/main \ -copt -I${PHP_LIB_ROOT}/Zend \ -copt -I${PHP_LIB_ROOT}/TSRM 

Macros. I love you and hate you! No, I just hate it.

All you need to know about #define in the interop C part> K / N is

Everyth of the Kotlin property. Other macros are not supported.

And then we remember that the PHP extension is a macro on a macro and chases the macro and try not to cry.

But it is not all that bad. To work around this situation, K / N developers provided a coil of blue electrical tape to attach custom declarations to the def file. It looks like this (for example, take the macro Z_TYPE_P )

 headers = php.h --- static inline zend_uchar __zp_get_arg_type(zval *z_value) { return Z_TYPE_P(z_value); } 

Now in the K / N code you can use the function __zp_get_arg_type

Chapter two PHP INI-settings or macro with a sub-subdirect.

This is the “ray of good” in the direction of the PHP source.

There are 4 macros for extracting settings:

 INI_INT(val) INI_FLT(val) INI_STR(val) INI_BOOL(val) 

Where val is a string with the name of the setting.

And now let's take a look at the example of INI_STR , and see how this macro is defined.

 #define INI_STR(name) zend_ini_string_ex((name), sizeof(name)-1, 0, NULL) 

Already noticed his "fatal flaw"?

If not, then I'll tell you - this is the sizeof function. When you use a macro directly, then everything is fine:

 php_printf("The value is : %s", INI_STR("my.ini")); 

When you use it through a proxy function from a .def file, the carriage turns into a pumpkin, and sizeof (name) returns the size of the pointer. Checkmate Kotlin Native.

Bypass options, in fact, only two.

1. To use not macros, but functions to which they are attached.
2. Hardcode function wrappers for each required setting.

The first option is better for everyone than the second, except for one thing - no one will guarantee that the macro declaration will not change. Therefore, for my project, I, with a deep sense of dissatisfaction, chose the second option.

Chapter Three Debag? What debag?

Act 1 - interop.

At one point, after winding up the blue tape to the def-file of the 20 next proxy functions, I received a remarkable error.

 Exception in thread "main" java.lang.Error: /tmp/tmp399964332777824085.c:103:38: error: too many arguments to function call, expected 2, have 3 at org.jetbrains.kotlin.native.interop.indexer.UtilsKt.ensureNoCompileErrors(Utils.kt:137) at org.jetbrains.kotlin.native.interop.indexer.IndexerKt.indexDeclarations(Indexer.kt:902) at org.jetbrains.kotlin.native.interop.indexer.IndexerKt.buildNativeIndexImpl(Indexer.kt:892) at org.jetbrains.kotlin.native.interop.indexer.NativeIndexKt.buildNativeIndex(NativeIndex.kt:56) at org.jetbrains.kotlin.native.interop.gen.jvm.MainKt.processCLib(main.kt:283) at org.jetbrains.kotlin.native.interop.gen.jvm.MainKt.interop(main.kt:38) at org.jetbrains.kotlin.cli.utilities.InteropCompilerKt.invokeInterop(InteropCompiler.kt:100) at org.jetbrains.kotlin.cli.utilities.MainKt.main(main.kt:29) 

Comment the half, rebuild, if the commentary repeated the half of the rest, we collect ... And considering that the compile process of the headers is quite long ... (yes, it seemed so faster than to climb the top ten source files and meticulously, with a magnifying glass, to verify).

The second "ray of good" goes in the direction of JetBrains.

Act 2 - runtime.

I receive in segmentation fault . Well, ok, it happens. I climb into the debugger. Ummm ... STA?

 Program received signal SIGSEGV, Segmentation fault. kfun:kotlinx.cinterop.toKString@kotlinx.cinterop.CPointer<kotlinx.cinterop.ByteVarOf<kotlin.Byte>>.()kotlin.String () at /opt/buildAgent/work/4d622a065c544371/Interop/Runtime/src/main/kotlin/kotlinx/cinterop/Utils.kt:402 402 /opt/buildAgent/work/4d622a065c544371/Interop/Runtime/src/main/kotlin/kotlinx/cinterop/Utils.kt: No such file or directory. 

Chapter Four I poured tea into your tea so that you can drink tea while you drink tea.

Here it is necessary to tell how that figovin that I make works.

You write DSL describing the future PHP extension, write K / N code with the implementation of functions, classes and methods, then run make and, miraculously, get a ready-made library that can be connected to PHP.

The assembly can be divided into 4 stages:

1. Creating a layer between C and K / N (the same cinterop )
2. C-code extension generation
3. Compiling library with logic
4. Compiling target library

The task is to add the ability to create instances of a PHP class in the K / N code. For example, so that a class can have a getInstance() method defined. And you want to make it so that it is convenient to use.

In C, this problem is solved once or twice.

 zval *obj = malloc(sizeof(zval)); object_init_ex(obj, myClass); 

It would seem simple - take it and carry it to K / N, but here’s myClass ...

But myClass is a global variable of the type zend_class_entry* , declared in the C code of the project and with an unknown name in advance.

Watch your hands. It is necessary to compile the library from K / N code, which will have a function, which needs to have access to myClass , which is defined in the generated, but not compiled C code, from which this function will be called later.

In the end, the implementation of this functionality led to the addition of two new artifacts: .h and .kt at the stage of code generation, complication of the cinterop stage and an epic backup, which I will tell you at the very end.

Chapter Five What's in a name?

Tale of why:

 enum class ArgumentType { PHP_STRING, PHP_LONG, PHP_DOUBLE, PHP_NULL, ... } 

better than:

 enum class ArgumentType { STRING, LONG, DOUBLE, NULL, ... } 

Yes, there is even no need to explain. This is what ArgumentType.NULL turns into in the header file of the Kotlin library:

 struct { extension_kt_kref_php_extension_dsl_ArgumentType (*get)(); /* enum entry for NULL. */ } NULL; 

And that's how gcc responds to it.

 /root/simpleExtension/phpmodule/extension_kt_api.h:113:17: error: expected identifier or '(' before 'void' } NULL; ^ 

A curtain! Watch out for names.

The last chapter. Do not praise yourself - no one praises.

By and large, I achieved my goals. Immersed in the topic, “framework” for writing PHP extensions on Kotlin Native, in general, is ready. It remains to add some, not the most critical, functionality and polish.

The project itself and, I hope, good documentation for it, can be viewed on the githaba .

What can I say about K / N? Only good. It is a pleasure to write on it, but small shoals and roughness can be attributed to the fact that he has not even got out of the cradle :)

Chapter last. Rays of good, without quotes.

But now, absolutely seriously and with deep respect, I want to thank the guys from JetBrains and the residents of the Kotlin Native slack channel. You are super!

And a special thank you to Nikolai Igotti .

Bonus Epic fakap.

The context is described in chapter four.

Actually, when everything was added to the state in which it was compiled without errors, a problem arose - during testing, PHP was opened to me from a completely unfamiliar side.

 # php -dextension=./phpmodule/modules/extension.so -r "var_dump(ExampleClass::getInstance());" *RECURSION* # 

"Figase!" - I thought, I got into the PHP source and found this piece.

 case IS_OBJECT: if (Z_IS_RECURSIVE_P(struc)) { PUTS("*RECURSION*\n"); return; } 

Adding debugging:

 printf("%u", Z_IS_RECURSIVE_P(struc)) 

Led to:

 undefined symbol: Z_IS_RECURSIVE_P in Unknown on line 0 

“Damn!” I thought again.

At that moment, when I guessed to look at the php.h (7.1.8) actually used on the linux host, and not the one that I pulled from the master brunch (7.3.x) from the github, the day passed. Straight ashamed.

But as it turned out, it was not a reel.

The correct recursion check code, at all stages of the life of the object under my control, reported that everything is OK and should work. And this means that you should carefully look at the places that I do not control. That was exactly one - in which my object is returned to the function var_dump

 RETURN_OBJ( example_symbols()->kotlin.root.php.extension.proxy.objectToZval( example_symbols()->kotlin.root.exampleclass.getInstance(/* */) ) ) 

Open the macro RETURN_OBJ to the end. Remove from the nervous and pregnant monitors!

 1) RETURN_OBJ(r) 2) { RETVAL_OBJ(r); return; } 3) { ZVAL_OBJ(return_value, r); return; } 4) { do { zval *__z = (return_value); Z_OBJ_P(__z) = (r); Z_TYPE_INFO_P(__z) = IS_OBJECT_EX; } while (0); return; } 5) { do { zval *__z = (return_value); Z_OBJ(*(__z)) = (r); Z_TYPE_INFO(*(__z)) = (IS_OBJECT | (IS_TYPE_REFCOUNTED << Z_TYPE_FLAGS_SHIFT)); } while (0); return; } 6) { do { zval *__z = (return_value); (*(__z)).value.obj = (r); (*(__z)).u1.type_info = (8 | ((1<<0) << 8)); } while (0); return; } 

This is where I felt ashamed the second time. I, completely in the blue eye, shoved zval* where waited for zend_object* and spent almost two days searching for an error.

Thank you for your attention, all Kotlin! :)

Ps. If there is a kind soul who subtracts my clumsy English and corrects the documentation - there will be no limit to my gratitude.