Calling Go functions from other languages

Since version 1.5, the Go compiler supports several build modes defined by the buildmode flag. They are also called Go ( Go Execution Modes ) execution modes. With their help, go tool can compile Go packages in several formats, including archives and shared libraries of Go (shared libraries), archives and libraries of shared use of C, and from version 1.8 - dynamic Go plugins.

In this article, we will look at compiling Go packages into C libraries. In this build mode, the compiler generates a standard binary object file (shared object) (.so), passing Go functions as a C-style API. We'll talk about how to create Go libraries that can be called from C, Python, Ruby, Node and Java.

All code is available on GitHub .

Go Code

First write the code on Go. Suppose we have an awesome library, the goal is to make it available to other languages. Before you compile the code into the library, you must meet four conditions:

• The package must be related to packages of type main . Then the compiler will compile it and all dependencies in one binary file of the common object.
• The source must import the pseudo-package "C" .
• To annotate functions that need to be made available to other languages, use the //export comment.
• An empty main function must be declared.

The following Go source exports four functions: Add , Cosine , Sort and Log . I must admit that the awesome library is not so impressive. However, its various function signatures (function signatures) will help us to study the possible consequences of type mapping.

The awesome.go file:

 package main import "C" import ( "fmt" "math" "sort" "sync" ) var count int var mtx sync.Mutex //export Add func Add(a, b int) int { return a + b } //export Cosine func Cosine(x float64) float64 { return math.Cos(x) } //export Sort func Sort(vals []int) { sort.Ints(vals) } //export Log func Log(msg string) int { mtx.Lock() defer mtx.Unlock() fmt.Println(msg) count++ return count } func main() {} 

The package is compiled with the -buildmode=c-shared flag to create a binary object file:

 go build -o awesome.so -buildmode=c-shared awesome.go 

The compiler creates the awesome.h header C-file and the awesome.so object awesome.so :

 -rw-rw-r — 1362 Feb 11 07:59 awesome.h -rw-rw-r — 1997880 Feb 11 07:59 awesome.so 

Please note that the .so file size is about 2 MB. Quite a lot for such a small library. The fact is that the entire Runtime mechanic Go and the dependent packages are pushed into this file.

Header file

It defines C-types, which, using cgo semantics, are mapped to compatible Go-types.

 /* Created by “go tool cgo” — DO NOT EDIT. */ ... typedef long long GoInt64; typedef unsigned long long GoUint64; typedef GoInt64 GoInt; typedef double GoFloat64; ... typedef struct { const char *p; GoInt n; } GoString; typedef struct { void *data; GoInt len; GoInt cap; } GoSlice; ... #endif ... extern GoInt Add(GoInt p0, GoInt p1); extern GoFloat64 Cosine(GoFloat64 p0); extern void Sort(GoSlice p0); extern GoInt Log(GoString p0); ... 

Shared object file

This is a 64-bit binary ELF file of the shared object. You can verify its contents with the file command.

 $> file awesome.so awesome.so: ELF 64-bit LSB shared object, x86–64, version 1 (SYSV), dynamically linked, BuildID[sha1]=1fcf29a2779a335371f17219fffbdc47b2ed378a, not stripped 

Using the nm and grep commands, you can verify that our Go functions are exported to an object file.

 $> nm awesome.so | grep -e "T Add" -e "T Cosine" -e "T Sort" -e "T Log" 00000000000d0db0 T Add 00000000000d0e30 T Cosine 00000000000d0f30 T Log 00000000000d0eb0 T Sort 

Of c

There are two ways to use the library to call Go functions from C. First bind the library: statically at compile stage, dynamically at runtime. Or dynamically load and link the Go function symbols.

Dynamic linking

In this case, we use the header file to create static links between types and functions exported in the object file. The code is simple and clean (some print expressions are omitted):

Client1.c file

 #include <stdio.h> #include "awesome.h" int main() { printf("Using awesome lib from C:\n"); GoInt a = 12; GoInt b = 99; printf("awesome.Add(12,99) = %d\n", Add(a, b)); printf("awesome.Cosine(1) = %f\n", (float)(Cosine(1.0))); GoInt data[6] = {77, 12, 5, 99, 28, 23}; GoSlice nums = {data, 6, 6}; Sort(nums); ... GoString msg = {"Hello from C!", 13}; Log(msg); } 

Now compile the C-code with the library of objects:

 $> gcc -o client client1.c ./awesome.so 

When the resulting binary file is launched, it is linked with the awesome.so library, calls exported functions from Go and returns:

 $> ./client awesome.Add(12,99) = 111 awesome.Cosine(1) = 0.540302 awesome.Sort(77,12,5,99,28,23): 5,12,23,28,77,99, Hello from C! 

Dynamic loading

With this approach, C-code uses the dynamic link loader library for dynamic loading and binding of exported symbols. The functions defined in dhfcn.h apply:

• to open a library file - dlopen,
• to search for characters - dlsym,
• to get errors - dlerror,
• to close the library file - dlclose.

This version will be longer because the binding and linking is done in your source code. But it does the same thing as the previous version (some error handling and print expressions are omitted):

Client2.c file

 #include <stdlib.h> #include <stdio.h> #include <dlfcn.h> // define types needed typedef long long go_int; typedef double go_float64; typedef struct{void *arr; go_int len; go_int cap;} go_slice; typedef struct{const char *p; go_int len;} go_str; int main(int argc, char **argv) { void *handle; char *error; handle = dlopen ("./awesome.so", RTLD_LAZY); if (!handle) { fputs (dlerror(), stderr); exit(1); } go_int (*add)(go_int, go_int) = dlsym(handle, "Add"); if ((error = dlerror()) != NULL) { ... } go_int sum = (*add)(12, 99); printf("awesome.Add(12, 99) = %d\n", sum); go_float64 (*cosine)(go_float64) = dlsym(handle, "Cosine"); go_float64 cos = (*cosine)(1.0); printf("awesome.Cosine(1) = %f\n", cos); void (*sort)(go_slice) = dlsym(handle, "Sort"); go_int data[5] = {44,23,7,66,2}; go_slice nums = {data, 5, 5}; sort(nums); go_int (*log)(go_str) = dlsym(handle, "Log"); go_str msg = {"Hello from C!", 13}; log(msg); dlclose(handle); } 

In the previous code, we defined our own subset of Go-compatible C types: go_int , go_float , go_slice and go_str . We used dlsym to load the characters Add , Cosine , Sort and Log , followed by binding to the corresponding function pointers. Then compiled the code, referring to the dl library (not awesome.so):

 $> gcc -o client client2.c -ldl 

When executing the code, the C binary file loads and links the awesome.so library. The result is:

 $> ./client awesome.Add(12, 99) = 111 awesome.Cosine(1) = 0.540302 awesome.Sort(44,23,7,66,2): 2,7,23,44,66, Hello from C! 

From python

Python is a bit simpler. We use the foreign function library ctypes to call Go functions from the awesome.so library, as shown in the following example (some print expressions are omitted):

Client.py file

 from ctypes import * lib = cdll.LoadLibrary("./awesome.so") lib.Add.argtypes = [c_longlong, c_longlong] print "awesome.Add(12,99) = %d" % lib.Add(12,99) lib.Cosine.argtypes = [c_double] lib.Cosine.restype = c_double cos = lib.Cosine(1) print "awesome.Cosine(1) = %f" % cos class GoSlice(Structure): _fields_ = [("data", POINTER(c_void_p)), ("len", c_longlong), ("cap", c_longlong)] nums = GoSlice((c_void_p * 5)(74, 4, 122, 9, 12), 5, 5) lib.Sort.argtypes = [GoSlice] lib.Sort.restype = None lib.Sort(nums) class GoString(Structure): _fields_ = [("p", c_char_p), ("n", c_longlong)] lib.Log.argtypes = [GoString] msg = GoString(b"Hello Python!", 13) lib.Log(msg) 

Notice that the lib variable represents the loaded characters from the shared object file. We declare the Python GoString and GoSlice classes to match them with the corresponding C structure types. When executing Python code, Go functions are called from a common object:

 $> python client.py awesome.Add(12,99) = 111 awesome.Cosine(1) = 0.540302 awesome.Sort(74,4,122,9,12) = [ 4 9 12 74 122 ] Hello Python! 

Ruby out

Here everything is done according to the same principle as above. Use the FFI gem to dynamically load and call the exported functions from awesome.so.

Client.rb file

 require 'ffi' module Awesome extend FFI::Library ffi_lib './awesome.so' class GoSlice < FFI::Struct layout :data, :pointer, :len, :long_long, :cap, :long_long end class GoString < FFI::Struct layout :p, :pointer, :len, :long_long end attach_function :Add, [:long_long, :long_long], :long_long attach_function :Cosine, [:double], :double attach_function :Sort, [GoSlice.by_value], :void attach_function :Log, [GoString.by_value], :int end print "awesome.Add(12, 99) = ", Awesome.Add(12, 99), "\n" print "awesome.Cosine(1) = ", Awesome.Cosine(1), "\n" nums = [92,101,3,44,7] ptr = FFI::MemoryPointer.new :long_long, nums.size ptr.write_array_of_long_long nums slice = Awesome::GoSlice.new slice[:data] = ptr slice[:len] = nums.size slice[:cap] = nums.size Awesome.Sort(slice) msg = "Hello Ruby!" gostr = Awesome::GoString.new gostr[:p] = FFI::MemoryPointer.from_string(msg) gostr[:len] = msg.size Awesome.Log(gostr) 

In Ruby, we need to extend the FFI module to declare characters to be loaded from the shared library. We declare the Ruby GoSlice and GoString classes to match them with the corresponding C-structures. When executing the code, it calls the Go exported functions:

 $> ruby client.rb awesome.Add(12, 99) = 111 awesome.Cosine(1) = 0.5403023058681398 awesome.Sort([92, 101, 3, 44, 7]) = [3, 7, 44, 92, 101] Hello Ruby! 

From node

We will use the node-ffi external functions library (and a couple of dependent packages) to dynamically load and call the exported Go functions from awesome.so:

Client.js file

 var ref = require("ref"); var ffi = require("ffi"); var Struct = require("ref-struct"); var ArrayType = require("ref-array"); var LongArray = ArrayType(ref.types.longlong); var GoSlice = Struct({ data: LongArray, len: "longlong", cap: "longlong" }); var GoString = Struct({ p: "string", n: "longlong" }); var awesome = ffi.Library("./awesome.so", { Add: ["longlong", ["longlong", "longlong"]], Cosine: ["double", ["double"]], Sort: ["void", [GoSlice]], Log: ["longlong", [GoString]] }); console.log("awesome.Add(12, 99) = ", awesome.Add(12, 99)); console.log("awesome.Cosine(1) = ", awesome.Cosine(1)); nums = LongArray([12,54,0,423,9]); var slice = new GoSlice(); slice["data"] = nums; slice["len"] = 5; slice["cap"] = 5; awesome.Sort(slice); str = new GoString(); str["p"] = "Hello Node!"; str["n"] = 11; awesome.Log(str); 

Node uses an ffi object to declare and load characters from the shared library. We will also declare the Node GoSlice and GoString structural objects to match them with the corresponding C-structures. When executing code, the exported functions are called:

 awesome.Add(12, 99) = 111 awesome.Cosine(1) = 0.5403023058681398 awesome.Sort([12,54,0,423,9] = [ 0, 9, 12, 54, 423 ] Hello Node! 

From java

To call exported functions, we will use the Java Native Access (JNA) library (some expressions are omitted or given by abbreviations):

Client.java file

 import com.sun.jna.*; public class Client { public interface Awesome extends Library { public class GoSlice extends Structure { ... public Pointer data; public long len; public long cap; } public class GoString extends Structure { ... public String p; public long n; } public long Add(long a, long b); public double Cosine(double val); public void Sort(GoSlice.ByValue vals); public long Log(GoString.ByValue str); } static public void main(String argv[]) { Awesome awesome = (Awesome) Native.loadLibrary( "./awesome.so", Awesome.class); System.out.printf(... awesome.Add(12, 99)); System.out.printf(... awesome.Cosine(1.0)); long[] nums = new long[]{53,11,5,2,88}; Memory arr = new Memory(... Native.getNativeSize(Long.TYPE)); Awesome.GoSlice.ByValue slice = new Awesome.GoSlice.ByValue(); slice.data = arr; slice.len = nums.length; slice.cap = nums.length; awesome.Sort(slice); Awesome.GoString.ByValue str = new Awesome.GoString.ByValue(); str.p = "Hello Java!"; str.n = str.p.length(); awesome.Log(str); } } 

To use JNA, we define an Awesome Java interface that will represent characters loaded from awesome.so. We will also declare the GoSlice and GoString classes to match them with the corresponding C-structures. We compile and run the code, the exported functions are called:

 $> javac -cp jna.jar Client.java $> java -cp .:jna.jar Client awesome.Add(12, 99) = 111 awesome.Cosine(1.0) = 0.5403023058681398 awesome.Sort(53,11,5,2,88) = [2 5 11 53 88 ] Hello Java! 

Conclusion

We looked at how to create a Go library for use by other languages. By compiling Go packages into C-style libraries, you can easily access your Go-projects with C, Python, Ruby, Node, Java, etc. using in-process integration of binary object files. Therefore, next time when you make a super API on Go, don't forget to share it with developers in other languages.