Have you ever looked for an alternative to Emacs Lisp? Let's try to add another programming language to Emacs.

In this article:

• The potential benefits to be gained when you can extend Emacs to Go;
• Define how Go and Emacs Lisp interact;
• We will touch on some details of the implementation of the described transcompiler;

An article may be of interest to users of Emacs, as well as those who care about all these countless implementations of countless programming languages.

At the very end of the article is a link to a work in progress project that allows you to convert Go to Emacs Lisp.

Choosing Emacs Go

Like any other programming language, Emacs Lisp has a number of "flaws", which we will politically correctly call "design tradeoffs". It is rather difficult to say "better" or "worse" about certain properties of the programming language at the objective level, because there will almost always be defenders of opposite positions. We, as users of programming languages, can try to choose the language whose compromises are easier for us to accept due to our tasks or personal preferences. The key point is the choice.

Suppose we chose Go. How will you use Go to interact with the editor?

Your options:

1. Use Emacs modules to run Go functions. Inspiration can be drawn from the project go-emacs .
2. Find (or write) the Go interpreter, embed it in Emacs by patching or the same C modules, and then call eval from the editor.
3. Stream Go to Emacs Lisp bytecode.

There can be more ways, but none of them will be closer to the "native" list than (3). It allows you to have the same virtual machine at the execution level as a regular Emacs Lisp.

This, in turn, means that:

• Emacs Lisp code will be able to call translated Go code;
• FFI is free. Calling a function already defined in Emacs from Go is as efficient as possible;
• Easily distribute converted packages (native to Emacs format);

If this is your first time hearing about Emacs bytecode, read the article by Chris Wellons .

Why go?

In place of Go, there could potentially be any other programming language.

There are several reasons for which the choice made becomes more reasonable. The main ones are:

• Language compiler inside the standard library;
• Laconic specification;
• Modest runtime;
• Tooling;

There are also those properties that could be arguments in favor of choice, but specifically for me they were less significant:

• Go is a fairly popular language with C-like syntax (i.e., this is not for you Scheme);
• Static typing;

Language compiler inside the standard library

go/* packages make it much easier to write Go tools.

No need to write parser, typechecker, and other delights of the frontend compiler. Over 20 lines of code we can get AST and type information for the whole package.

Documentation is mostly good. And for go/types in my opinion - exemplary .

Initially for me it was a killer argument. The task seemed to be 90% solved thanks to this secret weapon: "it remains only to convert AST to Emacs byte-code".

Laconic specification

Creating an implementation that is more (~ 80%) conforming to the specification is quite a feasible task for one person. The Go specification is easy to read, it can be mastered in the evening.

Specification Features:

1. Some points cast doubt on the unambiguous interpretation. Short price;
2. In addition to the specification there is also Effective Go . Without it, white spots will remain in the specification;

Modest runtime

The more features in the language that are implemented in the runtime library, the more difficult it will be to transcompile.

If at least temporarily throwing overboard the gorutines and channels, then a compact core will remain, which can be fully realized in terms of Emacs Lisp without loss of performance.

Tooling

It is damn nice when many familiar features work in several of your favorite editors, moreover, uniformly.

For Go, many of the functions that are usually re-inventoried for each IDE separately are implemented as separate utilities. The simplest example known to every Go developer is gofmt . To a large extent this is facilitated by the above go/types ,

go/build and other packages from the go/* group.

Sublime text, Emacs, Visual Studio Code - choose any of them, install the plugin (s), and enjoy refactoring through gorename , a lot of linkers and automatic import'ami. And auto-completion ... exceeds company-elisp in many aspects.

Refactoring and maintaining a project on Emacs Lisp after 1000 lines of code is already uncomfortable for me personally. Emacs is much more convenient to program on Go than on Emacs Lisp.

What does Emacs Go look like

Let's fantasize about what Go might look like for Emacs. How comfortable and functional would it be?

Bridge types

Before talking directly about calling Lisp functions, you need to think about the bridge that connects two programming languages ​​that work on the same computational model.

With primitive types like int , float64 , string and others, everything is more or less simple. Both Go and Emacs Lisp have these types.

Of interest are slices, symbolic types ( symbol ) and unsigned integer types of fixed width ( uintX ).

• We implement slices in runtime (for example, on the same Emacs Lisp);
• Characters are represented as opaque type;
• Unsigned arithmetic with deterministic overflow - we emulate;

A type that can express an "arbitrary type object" that is returned by Emacs Lisp function is called lisp.Object . Its definition is given under the spoiler lisp.Object: implementation details .

 ;;   <vector> ;; [vector] <index> ;; [vector index] aref ;; [elem] ;; Slice  offset (  subslice) <slice> ;; [slice] car ;; [data] <index> ;; [data index] aref ;; [elem] ;; Slice   subslice <slice> ;; [slice] dup ;; [slice slice] (1) car ;; [slice data] stack-ref 1 ;; [slice data slice] cdr ;; [slice data slice.cdr] car ;; [slice data offset] <index> ;; [slice data offset index] plus ;; [slice data real-index] aref ;; [slice elem] stack-set 1 ;; [elem] (2) ;; (1)  <slice>    ,  ;;   . ;; (2)        ;;  slice  . 

 type Symbol interface { symbol() } type Object interface { object() //  ... } //       -. // Intern returns the canonical symbol with specified name. func Intern(name string) Symbol 

 type Object interface { object() Int() int Float() float64 String() string // ... etc. //     : IsInt() bool //     GetInt() (val int, ok bool) //  "comma, ok"-style  // ...    . } 

Emacs Lisp from Go

If the signature of the function is unknown, then the only thing that remains is to take a variable number of arguments of an arbitrary type, and return lisp.Object .

 pair := lisp.Call("cons", 1, "2") a, b := lisp.Call("car", pair), lisp.Call("cdr", pair) lisp.Call("insert", "Hello, Emacs!") sum := lisp.Call("+", 1, 2).Int() 

Manually annotated functions can be called in a more convenient way.

 part := "c" lisp.Insert("Hello, Emacs!") //  void s := lisp.Concat("a", "b", part) //  string,  ...string 

 ;;   ,   FFI. (ffi-declare (concat Concat (:string &parts) :string) (message Message (:string format :any &args) :string) (insert Insert (:any &args) :void) (+ IntAdd (:int &xs) :int)) ;;  , ,  Go . 

 // IntAdd - ... <  +  Emacs> //$GO-ffi:+ func IntAdd(xs ...int) int // ...   

Go from Emacs Lisp

The result of the transcompilation is the Emacs Lisp package, in which all the characters from Go have a transformed look.

An identifier mapping scheme can be, for example:

 package "foo" func "f" => "$GO-foo.f" package "foo/bar" func "f" => "$GO-foo/bar.g" package "foo" func (typ) "m" => "$GO-foo.typ.m" package "foo" var "v" => "$GO-foo.v" 

Accordingly, in order to call a function or use a variable, you need to know which Go package it belonged to (and its name, of course). The $GO prefix avoids conflicts with names already defined in Emacs.

Transcompilation subtleties

Bytecode or lapcode?

As an output format, you can choose among three options:

1. Emacs Lisp code (source-to-source compilation)
2. Bytecode
3. Lapcode (Lisp Assembly Program)

The first option loses much to the other options, because it will not allow for the effective translation of the return statement , and it is also more difficult to implement the goto (which is in Go).

The second and third options are almost equivalent in their capabilities.

• Bytecode is an analogue of machine code, the lowest level;
• Lapcode is an assembly language for a virtual machine with a stack architecture;

The Emacs compiler can optimize at the source code level and lapcode views.

If we choose lapcode, we can additionally apply low-level optimizations,

implemented by emacs developers.

Generated Code Performance

Go, which runs inside Emacs VM cannot be faster than Emacs Lisp.

Or maybe?

Emacs Lisp has dynamic scoping for variables. If you look at "emacs / lisp / emacs-lisp / byte-opt.el" , you can find many references to this feature of the language; because of it, some optimizations are either impossible or much more difficult.

There are no constants in Emacs Lisp. Names declared using defconstant less immutable than those defined by defvar . In Go, constants are embedded in the place of use, which allows you to collapse more constant expressions.

Optimizing Go code is easier, so you can expect at least performance that is not inferior to normal Emacs Lisp code. Potentially, overtaking in terms of speed is real.

Implementation difficulties

Even without Gorutin, there are Go features that do not have an obvious and / or optimal implementation within Emacs VM.

The most interesting difficulty are pointers.

In the context of the task, we can distinguish two categories of values ​​in Emacs Lisp:

• Reference types ( string , vector , list / cons )
• Value Types ( integer and float )

For reference types, the problem is solved easier.

Taking an address from an int or float variable requires processing

more boundary cases.

We should also remember that the operator = defined for pointers,

therefore, the proposed solution must respect the identity of the addresses of the objects.

Wrapping the number in cons , we fly by semantics,

because the value from which the address was taken,

will not change if the data stored in cons changes.

If all numbers are initially created in boxed form (inside cons ),

greatly increase the number of allocations.

Unpacking will require additional car instructions every time a value is read.

Implementing pointers through cons has a significant flaw: &x != &x ,

because (eq (cons x nil) (cons x nil)) always false.

Correct emulation of pointer semantics is an open question.

I will be glad to hear your ideas for their implementation.

Seems like go-ism inside emacs

The goism project is a tool that allows you to get near-optimal Emacs Lisp bytecode from Go packets.

The runtime library was originally written in Lisp, but more recently it has been completely rewritten to be broadcast in lapcode Go.

emacs / rt is currently one of the largest packages written with goism .

At the moment, goism is not particularly friendly towards the end user,

have to work with your hands to properly assemble and configure it

( guick start guide should simplify the task).

Why is the article written right now, and not when a more stable version was released? The answer is quite simple: there is no guarantee that this version will ever be ready, plus you can refine it for a very, very long time.

I would like to know if this idea seems interesting and useful to members of the habr-community.