Sweet.js: Syntax JavaScript Extensions

Let's try to look at Sweet.js , a compiler that implements hygienic macros for JavaScript.

It works quite simply - you define a set of patterns by which the search is performed by the syntax tree. When a macro matches, it receives a piece of wood that it needs and the body of the macro determines how this piece of wood should be transformed. Further, the result is embedded back into the tree and the procedure continues from that place.

Sweet.js operates with its own syntax tree format, almost at the token level, with minimal structure. On the one hand, this makes it possible to define rather exotic syntaxes for your macros, on the other hand, it makes writing macros somewhat more complicated, as if they were defined above standard AST JavaScript.
')


Let's start with the simplest example, but first you need to install Sweet.js:

npm install --global sweet.js 

After that, we should have the sjs utility available. Let's write a macro that will swap the values ​​of two variables, put the following code in the swap.sjs file:

 macro swap { rule { $x , $y } => { var tmp = $x; $x = $y; $y = tmp } } var x = 11; var y = 12; swap x, y; swap y, x; 

Now, in order to get ES5 compliant JavaScript code, we just need to feed it to the sjs -r ./swap.sjs compiler sjs -r ./swap.sjs

 var x = 11; var y = 12; var tmp = x; x = y; y = tmp; var tmp$2 = y; y = x; x = tmp$2; 

The moment that is worth paying attention to is that Sweet.js generated the variable names when the macro was expanded, thus eliminating the possibility of a name conflict. This means that Sweet.js implements hygienic macros.

Now let's write something useful. How about a set of macros for writing tests in the style of BDD. Let's start with the simplest.

 let describe = macro { rule { $name:lit { $body ... } } => { describe($name, function () { $body ... }); } } let it = macro { rule { $name:lit { $body ... } } => { it($name, function () { $body ... }); } } describe "My functionality" { it "works!" { } } 

In contrast to the macro name form, we used let name = macro - this is done in order to eliminate infinite recursion. Since the describe and it macros return a set of tokens with names that match the names of the macros themselves, Sweet.js will try to use the corresponding macros again and again until the stack ends. The let form helps to avoid this, since it does not create a binding for the name of the macro inside the syntax that is returned by the macro.

Let's see what we have

 describe('My functionality', function () { it('works!', function () { }); }); 

This is more useful than the swap macro, which we wrote at the very beginning - it allows you to save on writing code and use syntactic constructions that are closer to the subject area.

Let's see what else we can do with macros for writing tests. How about a set of macros for writing assertions (assertions)? Since macros have access to the very structure of the code, we can use this to write statements with informative messages about the non-fulfillment of statements. At the same time, let's see how Sweet.js allows you to write infix macros.

What will it all look like? I suggest the following syntax:

 2 + 2 should == 4 "aabbcc" should contain "bb" [1, 2] should be truthy xy() should throw 

At the same time, if an assertion is not fulfilled, I want to see an informative error message that will not only show the values ​​of the current variables in the undefined has no method x style undefined has no method x , but will display which code exactly led to this. For example, 2 + 2 should == 5 should result in error message 2 + 2 should be equal to 5 .

Let's start with the fact that we write a macro that will receive any JavaScript expression and generate a line of code, for this expression - “like parsing, just the opposite”. We need this to generate informative error messages.

 macro fmt { case { _ ( $val:expr ) } => { function fmt(v) { return v.map(function(x){ return x.token.inner ? x.token.value[0] + fmt(x.token.inner) + x.token.value[1] : x.token.value; }).join(''); } return [makeValue('`' + fmt(#{$val}) + '`', #{here})]; } } 

Unlike the previous examples, this macro is a case macro. Unlike rule macros that we used earlier, case macros allow you to use the full power of JavaScript to define a syntactic transformation.

I will not describe in detail what this macro does. But the scheme is this - we define the fmt function that bypasses the syntax tree and generates a line of code from it. Then we construct another syntax tree, which consists of a single node-string and return it as the result of a macro.

 fmt(1 + 1) // "1+1" fmt(xy(1, 2)) // "xy(1,2)" 

As you can see, everything works with a bang, except for the fact that the string is obtained without spaces. Write the best version of the macro fmt remains as an exercise for the reader.

We now turn to the direct definition of syntax for assertions. We will use the assert module from the standard Node.js library for the statements themselves and simply define macros that will be compiled into function calls from this module.

 var assert = require('assert'); macro should { rule infix { $lhs:expr | == $rhs:expr } => { assert.deepEqual( $lhs, $rhs, fmt($lhs) + " should be equal to " + fmt($rhs)); } rule infix { $lhs:expr | be truthy } => { assert.ok( $lhs, fmt($lhs) + " should be truthy"); } rule infix { $lhs:expr | contain $rhs } => { assert.ok( $lhs.indexOf($rhs) > -1, fmt($lhs) + " should contain " + fmt($rhs)); } rule infix { $lhs:expr | throw } => { assert.throws( function() { $lhs }, Error, fmt($lhs) + " should throw"); } } 

We used the rule infix to define infix rules, the symbol | in the template shows where the symbol of the macro name should be located, in this case should .

Now a set of statements

 2 + 2 should == 4 "aabbcc" should contain "bb" [1, 2] should be truthy xy() should throw 

will be revealed in the next ES5-valid code

 var assert = require('assert'); assert.deepEqual(2 + 2, 4, '`2+2`' + ' should be equal to ' + '`4`'); assert.ok('aabbcc'.indexOf('bb') > -1, '`aabbcc`' + ' should contain ' + '`bb`'); assert.ok([ 1, 2 ], '`[1,2]`' + ' should be truthy'); assert.throws(function () { xy(); }, Error, '`xy()`' + ' should throw'); 

Mission accomplished! Now you can start writing your own macros for your tasks or define your syntax for any libraries or frameworks.

All the macros I defined in this article (and even a little bit more) are available on npm and on github:

• sweet-bdd - describe, it, beforeEach, afterEach, ... macros
• sweet-assertions - should macro

In order to use them, you must first install the necessary packages from npm:

 % npm install --global mocha sweet.js % npm install sweet-bdd sweet-assertions 

And then compile and test the code.

 describe "additions" { it "works" { 2 + 2 should == 4 } } 

using the following commands

 % sjs -m sweet-bdd -m sweet-assertions ./specs.sjs > specs.js % mocha specs.js 

Other libraries with macros are also available at npm. I suggest, for example, to look at a sparkler that implements pattern matching in JavaScript:

 function myPatterns { // Match literals case 42 => 'The meaning of life' // Tag checking for JS types using Object::toString case a @ String => 'Hello ' + a // Array destructuring case [...front, back] => back.concat(front) // Object destructuring case { foo: 'bar', x, 'y' } => x // Custom extractors case Email{ user, domain: 'foo.com' } => user // Rest arguments case (a, b, ...rest) => rest // Rest patterns (mapping a pattern over many values) case [...{ x, y }] => _.zip(x, y) // Guards case x @ Number if x > 10 => x } 

I think it was interesting. About all comments, suggestions, please in the comments or who are shy, to me by email .

UPDATE. I forgot to say that Sweet.js has the ability to generate code maps (source maps), so there should be no difficulty with debugging (at least in the browser).