Nsynjs - JavaScript engine with synchronous streams and without callbacks

In this article I will talk about the result of my second attempt at dealing with callbacks in JavaScript. The first attempt was described in a previous article . In the comments to it, I was prompted by some ideas that were implemented in the new project - nsynjs (next synjs).



TLDR: nsynjs is a JavaScript engine that can wait for the execution of callbacks and execute instructions sequentially.

This is achieved by the fact that nsynjs breaks the code of the executable function into separate operators and expressions, wraps them into internal functions, and executes them one by one.
')
Nsynjs allows you to write fully consistent code, like this:

var i=0; while(i<5) { wait(1000); // <<--      console.log(i, new Date()); i++; } 

or such

 function getStats(userId) { return { // <<-- ,      friends: dbQuery("select * from firends where user_id = "+userId).data, comments: dbQuery("select * from comments where user_id = "+userId).data, likes: dbQuery("select * from likes where user_id = "+userId).data, }; } 

Nsynjs supports most ECMAScript 2015 constructs, including loops, conditional statements, exceptions, try-catch blocks, closures (it would be better to translate as "context variables"), and so on.

Compared to Babel, he:

• still easier (81kb without minimizing),
• has no dependencies
• does not require compilation
• performed much faster
• allows you to start and stop long-lived streams.

To illustrate, consider a small example of a web application that:

1. Receives a list of files via ajax request
2. For each file in the list:
3. Receives a file via ajax request
4. Writes the contents of the file to the page.
5. Wait 1 sec

A synchronous pseudo-code for this application would look like this (looking ahead, the real code is almost the same):

 var data = ajaxGetJson("data/index.json"); for(var i in data) { var el = ajaxGetJson("data/"+data[i]); progressDiv.append("<div>"+el+"</div>"); wait(1000); }; 

The first thing to take care of is to identify all the asynchronous functions that we need and wrap them in a wrapper function to call them from the synchronous code in the future.

The wrapper function should usually do the following:

• accept a pointer to the state of the calling thread as a parameter (for example, ctx)
• call wrapped function with a callback
• return the object as a parameter of the operator return, assign the result of the callback to some property of this object
• call ctx.resume () in a callback (if there are several callbacks, then select the latest one)
• set the destructor function to be called in case of a thread interruption.

For all wrapper functions, the 'synjsHasCallback' property must be set to true.

Create a simple wrapper for setTimeout. Since we do not get any data from this function, the return operator is not needed here. The result is the following code:

 var wait = function (ctx, ms) { setTimeout(function () { console.log('firing timeout'); ctx.resume(); // <<--     }, ms); }; wait.synjsHasCallback = true; // <<--   nsynjs,   -    

She, in principle, will work. But a problem may arise if the calling thread was stopped while the callback was in progress: the setTimeout function would still be called and the message printed. To avoid this, you must also cancel the timeout when the stream is stopped. This can be done by setting the destructor.

The wrapper will then turn out like this:

 var wait = function (ctx, ms) { var timeoutId = setTimeout(function () { console.log('firing timeout'); ctx.resume(); }, ms); ctx.setDestructor(function () { console.log('clear timeout'); clearTimeout(timeoutId); }); }; wait.synjsHasCallback = true; 

We also need a wrapper over the jQuery library's getJSON function. In the simplest case, it will look like this:

 var ajaxGetJson = function (ctx,url) { var res = {}; $.getJSON(url, function (data) { res.data = data; ctx.resume(); }); return res; }; ajaxGetJson.synjsHasCallback = true; 

This code will only work if getJSON successfully received the data. On error, ctx.resume () will not be called, and the calling thread will never resume. To handle errors, the code must be modified code like this:

 var ajaxGetJson = function (ctx,url) { var res = {}; var ex; $.getJSON(url, function (data) { res.data = data; // <<--   ,   }) .fail(function(e) { ex = e; // <<--   ,   }) .always(function() { ctx.resume(ex); // <<--      , //        }); return res; }; ajaxGetJson.synjsHasCallback = true; 

To getJSON to forcibly stop when the calling thread stops, you can add a destructor:

 var ajaxGetJson = function (ctx,url) { var res = {}; var ex; var ajax = $.getJSON(url, function (data) { res.data = data; // <<--   ,   }) .fail(function(e) { ex = e; // <<--   ,   }) .always(function() { ctx.resume(ex); // <<--      , //        }); ctx.setDestructor(function () { ajax.abort(); }); return res; }; 

When the wrappers are ready, we can write the application logic itself:

 function process() { var log = $('#log'); log.append("<div>Started...</div>"); //       synjsCtx,   //       var data = ajaxGetJson(synjsCtx, "data/index.json").data; log.append("<div>Length: "+data.length+"</div>"); for(var i in data) { log.append("<div>"+i+", "+data[i]+"</div>"); var el = ajaxGetJson(synjsCtx, "data/"+data[i]); log.append("<div>"+el.data.descr+","+"</div>"); wait(synjsCtx,1000); } log.append('Done'); } 

Since the ajaxGetJson function may throw an exception in some cases, it makes sense to enclose it in a try-catch block:

 function process() { var log = $('#log'); log.append("<div>Started...</div>"); var data = ajaxGetJson(synjsCtx, "data/index.json").data; log.append("<div>Length: "+data.length+"</div>"); for(var i in data) { log.append("<div>"+i+", "+data[i]+"</div>"); try { var el = ajaxGetJson(synjsCtx, "data/"+data[i]); log.append("<div>"+el.data.descr+","+"</div>"); } catch (ex) { log.append("<div>Error: "+ex.statusText+"</div>"); } wait(synjsCtx,1000); } log.append('Done'); } 

The last step is to call our synchronous function through the nsynjs engine:

 nsynjs.run(process,{},function () { console.log('process() done.'); }); 

nsynjs.run accepts the following parameters:

 var ctx = nsynjs.run(myFunct,obj, param1, param2 [, param3 etc], callback) 

• myFunct: a pointer to the function that you want to perform in synchronous mode
• obj: an object that will be accessible through this in the myFunct function
• param1, param2, etc - parameters for myFunct
• callback: a callback to be called when myFunct completes.

Return value: The context of the stream state.

Under the hood

When you call a function through nsynjs, the engine checks for the presence and, if necessary, creates the synjsBin property of this function. This property stores a tree structure equivalent to the compiled function code. Further, the engine creates a stream state context in which local variables, stacks, program counters, and other information necessary for stopping / resuming execution are stored. After that, the main loop is started, in which the program counter sequentially iterates over the synjsBin elements, and executes them using the state context as a storage.

When executing a synchronous code that contains calls to other functions, nsynjs recognizes three types of called functions:

• synchronous
• wrappers over cabbies
• native.

The type of the function is determined in runtime by analyzing the following properties:

• if the function pointer has the synjsBin property, then the function will be executed via nsynjs in synchronous mode
• if the function pointer has a synjsHasCallback property, then this is a wrapper function, so nsynjs will stop execution on it. The wrapper function must itself take care of resuming the calling synchronous stream by calling ctx.resume () in the flask.
• All other functions are considered native, and return the result immediately.

Performance

When parsing, the nsynjs engine tries to analyze and optimize the code elements of the original function. For example, consider the cycle:

 for(i=0; i<arr.length; i++) { res += arr[i]; } 

This cycle will be optimized and compiled into one internal function that will be executed almost as fast as the native code:

 this.execute = function(state) { for(state.localVars.i=0; state.localVars.i<arr.length; state.localVars.i++) { state.localVars.res += state.localVars.arr[state.localVars.i]; } } 

However, if there are function calls in the code element, as well as the continue, break, and return statements, the optimization for them, as well as for all parent elements, will not be executed.

The impossibility of optimizing expressions with function calls is due to the fact that a pointer to a function, and therefore its type, can be calculated only during execution.

For example operator:

 var n = Math.E 

will be optimized in one function:

 this.execute = function(state,prev, v) { return state.localVars.n = Math.E } 

If the operator has a function call, then nsynjs cannot know the type of the function being called in advance:

 var n = Math.random() 

Therefore, the entire operator will be executed in steps:

 this.execute = function(state) { return Math } .. this.execute = function(state,prev) { return prev.random } .. this.execute = function(state,prev) { return prev() } .. this.execute = function(state,prev, v) { return state.localVars.n = v } 

Links

→ GitHub repository
→ Examples
→ Tests
→ NPM