Asynchrony: why it does not do it right?

Asynchronous programs are awfully awkward to write. It is so inconvenient that even in node.js , declared as “we have everything correct-asynchronous”, they added synchronous analogs of asynchronous functions. What can we say about the Python syntax that does not allow declaring a lambda with any complex code inside ...

It's funny that a beautiful solution of the problem does not require anything extraordinary, but for some reason it has not yet been implemented.

The essence of the problem

Suppose we have this synchronous code:

var f = open(args);
checkConditions(f);
var result = readAll(f);
checkResult(result);

Asynchronous analog will look much worse:

asyncOpen(args, function (error, f){
if (error)
throw error;
checkConditions(f);
asyncReadAll(f, function (error, result){
if (error)
throw error;
checkResult(result);
});
});

The longer the call chain, the worse the code.

Maybe you are not scared enough? Then try to write an analogue of the following code, replacing all calls with asynchronous ones:

while ( true )
{
var result = getChunk(args1);
while (needsPreprocessing(result))
{
result = preprocess(result);
if (!result)
result = obtainFallback(args2);
}
processResult(result);
}

And it's not that it takes a lot of beeches. The main ambush - the synchronous code given above is remarkably read. But in that asynchronous ugliness that you can do, even you yourself cannot figure it out in a couple of hours.
')
By the way, the example is not unfounded. The remote similarity had to somehow be implemented on a python, and precisely in asynchronous form.

Solutions with a pickaxe and scrap

You may have seen a concept such as Promise in node.js. So, it is no more . The most ordinary callbacks turned out to be much more human. Therefore, I will not tell about Promise.

And I will tell you about the Do library. This library is based on the concept of continuables. Here is an example showing the difference in approaches:

// callback-style
asyncFunc(args, function (error, result){
if (error)
throw error;
doSomething(result);
});

// continuables-style
var continuable = continuableFunc(args);
continuable( function (result){ // callback
doSomething(result);
}, function (error){ // errback
throw error;
});

// continuables-style short
continuableFunc(args)(doSomething, errorHandler);

Continuable is a function that returns another function that accepts callback and errback as parameters and makes an asynchronous call.

In some cases, this approach allows you to significantly simplify the code - take a look at the "continuables-style short". Here, we use doSomething directly as a callback, since the function signature suits us, and we use some kind of “standard” errorHandler, defined elsewhere, as errback.

Do know how much. Parallel calls, asynchronous map, some other interesting things. You can read more about this in the article " Combo Do Library ". There you can read about how to convert functions, sharpened by callback-style (standard for node.js) in the continuables-style.

However, back to the examples with which I started. How can Do help in our case? Actually, here's what:

Do.chain(
continuableOpen(args),
function (f){
checkConditions(f);
return continuableReadAll(f);
}
)( function (result){
checkResult(result);
}, errorHandler);

This is the continuables-style analogue of the very first example. Well, maybe a bit better compared to callback-style, but maybe not. At least the growth of indents with increasing chain length is stopped, and the error handler has concentrated at one point. But the code looks scary, especially when compared to the original four-line synchronous version. A more complex example is that with cycles - Do not go too hard, again you will have to make a terrible garden.

yield to the rescue

Kirk and scrap did not help, I want something sublime. It would be desirable, that the asynchronous call was not more complicated than the synchronous one. And ideally, it was almost no different from him. And it is possible.

The best solution infrastructure is described by Ivan Sagalayev in the “ ADISP ” article. ADISP is the Python library written by him, which brings happiness.

Something similar can be collected on JS, Er.js serves as an example, but there was a lot of magic there for the first acquaintance, therefore I recommend the article by Sagalaev.

The approach used in ADISP allows you to write code in the following style:

var func = process( function (){
while ( true )
{
var result = yield getChunk(args1);
while ( yield needsPreprocessing(result))
{
result = yield preprocess(result);
if (!result)
result = yield obtainFallback(args2);
}
yield processResult(result);
}
});

Yes, this is the most terrible example with cycles. All calls are asynchronous. The frame function func is shown only to show that it will have to be decorated. process - decorator, similar to that described by Sagalayev. getChunk, needsPreprocessing, preprocess, obtainFallback, processResult are the asynchronous functions decorated by the async decorator in ADISP terminology.

The approach works wherever there is a Python-style yield. That is, excellent asynchronous node.js in the span, because V8 does not yet support yield.

Native solution

Do you need something else when using the yield trick we can achieve such good results? I think so, because:

- Using the yield keyword in the context of asynchronous calls looks weird. Still, this word is intended for several other things.
- The need to decorate the framing function - an inconvenience and an extra reason for error
- Although the code of the same ADISP is not complicated, but in order to understand how this thing works, one has to pretty much break the brain. I somehow had to use ADISP in a slightly modified form. I ran into a strange behavior and long and painfully delved into what it was. The jamb was in a completely different place, but the chances of going crazy while debugging were more than real.

The yield example clearly shows that there is everything in the execution environment for implementing asynchronous calls in a convenient, beautiful way. So much so that even without touching the internal mechanisms can achieve the desired. A legitimate question - why not provide a built-in, native solution, the implementation of which should not be too complex?

In essence, the execution environment must remember the context at the place of the asynchronous call and stop the execution, and at the time of the callback call, just restore everything in the required form and, if necessary, throw an exception. And she knows how to do it, at least potentially. The question is how to tell her when we need this trick.

Non-blocking library functions

As a rule, asynchronous functions are implemented either in the core of the language (for example, setTimeout) or in library functions (for example, the functions of the fs module in node.js). Thus, the problem of beautiful asynchronous calls is primarily related to library functions.

This means a wonderful thing - beautiful asynchronous calls can be entered by simply adding a special convention for asynchronous library functions. No need to change the language, no need to invent a new keyword and puzzle over backward compatibility. Just give the library author a way to indicate that the asynchronous library function needs a way to bring the context from which it was called back to life, and the current execution needs to be stopped. Such functions can be safely used, for example, as follows:

while ( true )
{
doSomePeriodicTask();
nbSleep(1000);
}

Here, nbSleep is a non-blocking call to sleep, which will actually interrupt the execution at the call point and someday start it again from the same point, using the saved context as a callback.

Let us even have to have a pair of functions - one usual, with a callback (after all, in some cases, the option with a callback is preferable), and the second non-blocking. It's not scary, if you wish, you can make a wrapper:

var asyncUnlink = fs.unlink;
fs.unlink = function (fName, callback){
if (callback)
return asyncUnlink(fName, callback);
return nbUnlink(fName);
};

At least, synchronous counterparts that can be used for troubles can be thrown out in FIG, replacing their use with the use of non-blocking counterparts.

Keyword async?

If we still want to add beautiful asynchronous calls at the language level, then, apparently, we can not do without a new keyword. It is clear that this is more of a fantasy: a change of language is too free, as opposed to a change in the environment of performance. However, let's take a look at what could happen:

var result1 = async (callback, myAsyncFunc(args, callback)); // long form
var result2 = async myAsyncFunc(args); // short form
var result3 = async (cb, createTask(args, cb), function (task){TaskManager.register(task);});

- Long form: callback is the name of the variable in which the return context will be stored for transmission to the asynchronous function
- Short form: the return context will be added by the last argument
- The third option is “turned inside out”: the return value will be transferred to the lambda (register the created asynchronous “task” in some kind of “manager” - maybe we want to cancel it?), And return to the call point in the usual way for async

Why maybe you need support at the language level? I think only if we need to do something like that with our cunning callback. For example, give it to several functions (oh, what a vicious practice it will be). In most cases, there should be enough support at the level of library functions. And certainly calls to non-blocking library functions will look better than the dominance of the async keyword.

The idea of ​​an "inside-out" call, by the way, is applicable to non-blocking library functions; it is enough to pass a function to them to call just before the end of the current execution.

At last

I hope that someday non-blocking library functions will be added to V8 and node.js, making them even more asynchronous and beautiful. I hope that they will also be added in Python. I hope that this will not stop and in all new and potentially favorite languages ​​and environments instead of synchronous functions there will be non-blocking functions — wherever it makes sense.

* Source Code Highlighter .