Asynchronous recursive iterator and fight with callback in Node.js

On Habré, there were already quite a lot of articles about new features of the ECMAScript 6 standard (for example, “We refuse callbacks: Generators in ECMAScript 6” ) and many use these features.

Using examples from this article, we can:
1. It is easy to write any iterator, for example, an iterator on a tree
2. Write a pseudo-synchronous code (struggle with callback)

But what if we need to write a recursive iterator on the tree, and getting the child nodes requires calling an asynchronous function with a callback transfer?
')
A simple recursive iterator (without invoking asynchronous functions) could look pretty simple and short:

function* iterTree(treenode) { var children = getChildren(treenode); if (children) { // inner node for (let i=0; i < children.length; i++) { yield* iterTree(children[i]); // (*) recursion } } else { // leaf node yield treenode; } } 

The above example is simple because it uses the synchronous call to the getChildren function. If the getChildren function is asynchronous and requires the transfer of a callback (for example, this is receiving data from a disk or via a network), then everything becomes much more complicated. We can no longer write so easily (following the examples from the above article)

 let children = yield getChildren(treenode, resumecallback); 

In this case, the iterTree function will stop at the place where the yield statement is called and transfer control to the calling function. But we have a recursive function - and the calling function will also transfer control higher (operator yield *). As a result, the user of our iterator will receive tree nodes interspersed with unexpected values.

You can of course write special codes inside the function that uses this iterator to distinguish between the return values ​​of the iterator and the return values ​​when calling asynchronous functions. You can also memorize the path from the root to the current vertex (actually creating a structure like a stack) and use this path in the callback function. But in any case, the code becomes quite complicated - at least it stops being easy to read.

And yet, is it possible to write a simple and readable tree traversal iterator that requires calling asynchronous functions?

Somewhat surprisingly, the new features of the language are perfectly combined with the good old node-fiber . There are many libraries based on node-fiber, for example, wait.for or node-sync .
For example, using wait.for , you can write code that is almost identical to the simple iterator at the beginning of the article:

 function* iterTree(treenode) { var children = wait.for(getChildren, treenode); //  if (children) { // inner node for (let i=0; i < children.length; i++) { yield* iterTree(children[i]); // (*) recursion } } else { // leaf node yield treenode; } } 

Using node-sync you can get a similar result.

Sharing new features of ES6 and node-fiber (as well as libraries based on them) allows us to significantly simplify codes.

Sometimes combining alternative approaches gives an amazing result.