Waiting for several events in nodejs

Probably everyone who begins to learn nodejs has difficulty switching to event-oriented programming. Everything is simple, as long as we can perform actions sequentially: start the next, waiting for the final event from the previous one. But what to do if there are many such actions and they are long in time? And if we can not continue until we wait for the completion of each of them?

Everything is simple

A small digression. Any event in the nodejs can be associated with a handler function that will be called when the event occurs. Moreover, it does not matter how it will be transferred to the process that triggers the event: as a parameter of the asynchronous function or by “binding” the function to this event. One thing is important: we need to wait for the completion of each of our handler functions.
To begin with, consider the simplest example. We have several long actions (we will use the setTimeout function) and we will run the next action only after the previous one is completed.
So, an example:

console.log("begin"); setTimeout(function () { console.log("2000ms timeout"); setTimeout(function () { console.log("1500ms timeout"); setTimeout(function () { console.log("1000ms timeout"); setTimeout(function () { console.log("final"); }, 500); }, 1000); }, 1500); }, 2000); console.log("end"); 

We start some process with a length of 2000 ms, after its completion we launch the second one for 1500 ms, then the third one for 1000 ms and, finally, the last one, for 500 ms. The result will be as follows:

 begin
 end
 2000ms timeout
 1500ms timeout
 1000ms timeout
 final

Everything is bad if the actions performed by the program really have to be performed sequentially, and there is no way to start the second action before the end of the first one, and the third - before the second one. But if it is possible, then it is not possible, but necessary!

Why is it necessary?

What is the point of suspending the whole process and waiting for the slow subsystem to do everything we want from it, if we have something to do besides waiting? No Therefore, the simultaneous launch of several long operations, and these may include work with the network, disk subsystem, can significantly speed up the execution of the program.
')

A little harder

Suppose that all our operations are independent of each other. Then they can be run in parallel:

 console.log("begin"); setTimeout(function () { console.log("2000ms timeout"); }, 2000); setTimeout(function () { console.log("1500ms timeout"); }, 1500); setTimeout(function () { console.log("1000ms timeout"); }, 1000); setTimeout(function () { console.log("final"); }, 500); console.log("end"); 

Result of performance:

 begin
 end
 final
 1000ms timeout
 1500ms timeout
 2000ms timeout

The main program completed its execution, and then began to consistently work out callback functions for each of the “long actions”. But the main thing - the program execution time has decreased by almost two and a half times!
Again, there is nothing difficult in this example, but only as long as one of the actions does not have to wait until the others are completed.

Even harder

Suppose we can start the last action only after waiting for the completion of all the previous ones. The first thing that comes to mind is the organization of the counter. First, its value will be equal to the number of our callback functions, which must be waited for. At the end of each of them, the counter will decrease, and when the last one reaches zero, our “waiting function” will be called.
For example, like this:

 var counter = 3; console.log("begin"); setTimeout(function () { console.log("2000ms timeout"); if (-- counter == 0) final(); }, 2000); setTimeout(function () { console.log("1500ms timeout"); if (-- counter == 0) final(); }, 1500); setTimeout(function () { console.log("1000ms timeout"); if (-- counter == 0) final(); }, 1000); function final() { setTimeout(function () { console.log("final"); }, 500); } console.log("end"); 

Result:

 begin
 end
 1000ms timeout
 1500ms timeout
 2000ms timeout
 final

All perfectly!
The problems will begin when one day we forget to add one to the counter or add a modification to the callback function and a call to the “waiting function”.
“All work with the meter must be wrapped in some object!” - I thought. But, as it turned out, I’m not the only one: Tim Caswell has already proposed something similar, and I took his idea as a basis.
Wrapper:

 function Combo(finalCallback) { this.finalCallback = finalCallback; this.result = []; this.counter = 0; } Combo.prototype = { "add" : function (callback) { var that = this; this.counter ++; return function () { that.result[that.counter - 1] = callback.apply(this, arguments); that.check(); }; }, "check" : function () { var that = this; this.counter --; if (this.counter == 0) process.nextTick(function () { that.finalCallback.call(that, that.result); }); } }; 

When an object is created, the constructor, as a parameter, receives a “waiting function”, which the wrapper will launch upon completion of all “expected functions”. The add method increments the counter and creates a wrapper function for the user “expected function”. The created wrapper function runs the corresponding “expected function”, passing all the parameters received to it, and then runs the check method. The check method, in turn, reduces the counter and, when the latter reaches zero, starts the “waiting function” passed in the constructor. In this case, just in case, the results of the work of “expected functions” are saved, which are transmitted as a parameter of the “waiting function”.
Example:

 var test = new Combo( function (result) { console.log("final"); console.log("result:"); for (var i in result) { console.log(" \"" + i + "\" : \"" + result[i] + "\""); } }); console.log("begin"); setTimeout(test.add(function () { console.log("2000ms"); return "2000ms"; }), 2000); setTimeout(test.add(function () { console.log("1500ms"); return "1500ms"; }), 1500); setTimeout(test.add(function () { console.log("1000ms"); return "1000ms"; }), 1000); console.log("end"); 

Result of work:

 begin
 end
 1000ms
 1500ms
 2000ms
 final
 result:
   "0": "2000ms"
   "1": "1500ms"
   "2": "1000ms"

Beauty!
Now we can slip the closing events (for example, the end event for threads) to the wrapper function created by the add method and our “waiting function” will be launched only after all the “expected functions” have completed.
The main thing is not to forget to wrap up the new “expected functions”.

Most difficult

But what if one of the actions failed? For asynchronous functions that take a callback function as a parameter, there will be no problems: the first parameter of the callback function is passed a special variable containing error information, if any. But, for example, in the event of a failure, the threads “throw out” the error event. At the same time, the final event “end” will not be thrown out, which means that one of the “expected functions” will not be launched either. As a result, the “waiting function” will never be started.
If you can complete the program with the release of an exception - very good. But what to do if such a situation should be processed and continued? You need to add a mechanism to remove the "expected features"!
For example:

 function Combo(finalCallback) { this.finalCallback = finalCallback; this.result = {}; this.counter = 0; } Combo.prototype = { "add" : function (callback, id) { var that = this; if (!id) id = this.counter; this.counter ++; return function () { if (!that.result.hasOwnProperty(id)) { that.result[id] = callback.apply(this, arguments); that.check(); } }; }, "remove" : function (id, result) { this.result[id] = result; this.check(); }, "check" : function () { var that = this; this.counter --; if (this.counter == 0) process.nextTick(function () { that.finalCallback.call(that, that.result); }); } }; 

As a second, optional parameter, the add method is passed an identifier by which, if necessary, you can remove the “expected function”. And besides, the result of the “expected function” is now stored in the field with the specified identifier. The remove method is passed the identifier of the “expected function” and “the result by mistake”, which will be stored in the object-keeper of the results.
Example:

 var test = new Combo(function (result) { console.log("final"); console.log("result:"); for (var i in result) { console.log(" \"" + i + "\" : \"" + result[i] + "\""); } }); console.log("begin"); setTimeout(test.add(function () { console.log("2000ms"); return "2000ms"; }), 2000); setTimeout(test.add(function () { console.log("1500ms"); return "1500ms"; }, "1500ms"), 1500); setTimeout(test.add(function () { console.log("1000ms"); return "1000ms"; }), 1000); // error setTimeout(function () { console.log("something wrong in 1500ms on 1250ms"); test.remove("1500ms", "error in 1500ms"); }, 1250); console.log("end"); 

Result:

 begin
 end
 1000ms
 something wrong in 1500ms on 1250ms
 2000ms
 final
 result:
  "0": "2000ms"
  "2": "1000ms"
  "1500ms": "error in 1500ms"

Conclusion

Of course, the wrapper can and should be improved, but for me the development of this object has become a very good warm-up. If everything is done leisurely and thoughtfully, event-oriented programming is not so difficult.
And, of course, do not forget about the help of the community: perhaps, someone has already solved problems similar to yours.