Bike: Promises in Node.js
Good afternoon, Habrahabr.
There was a fairly simple task: to retrieve a set of documents from the database, convert each document and send all converted documents to the user, the order cannot be changed, an asynchronous function is used to process the document. If an error has occurred on some document - we do not send documents, only an error and finish processing the documents.
To solve the problem, the Q library was chosen, since the Promise hike itself is pretty to me. But there was one catch, seemingly an elementary task, and it takes more than a second, or rather 1300 ms, instead of the expected 50-80 ms. In order to figure out how everything works and get asynchronous, it was decided to write a specialized "bicycle" for this task.
')
First of all, I would like to tell you how it was implemented initially.
1. A procedure that took place sequentially in an array and returned us a promise.
2. The procedure that processed the document and returned us a promise.
3. And the parent procedure that sends the result to the user
Some may notice that there are a lot of various “promises” even where they are not much needed, and that such a large looping has led to such speed, but the procedure gives only 20 simple documents, the transformations are primitive and they have to be fulfilled so much time. good for nothing.
The network is full of descriptions of what and how. I will describe in brief. Promise is a kind of promise. Some function promises us a result, we can get it with the help of then (success, error) , in turn, with successful processing, we can assign a new promise and also process it. In the particular case, it looks like this:
The result of each stage is transmitted as a parameter to the next and so sequentially. As a result, we process all errors in one block and get rid of the "noodles".
Inside it looks like this: events are created that are called upon successful completion or on error:
All this can be read here .
The theory is over, let's get down to practice.
The objective of this object is to create the events we need and issue a Promise.
Its task will be to track the events " completed " and " error " to call the necessary functions that are assigned via " then " and to track what this function returns there: if it returned us another promise, then connect to it in order that the following would work then if simply data, then perform the following then , so we can build chains of then .
So, the basic model is ready. It remains to make a binding for functions with callback
Its task is to make a wrapper for a function with callback with the possibility of specifying this and launch arguments.
Everything is simple: we are passed an array of functions, we tie them up with promisefn and, when they all execute, call resolve
After testing, the old approach (through the Q library) was rewritten toreplace a couple of ads and run under the same conditions. The result is positive - 50-100 ms (instead of the previous 1300 ms).
All source codes are available on Github , examples can also be found there. The invention of "bicycles" is useful at least in that it improves understanding.
Thanks for attention!
Foreword
There was a fairly simple task: to retrieve a set of documents from the database, convert each document and send all converted documents to the user, the order cannot be changed, an asynchronous function is used to process the document. If an error has occurred on some document - we do not send documents, only an error and finish processing the documents.
To solve the problem, the Q library was chosen, since the Promise hike itself is pretty to me. But there was one catch, seemingly an elementary task, and it takes more than a second, or rather 1300 ms, instead of the expected 50-80 ms. In order to figure out how everything works and get asynchronous, it was decided to write a specialized "bicycle" for this task.
')
How was the work with Q
First of all, I would like to tell you how it was implemented initially.
1. A procedure that took place sequentially in an array and returned us a promise.
forEachSerial
function forEachSerial(array, func) { var tickFunction = function (el, index, callback) { if (func.length == 2) func(el, callback); else if (func.length == 3) func(el, index, callback); } var functions = []; array.forEach(function (el, index) { functions.push(Q.nfcall(tickFunction, el, index)); }); return Q.all(functions); } 2. The procedure that processed the document and returned us a promise.
documentToJSON
function documentToJSON(el, options) { var obj = el.toObject(options); var deferred = Q.defer(); var columns = ['...','...','...']; forEachSerial(columns,function (el, next) { el.somyAsyncFunction(options, function (err, result) { if (err) next(err); else result.somyAsyncFunction2(options, function (err, result) { if (!err) obj[el] = result; next(err); }); }); }).then(function () { deferred.resolve(obj); }, function (err) { deferred.reject(err); }); return deferred.promise; } 3. And the parent procedure that sends the result to the user
sendResponse
exports.get = function (req, res, next) { dbModel.find(search, {}, { skip: from, limit: limit }).sort({column1: -1}).exec(function (err, result) { if (err) next(new errorsHandlers.internalError()); else { var result = []; forEachSerial(result,function (el, index, next) { documentToJSON(el,options).then(function (obj) { result[index] = obj; next() }, function (err) { next(new errorsHandlers.internalError()) }); }).then(function () { res.send({responce: result}); }, function (err) { next(new errorsHandlers.internalError()) }); } }); }; Some may notice that there are a lot of various “promises” even where they are not much needed, and that such a large looping has led to such speed, but the procedure gives only 20 simple documents, the transformations are primitive and they have to be fulfilled so much time. good for nothing.
We write our library promise
How does "it" work?
The network is full of descriptions of what and how. I will describe in brief. Promise is a kind of promise. Some function promises us a result, we can get it with the help of then (success, error) , in turn, with successful processing, we can assign a new promise and also process it. In the particular case, it looks like this:
Promise.then(step1).then(step2).then(step3).then(function () { //All OK }, function (err) { //Error in any step }); The result of each stage is transmitted as a parameter to the next and so sequentially. As a result, we process all errors in one block and get rid of the "noodles".
Inside it looks like this: events are created that are called upon successful completion or on error:
var promise = fs.stat("foo"); promise.addListener("success", function (value) { // ok }) promise.addListener("error", function (error) { // error }); All this can be read here .
The theory is over, let's get down to practice.
Let's start with a simple - Deferred
The objective of this object is to create the events we need and issue a Promise.
function deferred() { this.events = new EventEmitter(); // this.promise = new promise(this); // Promise this.thenDeferred = []; // , var self = this; // this.resolve = function () { self.events.emit('completed', arguments); } // this.reject = function (error) { self.events.emit('error', error); // self.thenDeferred.forEach(function (el) { el.reject(error); }); } } Object - Promise
Its task will be to track the events " completed " and " error " to call the necessary functions that are assigned via " then " and to track what this function returns there: if it returned us another promise, then connect to it in order that the following would work then if simply data, then perform the following then , so we can build chains of then .
function promise(def) { this.def = def; this.completed = false; this.events = def.events; var self = this; var thenDeferred; self._successListener = null; self._errorListener = null; // then - promise this.then = function (success, error) { if (success) self._successListener = success; if (error) self._errorListener = error; thenDeferred = new deferred(); self.def.thenDeferred.push(thenDeferred); return thenDeferred.promise; } // this.events.on('completed', function (result) { // , , var args = inputOfFunctionToArray(result); // , if (self.completed) return; self.completed = true; if (self._successListener) { var result; try { result = self._successListener.apply(self, args); } catch (e) { self.def.reject(e); result; } // Promise then, var promise; if (isPromise(result)) promise = result; else if (result instanceof deferred) promise = result.promise; if (promise && thenDeferred) { promise.then(function () { var args = arguments; process.nextTick(function () { thenDeferred.resolve.apply(self, args); }); }, function (error) { process.nextTick(function () { thenDeferred.reject(error); }); }); } else if (thenDeferred) process.nextTick(function () { // then thenDeferred.resolve.apply(self, [result]); }); } else if (thenDeferred) process.nextTick(function () { thenDeferred.resolve.apply(self, []); }); }); // this.events.on('error', function (error) { if (self.completed) return; self.completed = true; if (self._errorListener) process.nextTick(function () { self._errorListener.apply(self, [error]); }); }); } So, the basic model is ready. It remains to make a binding for functions with callback
PromiseFn
Its task is to make a wrapper for a function with callback with the possibility of specifying this and launch arguments.
var promisefn = function (bind, fn) { var def = new deferred(); //bind if (typeof bind === 'function' && !fn) { fn = bind; bind = def; } // callback var callback = function (err) { if (err) def.reject(err); else { var args = []; for (var key in arguments) args.push(arguments[key]); args.splice(0, 1); def.resolve.apply(bind, args); } }; var result = function () { var args = []; for (var key in arguments) args.push(arguments[key]); args.push(callback); process.nextTick(function () { fn.apply(bind, args); }); return def.promise; } return result; } And finally, ALL - sequential execution of functions with callback
Everything is simple: we are passed an array of functions, we tie them up with promisefn and, when they all execute, call resolve
var all = function (functions) { var def = new deferred(); process.nextTick(function () { var index = -1; var result = []; var next = function (err, arguments) { if (err) { def.reject(err); return; } if (arguments) result.push(inputOfFunctionToArray(arguments)); index++; if (index >= functions.length) { def.resolve(result); } else process.nextTick(function () { promisefn(functions[index])().then(function () { var args = arguments; process.nextTick(function () { next(err, args); }); }, function (err) { process.nextTick(function () { next(err); }); }); }); } process.nextTick(next); }); return def.promise; } Finally
After testing, the old approach (through the Q library) was rewritten to
All source codes are available on Github , examples can also be found there. The invention of "bicycles" is useful at least in that it improves understanding.
Thanks for attention!
Source: https://habr.com/ru/post/168929/
All Articles