Proper use of require in node.js

Foreword

Not so long ago, the project I am working on at the moment, began to use the ES2015 modular system. I will not focus on this JavaScript technology, because the article is not about that at all, but about how technology has led me to one thought.

As many know, ES2015 Modules are script importing / exporting that is extremely syntactically similar to python and many other programming languages. Example:

 // Helper.js export function includes(array, variable) { return array.indexOf(variable) !== -1; } // main.js import {includes} from 'Helper'; assets(includes([1,2,3], 2), true); 

All those who were interested in JavaScript modules know that importing and exporting is possible only at the top level of the module (the file with the code).

The following rough code example will cause errors:

 // sendEmail.js export default function sendEmails(emails_list) { import sender from 'sender'; export sender; //  - } 

 Exception: SyntaxError: import/export declarations may only appear at top level of a module 

Unlike ES2015 Modules - in the node.js modular system, importing and exporting is possible at any nesting level.

Similar code on node.js will not cause an error:

 // sendEmail.js module.exports = function sendEmails(emails_list) { const sender = require('sender'); exports.sender = sender; //  - } 

The advantage of this method is that the modules necessary in the handler are explicitly imported inside and do not clutter the module's name space (especially true if the imported module is needed only in one handler). There is also the possibility of deferred export of module data.

The main disadvantages:

1. You will find out about the absence of a module only when calling the appropriate handler.
2. The path to the imported module may change, which will lead to a change in each import location (for example, in your module, various handlers use lodash/object/defaults and you decide to upgrade to the 4.x version where you need to connect lodash/defaults ).

Debriefing

Most tasks for which node.js is used are the front-end or the main web server, and high load on node.js frequent. Skipping the ability of your server should be as high as possible.

Bandwidth measurement

To measure the bandwidth of a web server, a great utility from Apache - ab is used . If you are not familiar with her, I highly recommend it.

The web server code is the same except for the handlers.
The test was launched on node.js 6.0 using the ifnode module, made on the basis of express

Importing Modules Directly to a Handler

Code:

 const app = require('ifnode')(); const RequireTestingController = app.Controller({ root: '/', map: { 'GET /not_imported': 'notImportedAction' } }); RequireTestingController.notImportedAction = function(request, response, next) { const data = { message: 'test internal and external require' }; const _defaults = require('lodash/object/defaults'); const _assign = require('lodash/object/assign'); const _clone = require('lodash/lang/clone'); response.ok({ _defaults: _defaults(data, { lodash: 'defaults' }), _assign: _assign(data, { lodash: 'assign' }), _clone: _clone(data) }); }; 

Result:

 $ ab -n 15000 -c 30 -q "http://localhost:8080/not_imported" Server Hostname: localhost Server Port: 8080 Document Path: /not_imported Document Length: 233 bytes Concurrency Level: 30 Time taken for tests: 4.006 seconds Complete requests: 15000 Failed requests: 0 Total transferred: 6195000 bytes HTML transferred: 3495000 bytes Requests per second: 3744.32 [#/sec] (mean) Time per request: 8.012 [ms] (mean) Time per request: 0.267 [ms] (mean, across all concurrent requests) Transfer rate: 1510.16 [Kbytes/sec] received Percentage of the requests served within a certain time (ms) 50% 6 66% 7 75% 8 80% 8 90% 10 95% 15 98% 17 99% 20 100% 289 (longest request) 

Importing modules at the beginning of the file

Code:

 const app = require('ifnode')(); const _defaults = require('lodash/object/defaults'); const _assign = require('lodash/object/assign'); const _clone = require('lodash/lang/clone'); const RequireTestingController = app.Controller({ root: '/', map: { 'GET /already_imported': 'alreadyImportedAction' } }); RequireTestingController.alreadyImportedAction = function(request, response, next) { const data = { message: 'test internal and external require' }; response.ok({ _defaults: _defaults(data, { lodash: 'defaults' }), _assign: _assign(data, { lodash: 'assign' }), _clone: _clone(data) }); }; 

Result:

 $ ab -n 15000 -c 30 -q "http://localhost:8080/already_imported" Server Hostname: localhost Server Port: 8080 Document Path: /already_imported Document Length: 233 bytes Concurrency Level: 30 Time taken for tests: 3.241 seconds Complete requests: 15000 Failed requests: 0 Total transferred: 6195000 bytes HTML transferred: 3495000 bytes Requests per second: 4628.64 [#/sec] (mean) Time per request: 6.481 [ms] (mean) Time per request: 0.216 [ms] (mean, across all concurrent requests) Transfer rate: 1866.83 [Kbytes/sec] received Percentage of the requests served within a certain time (ms) 50% 5 66% 6 75% 6 80% 7 90% 8 95% 14 98% 17 99% 20 100% 38 (longest request) 

Results analysis

Importing modules at the beginning of the file reduced the time of one request by ~ 23% (!) (In comparison with importing directly to the handler), which is quite significant.

Such a big difference in the results lies in the work of the require function. Before importing, require refers to the absolute path search algorithm for the requested component (the algorithm is described in the node.js documentation). When the path was found, then require checks whether the module has been cached, and if not, it doesn’t do anything supernatural except to call the usual fs.readFileSync for .js and .json formats, and the undocumented process.dlopen to load C ++ modules.

Note: I tried to warm up the cache for the case of directly importing modules into the handler (before running the ab utility, the modules were already cached) - performance improved by 1-2%.

findings

If you use node.js as a server (there is no difference what is TCP / UDP or HTTP (S)), then:

1. Importing all modules must be done at the beginning of the file in order to avoid unnecessary synchronous operations associated with loading modules (one of the main anti-patterns of using node.js as an asynchronous server).
2. You can not waste resources on calculating the absolute path of the requested module (this is the main place for performance loss).