Rewrite Require.js using Promise. Part 1

In order not to have problems with dependencies and modules with a large number of browser javascript, usually use require.js . Also, many people know that this is just one of many of the AMD standard downloaders, and it has alternatives. But few people know how they are arranged inside. In fact, it is not difficult to write such a tool, and in this article we will write our version of the AMD bootloader step by step. At the same time let's deal with Promise , which recently appeared in browsers and will help us cope with asynchronous operations.

The basis of require.js is the require(dependencies, callback) function. The first argument is the list of modules to load, and the second is the function that will be called when the load is completed, with the modules in the arguments. Using Promise to write it is a snap:

 function require(deps, factory) { return Promise.all(deps.map(function(dependency) { if(!modules[dependency]) { modules[dependency] = loadScript(dependency); } return modules[dependency]; }).then(function(modules) { return factory.apply(null, modules); }); } 

')
Of course, this is not all, but there is a basis. Therefore, we will continue.

Module loading

Our first function will be the script loading function:

 function loadScript(name) { return new Promise(function(resolve, reject) { var el = document.createElement("script"); el.onload = resolve; el.onerror = reject; el.async = true; el.src = './' + name + '.js'; document.getElementsByTagName('body')[0].appendChild(el); }); } 

The download happens asynchronously, so we will return the Promise object, which will allow you to learn about the end.
The Promise constructor accepts a function in which an asynchronous process will occur. Two arguments are passed to it - resolve and reject . These are functions for reporting results. If successful, call resolve , and on error, reject .
To subscribe to the result, the Promise instance has a then method. But we may need to wait for the loading of several modules. And for this there is a special function aggregator Promise.all , which will collect several promises into one, and its result in case of success will be an array of results of loading all the necessary modules. With the help of these two simple functions, you can already get a minimally working prototype.

The github repository contains tags on key steps from this article. At the end of each chapter there is a link to github, where you can see the full version of the code written for this step. In addition, in the test folder are tests that show that our functionality works as it should. Travis-CI is connected to the project, which performed tests for each step.

See the code for this step on Github .

Ad modules

In fact, we have not loaded our modules. The thing is that when we add a script to a page, we lose control over it and do not know anything about the result of its work. Therefore, when we load the module A, we can slip the module B, but we will not notice. To prevent this from happening, you need to give the modules the opportunity to introduce themselves. The define() function is designed for this in the AMD standard. For example, the registration of module A looks like this:

 define('A', function() { return 'module A'; }); 

When the module is introduced by name, we will not confuse it with anything and will be able to mark it as loaded. To do this, we need to define the module registrar - the define function. At the last step, we simply waited for the successful loading of the script and did not check its contents. Now we will wait for a call to define . And at the time of the call, we will find our module and mark it as loaded.

To do this, we need to create a module blank at the beginning of the download, which can turn into a real module after loading. This can be done using deferred objects. They are close to Promise, but he hides the resolve and reject inside himself, and from the outside only gives the opportunity to know the result. Deferred objects have the resolve and reject methods, that is, having access to deferred, you can easily change its result. Also, the deferred object has a promise field in which the Promise is written, the result of which we specify. Deferred are easily made from Promise prescription from Stackoverflow .

When modules are loaded into require , we will create a deferred object for each module and save it to the cache (pendingModules).
define will be able to get it from there and call resolve to mark it as loaded and save it.

 function define(name, factory) { var module = factory(); if(pendingModules[name]) { pendingModules[name].resolve(module); delete pendingModules[name]; } else { modules[name] = module; } } 

It is also sometimes necessary to register a module before it is asked. Then it will not be in the pendingModules list, in this case we can immediately put it in modules.

The loadScript function will now save deferred-objects to the cache and return the promise of this object, according to which the require function will wait for the module to load.

 function loadScript(name) { var deferred = defer(), el = document.createElement("script"); pendingModules[name] = deferred; el.onerror = deferred.reject; el.async = true; el.src = './' + name + '.js'; document.getElementsByTagName('body')[0].appendChild(el); return deferred.promise; } 

See the code for this step on Github .

Dependencies in modules, loop detection

Hurray, now we can load modules. But sometimes the module may need other modules to work. For this in AMD
for the define function, another argument is provided - dependencies , which can be between name and factory.
When a module has dependencies, we cannot just take and call a factory, we must first load the dependencies.
Fortunately, for this we already have a require function, here it will have to be in place. Where previously there was just a call to factory() there will now be require :

 define(name, deps, factory) { ... - var module = factory(); + var module = require(deps, factory); ... } 

It is worth noting that now the module variable will not be a module, but the promise of a module. When we pass it to resolve, the promise of the source module will not be fulfilled, but will now wait for the dependencies to load. This is quite a convenient feature of Promise, when our asynchronous process stretches into several stages, we can resolve to transfer the Promise from the next stage, and the internal logic will recognize this and switch to waiting for the result from the new Promise.

When we load the dependencies of the modules, we are in danger. For example, module A depends on B, and B depends on A. Load module A, it will require module B. After loading, B will require A, and as a result, they will wait for each other indefinitely. The situation may be worse if there are not two modules in the chain, but more. You need to be able to stop such cyclical dependencies. To do this, we will save the download history to show a warning when we notice that our download went in a circle. We used require() to load module dependencies, but this function is a fixed set of arguments, written in the standard, it must be followed. Let's create our own internal function _require(deps, factory, path) to which we will be able to transmit information about the module load history, and in the public API we will make its call:

 function require(deps, factory) { return _require(deps, factory, []); } 

At first, our boot history will be empty, so we will pass an empty array as path. _require() will now have the same boot logic, plus history tracking.

 function loadScript(name, path) { var deferred = defer(); + deferred.path = path.concat(name); ... } function _require(deps, factory, path) { return Promise.all(deps.map(function (dependency) { + if(path.indexOf(dependency) > -1) { + return Promise.reject(new Error('Circular dependency: '+path.concat(dependency).join(' -> '))); + } ... } 

The global array with a list of all modules does not suit us, the load history of each module is different, we will save it to the deferred-object of the loadable module, so that it can then be read in define and transferred to _require if you need to load more modules. I note that we add a new module to the history via .concat() , instead of .push() , because we need an independent copy of the history in order not to spoil the history to other modules that were loaded to us. And instead of the usual throw new Error() we return Promise.reject (). This means that the promise did not come true, and an error handler will be called, just as it does when an error occurs during the loading of the script, only the message indicates another reason - a cycle in dependencies.

See the code for this step on Github .

Error processing

It is time to implement error reporting for users. The require function also provides a third argument, a function called in case of an error. Promise can tell us about an error if in .then() we pass two functions. The first one is already transmitted and called, if everything is good, the second one will be called up if something goes wrong.

An additional argument is called errback , as in the original require.js

 function _require(deps, factory, errback, path) { ... })).then(function (modules) { return factory.apply(null, modules); + }, function(reason) { + if(typeof errback === 'function') { + errback(reason); + } else { + console.error(reason); + } + return Promise.reject(reason); }); } 

In case the user does not care about errors, we will do it ourselves, we will display a message in the console. Also, it is not by chance that we return such a value in the error handler. The promise logic is designed in such a way that if we pass the function to an error case, then it considers that we will fix everything in it and we can continue working, similar to the try-catch block.
But for require.js, the loss of a module is fatal, we cannot continue to work without all modules, pass the error further using Promise.reject .

See the code for this step on Github .

Anonymous modules

The AMD standard provides the ability to define modules without a name. In this case, its name is determined by the script that is now loaded on the page. The document.currentScript property is not supported by all browsers, so we will have to define the current script in a different way. We will make the loading of the modules consistent, which means we will expect only one module at a time. Using Promise, you can easily get the implementation of a FIFO queue:

 var lastTask = Promise.resolve(), currentContext; function invokeLater(context, fn) { lastTask = lastTask.then(function() { currentContext = context; return fn(); }).then(function() { currentContext = null; }); } 

We always keep a promise from the last operation, the next operation will subscribe to its completion and leave a new promise.
We will use this queue to load scripts. Now we don’t have a pendingModules list, but one pendingModule, and the rest will be waiting.

 function loadScript(name, path) { var deferred = defer(); deferred.name = name; deferred.path = path.concat(name); invokeLater(deferred, function() { return new Promise(function(resolve, reject) { //    }); }); return deferred.promise; } 

The function still returns the deferred module, but it starts loading it not immediately, but in turn. And the module name is added to deferred to know which module we are going to wait for. And now we can write a fairly short define:

 define(function() { return 'module-content'; }); 

And the module will receive the name by the name of its file, by which we refer to it, and so it is possible not to specify it separately.

See the code for this step on Github .

Delayed initialization

When we meet define , it does not mean that it needs to be initialized immediately. Maybe no one has asked him yet and he may not need it. Therefore, it can be saved along with information about its dependencies and called only when it really comes in handy. It will also be useful if the modules can be declared in any order, and their dependencies will be dealt with at the very end, during application initialization.

Let's get a separate object predefines , and save modules in it, if nobody asked them.

 function define(name, deps, factory) { ... } else { - modules[name] = _require(deps, factory, null, []); + predefines[name] = [deps, factory, null]; } } 

And during require we will first check the predefines for the modules we are interested in.

 function _require(deps, factory, errback, path) { ... + var newPath = path.concat(dependency); + if(predefines[dependency]) { + modules[dependency] = _require.apply(null, predefines[dependency].concat([newPath])) + } else if (!modules[dependency]) { modules[dependency] = loadScript(dependency, newPath); } ... } 

This optimization will avoid unnecessary requests when we define modules in advance.

 define('A', ['B'], function() {}); define('B', function() {}); require(['A'], function() {}); 

Previously, we would have started loading the module 'B' from somewhere in the first step, and would not have found it. And now we can wait until all the modules are declared themselves, and only then call require . Also now it does not matter the order of their announcement.
It is enough that the entry point (call require) was last.

See the code for this step on Github .

Thus, we have already received a completely complete solution for loading modules and resolving dependencies. But require.js allows you to do much more with modules. Therefore, in the next part of the article, we will add support for settings and plugins, and we will make this functionality fully compatible with the original require.js.