Loading CommonJS modules in the browser without changing the source code

One day, sitting at a computer and pondering my next worthless undertaking, I suddenly realized that I needed a way to use the same code on the browser side and on the server side. I almost immediately guessed that for sure I was not the first to be so clever, and everything had been invented long ago for me - and I was not mistaken.

Indeed, my requirements were great, for example, RequireJS with its adapter for Node.js , which for some time successfully met my whims, until I again had a brilliant idea: “Why do I have to use porridge from two completely different module formats in one project? It is necessary to unify everything! ”
')
And again, the answer was not long in coming, there were a million browser implementations of CommonJS modules: all sorts of script skleyschiki, and server preprocessors, and synchronous loaders, and asynchronous - everything your heart desires. But they all ended up with one very important flaw. They somehow modified the source code of scripts and made the process of debugging them in browser inspectors very inconvenient.

We think

According to the specification, the global namespace of each module should be global only within the boundaries of the module itself. Beyond it, it should not spread. If such a module is loaded into the browser in an unchanged form, its code will be executed in the global namespace and will share it along with all other modules. To avoid this, the code for each module is usually wrapped in a function, and the module's namespace thus becomes local.

You can wrap the module in a function either on the server before loading it into the browser, or in the browser itself. In both cases, the executable code will be different from the source, which makes it difficult to debug.

We will go the other way. The essence of our method is that we will load each module into a separate iframe, thereby isolating it from the other modules. In the namespace of each such frame, the require function and the exports and module.exports objects will be predefined, as required by the CommonJS specification .

This method of loading scripts, unfortunately, was not without flaws. The first thing I encountered was the inconvenience of working with the DOM of the parent window and other global objects. To access them, you need to use the cumbersome window.parent.window construction, which, moreover, will be unnecessary if in the future we want to glue our modules for production. The solution to this problem, in some way, will be the creation of a global object in each frame, which will be a reference to the window of the parent window. Through this object we will be able to access from our modules to such things as the window itself, document, navigator, history itself, and so on, as well as, if necessary, to use global variables.

The second, not so obvious at first glance, disadvantage was the non-identity of global constructor functions (classes) Function, Date, String, etc. in contexts of different modules. This will not allow us, for example, to check whether an object belongs to any built-in class, if it was created in another module.

var doSomething = require("./someModule").doSomething; // doSomething -  ,    someModule console.log(doSomething instanceof Function); // false,   Function    Function  someModule (    someFunction) -    

This problem cannot be solved transparently; therefore, it is necessary to accept an agreement not to use constructions like the one mentioned above in the code. Or use them neatly. Specifically, in this example, the test for the belonging of a function to a class Function can be replaced, for example, in this way:

 console.log(typeof doSomething === "function"); // true 

Another nuance that makes life difficult for people who want to load their CommonJS modules into the browser is the synchronous nature of the CommonJS require function. The most common way to solve this problem is to load the required module with a synchronous AJAX request and then eval the loaded code, or create an anonymous function using new Function (). This method does not suit us, as the debugger in this case will stop pointing to the lines of code in the original file. We will again go the other way, which will allow us to run without problems with a debugger through the code untouched by the ruthless Eval.

The require function essentially only returns a cached module.exports object, which is exported by the loaded module. The code of the module itself is executed only once during the first attempt to load the module.

Bearing in mind the above, let’s go on a little trick - we will load our modules in advance, before the code that these modules will use is executed. At the same time, we will cache exports of all modules somewhere, from where our absolutely synchronous require function will return them.

Of course, this method is also not without flaws. In order to load all modules in advance, we need to know their identifiers (names) absolutely exactly. And this means that using our method, we will not be able to preload those modules whose identifiers are calculated during the execution of the application. That is, we can not do this:

 var a = someRandomValue(); require("./module" + a); 

However, to solve this problem, you can use for such cases, the usual AJAX-loading module and eval'om with all the ensuing consequences.

There is still a problem, which consists in the fact that the order of code execution in modules will differ from that in, for example, in Node.js conditions. Consider two small modules:

 //  "a" exports.result = doSomeExternalOperation(); 

 //  "b" prepareDataForSomeExternalOperation(); var a = require("./a"); 

In Node.js, obviously, the prepareDataForSomeExternalOperation function call will occur earlier than the doSomeExternalOperation call (but only if there have not been any other calls to require ("./ a") before). In our case, everything will be the opposite, since module a is loaded and executed before module b. With this drawback, we, unfortunately, will also have to put up. But in fairness it should be said that with the correct design of the modules of such situations should not arise. It is not good to execute any external actions in the main module code (for example, in the file system or some database) that implicitly affect the operation of other modules.

Here, in general, and everything that I wanted to tell about loading of modules in the browser. If someone described the method seemed useful and he decides to use it in his work (good, everything is quite elementary, and implementation difficulties should not arise), I will be immensely happy.

But if you suddenly have nothing to do now, or just curious, welcome to the description of the details of my implementation!

Code

Sources are freely available on Github .

I decided to format this utility in the form of a class, each instance of which will be a separate application with its own configuration (though so far the entire configuration consists of a single line - the path where all our scripts lie) and the module cache:

 function Comeon(path) { var self = this; self.path = path; self.modules = {}; } 

The class has the only public asynchronous require function, which starts the execution of a module with all its dependencies, and optionally accepts a callback function, which will be called after the main module completes its work and will receive its export parameter.

 Comeon.prototype.require = function require(moduleRequest, callback) { var self = this; loadNextModule.bind(self)(enqueueModule.bind(self)(getModuleId("", moduleRequest)), callback); } 

Before we consider the two main and most interesting functions enqueueModule and loadNextModule, consider a few auxiliary ones.

The searchRequires function accepts the URL parameter of the module file, loads it with a synchronous XHR request, and searches for the occurrences of calls to the require function in it. I want to note that we do not execute the loaded code, but just look for dependencies of the module using this function. The module file during this download is cached by the browser, which later will come in handy when connecting this module.

 var requirePattern = /(?:^|\s|=|;)require\(("|')([\w-\/\.]*)\1\)/g; function searchRequires(url) { var requires = []; var xhr = new XMLHttpRequest(); xhr.open("GET", url, false); xhr.onreadystatechange = function () { if ((xhr.readyState === 4) && (xhr.status === 200)) { var match; while ((match = requirePattern.exec(xhr.responseText)) !== null) { requires.push(match[2]); } } }; xhr.send(); return requires; } 

The functions getModuleId and getModuleContext are used to obtain the identifier and the path to the module, respectively.

 function getModuleId(moduleContext, moduleRequest) { var moduleId = []; (/^\.\.?\//.test(moduleRequest) ? (moduleContext + moduleRequest) : moduleRequest).replace(/\.(?:js|node)$/, "").split("/").forEach(function (value) { if (value === ".") { } else if (value === "..") { moduleId.pop(); } else if (/[\w\-\.]+/.test(value)) { moduleId.push(value); } }); return moduleId.join("/"); } 

 function getModuleContext(moduleId) { return moduleId.slice(0, moduleId.lastIndexOf("/") + 1); } 

The require function is the same function that, in the context of the modules, will return the requested cached exports. This function, having previously nailed to the context of the instance of our application and passing the first parameter to the path of the current module, we will put in the window of each frame.

 function require(moduleContext, moduleRequest) { var self = this; var moduleId = getModuleId(moduleContext, moduleRequest); if (self.modules[moduleId] && self.modules[moduleId].exports) { return self.modules[moduleId].exports; } else { throw Error("Module not found."); } } 

Well, and finally, consider two functions that perform all the basic work.

The recursive function enqueueModule adds the module passed as a parameter to the queue, and, by calling itself for each of the dependencies, it also adds them. As a result, we get a queue of module loading, at the very end of which there will be a main module - the entry point to the application. Thanks to this queue, each loadable module will already have at its disposal all cached modules on which it depends.

 function enqueueModule(moduleId) { var self = this; var moduleQueue = []; if (!self.modules[moduleId]) { self.modules[moduleId] = { url: self.path + moduleId + ".js?ts=" + (new Date()).valueOf() }; moduleQueue.push(moduleId); searchRequires(self.modules[moduleId].url).forEach(function (value) { Array.prototype.push.apply(moduleQueue, enqueueModule.bind(self)(getModuleId(getModuleContext(moduleId), value))); }); } return moduleQueue; } 

The loadNextModule function runs through the queue returned by the enqueueModule function and loads our modules into the browser in order (the browser will take the files from its cache, since we already loaded them to look for dependencies). To connect each module, as we agreed above, a separate iframe is used, in which we create global, exports and module.exports variables, as well as the require function. Each next iframe is loaded only after a full download of the previous script. When the load queue comes to an end, we call the callback function passed at the very beginning, if there is one, and transfer to it the export of the last module.

 function loadNextModule(moduleQueue, callback) { var self = this; if (moduleQueue.length) { var iframe = document.createElement("iframe"); iframe.src = "about:blank"; iframe.style.display = "none"; iframe.onload = function () { var moduleId = moduleQueue.pop(); var iframeWindow = this.contentWindow; var iframeDocument = this.contentDocument; iframeWindow.global = window; iframeWindow.require = require.bind(self, getModuleContext(moduleId)); iframeWindow.module = { exports: {} } iframeWindow.exports = iframeWindow.module.exports; var script = iframeDocument.createElement("script"); script.src = self.modules[moduleId].url; script.onload = function () { self.modules[moduleId].exports = iframeWindow.module.exports; if (moduleQueue.length) { loadNextModule.bind(self)(moduleQueue, callback); } else if (typeof callback === "function") { callback(self.modules[moduleId].exports); } }; iframeDocument.head.appendChild(script); }; document.body.appendChild(iframe); } else if (typeof callback === "function") { callback(); } } 

As a bonus, let's add functionality that allows us to start our application immediately after downloading the comeon.js script.

 var script = Array.prototype.slice.call(document.getElementsByTagName("script"), -1)[0]; var main = script.getAttribute("data-main"); if (main) { window.addEventListener("load", function () { var comeon = new Comeon(script.getAttribute("data-path") || "/"); comeon.require(main); }); } 

That's all. Now we can use the modules written in CommonJS format on the browser side and debug them at your pleasure. To do this, we just need to connect comeon.js with the path to the scripts and the name of the main module in the data-attributes:

 <script src="http://rawgithub.com/avgaltsev/comeon/master/comeon.js" data-path="scripts/" data-main="main"></script> 

Or, if during execution you need to connect several modules independent of each other, or if the application has several entry points, you can use a more verbose syntax:

 <script src="http://rawgithub.com/avgaltsev/comeon/master/comeon.js"></script> <script> window.onload = function () { var comeon = new Comeon("scripts/"); //   comeon.require("main"); //    comeon.require("another_main", function (exports) { console.log(exports); }); }; </script>