Expressive JavaScript: Modules

Content

• Introduction
• Values, Types and Operators
• Program structure
• Functions
• Data Structures: Objects and Arrays
• Higher order functions
• The secret life of objects
• Project: e-life
• Search and error handling
• Regular expressions
• Modules
• Project: programming language
• Javascript and browser
• Document Object Model
• Event handling
• Project: Platform Game
• Drawing on canvas
• HTTP
• Forms and form fields
• Project: Paint
• Node.js
• Project: website sharing experience
• Sandbox for code

A beginner programmer writes programs in the way ants build an anthill — bit by bit, without thinking about the overall structure. His programs are like sand. They may not stand for long, but growing up, they fall apart.

Having understood the problem, the programmer spends a lot of time thinking about the structure. His programs are obtained rigidly structured, like stone sculptures. They are hard, but when they need to change, they have to commit violence.
')
A master programmer knows when a structure is needed, and when it is necessary to leave things in a simple form. His programs are like clay - hard, but pliable.

Master Yuan-Ma, Book Programming

Each program has a structure. In particular, it is determined by how the programmer divides the code into functions and blocks within these functions. Programmers are free to create the structure of their program. The structure is determined more by the taste of the programmer than by the functionality of the program.

In the case of large programs, individual functions are already lost in the code, and we need a unit for organizing large-scale code. Modules group the program code for some specific features. In this chapter, we will look at the benefits of such division and the technique of creating modules in JavaScript.

Why do we need modules

There are several reasons why authors divide their books into chapters and sections. This helps the reader to understand how the book is built, and find the parts they need. It helps the author to concentrate on each particular part. The advantages of organizing programs in several files or modules are about the same. Structuring helps people unfamiliar with the code to find what they need, and helps programmers to store things related to each other in one place.

Some programs are organized according to a plain text model, where the order of the sequence is clearly defined, and where the reader is offered a consistent study of the program and a lot of prose (comments) to describe the code. This makes reading the code less intimidating (and reading someone else's code is usually scary), but it takes a lot of effort to create the code. Also, such a program is more difficult to change, because parts of the prose are interconnected more strongly than parts of the code. This style is called literary programming. Those chapters of the book in which projects are discussed can be considered literary code.

Typically, structuring something requires energy. In the early stages of the project, when you are not yet sure where it will be, and what modules are generally needed, I advocate a structureless minimalist code organization. Just place all the parts where it is convenient, until the code stabilizes. Thus, you do not have to spend time on rearranging pieces of the program, and you will not catch yourself in such a structure that is not suitable for your program.

Namespace

Most modern PLs have intermediate visibility areas (OBs) between the global (visible to all) and local (only this function is visible). JavaScript doesn't have this. By default, everything that needs to be seen outside the top-level function is in the global OB.

Namespace pollution (PI), when unrelated parts of the code share one set of variables, was mentioned in Chapter 4. There, the Math object was cited as an example of an object that groups functionality related to mathematics as a module.

Although JavaScript does not directly provide constructs for creating a module, we can use objects to create namespaces accessible from anywhere. And functions can be used to create isolated private namespaces inside a module. A little further we will discuss the method of constructing fairly convenient modules that isolate the UI with the help of basic language concepts.

Reuse

In a project that is not divided into modules, it is not clear which parts of the code are necessary for a particular function. In my program, spying on enemies (Chapter 9), I wrote the function of reading settings files. If I need to use it in another project, I will have to copy the parts of the old program that seem to be connected with this function to my new project. And if I find an error there, I will correct it only in the project I am working on at the moment, and I will forget to correct it in everyone else.

When you have a lot of such duplicated pieces of code, you will find that you spend a lot of time and effort on copying and updating them. If you place interconnected parts of programs into separate modules, they will be easier to monitor, correct and update, because wherever this functionality is required, you can simply load this module from a file.

This idea can be used even better if the relationships between different modules are clearly defined — who depend on whom. Then you can automate the process of installing and updating external modules (libraries).

If you further develop the idea - imagine an online service that tracks and distributes hundreds of thousands of such libraries, and you can search for the functionality you need among them, and when you find it, your project will automatically download it.

And there is such a service! It is called NPM (npmjs.org). NPM is an online module database and tool for downloading and upgrading modules on which your program depends. it grew out of Node.js, a JavaScript environment that does not require a browser, which we will discuss in Chapter 20, but can also be used in browser programs.

Decoupling

Another task of the modules is to isolate unrelated parts of the code in the way that object interfaces do. A well thought out module provides an interface for its use outside. When a module is updated or repaired, the existing interface remains unchanged so that other modules can use the new, updated version without changes to themselves.

A stable interface does not mean that new functions, methods or variables are not added to it. The main thing is that the existing functionality is not removed and its meaning does not change. A good interface allows the module to grow without breaking the old interface. And this means - to expose as little as possible of the internal kitchen of the module, and the interface language must be sufficiently flexible and powerful for use in various situations.

Interfaces that perform a simple task, like reading settings from a file, come out in such a natural way. For others, for example, for a text editor, which has many different aspects that require access from the outside (content, styles, user actions, etc.), the interface must be carefully considered.

Using functions as namespaces

Functions are the only thing in JavaScript that creates a new scope. If we need modules to have their own scope, we have to base them on functions.

Notice the simplest module that associates names with the numbers of the days of the week — as the Date object's getDay method does.

var names = ["", "", "", "", "", "", ""]; function dayName(number) { return names[number]; } console.log(dayName(1)); // →  

The dayName function is part of the module interface, and the names variable is not. But I would like not to pollute the global namespace.

You can do this:

 var dayName = function() { var names = ["", "", "", "", "", "", ""]; return function(number) { return names[number]; }; }(); console.log(dayName(3)); // →  

Now names is a local variable of an unnamed function. The function is created and immediately called, and its return value (the dayName function we already need) is stored in a variable. We can write many pages of code in a function, declare there hundreds of variables, and all of them will be internal for our module, and not for external code.

A similar pattern can be used to isolate code. The following module writes a value to the console, but does not provide any values ​​for use by other modules.

 (function() { function square(x) { return x * x; } var hundred = 100; console.log(square(hundred)); })(); // → 10000 

This code displays hundreds of squares, but in reality it could be a module that adds a method to some kind of prototype, or a custom widget on a web page. It is wrapped in a function to prevent global pollution from the variables it uses.

Why do we enclose the function in parentheses? This is due to the JavaScript syntax glitch. If an expression starts with the function keyword, this is a functional expression. And if the statement starts with function, this is a declaration of a function that requires a name, and since it is not an expression, it cannot be called with brackets () after it. You can think of bracing as a trick so that the function is forcibly interpreted as an expression.

Objects as interfaces

Imagine that we need to add another function to our day of the week module. We can no longer return a function, but must wrap the two functions in an object.

 var weekDay = function() { var names = ["", "", "", "", "", "", ""]; return { name: function(number) { return names[number]; }, number: function(name) { return names.indexOf(name); } }; }(); console.log(weekDay.name(weekDay.number("Sunday"))); // → Sunday 

When a module is large, it is inconvenient to collect all the returned values ​​into the object at the end of the function, because many return functions will be large, and it would be more convenient for you to write them somewhere else, next to their associated code. It is convenient to declare an object (usually called exports) and add properties to it every time we need to export something. In the following example, the module function takes an interface object as an argument, allowing the code outside the function to create it and store it in a variable. Outside of the function, this refers to the global scope object.

 (function(exports) { var names = ["", "", "", "", "", "", ""]; exports.name = function(number) { return names[number]; }; exports.number = function(name) { return names.indexOf(name); }; })(this.weekDay = {}); console.log(weekDay.name(weekDay.number("Saturday"))); // → Saturday 

Disconnect from global scope

This template is often used in JavaScript modules intended for the browser. The module will take one global variable and wrap its code into a function so that it has its own namespace. But there are problems with this template when many modules require the same name, or when you need to load two versions of a module at the same time.

By tweaking something, we can make a system that allows one module to access the interface object of another, without accessing the global OB. Our goal is the require function, which, getting the name of the module, will load its file (from disk or from the network, depending on the platform) and return the corresponding value with the interface.

This approach solves the problems mentioned earlier, and it has another advantage - the dependencies of your program become apparent, and therefore it is more difficult to accidentally call a module you do not need without a clear declaration.

We need two things. First, the readFile function, which returns the contents of the file as a string. In standard JavaScript, there is no such function, but different environments, such as a browser or Node.js, provide their own ways of accessing files. For now, pretend that we have such a function. Secondly, we need the ability to execute the contents of this line as code.

We execute data as code

There are several ways to get data (a line of code) and execute it as part of the current program.

The most obvious is the eval statement, which executes a line of code in the current environment. This is a bad idea - it violates some of the properties of the environment that it usually has, such as isolation from the outside world.

 function evalAndReturnX(code) { eval(code); return x; } console.log(evalAndReturnX("var x = 2")); // → 2 

The best way is to use the Function constructor. It takes two arguments - a string containing a comma-separated list of argument names, and a string containing the function body.

 var plusOne = new Function("n", "return n + 1;"); console.log(plusOne(4)); // → 5 

This is what we need. We will wrap the module code into a function, and its scope will become the scope of our module.

Require

Here is the minimum version of the require function:

 function require(name) { var code = new Function("exports", readFile(name)); var exports = {}; code(exports); return exports; } console.log(require("weekDay").name(1)); // →  

Since the new Function constructor wraps the module code into a function, we don’t need to write a function that wraps the namespace inside the module itself. And since exports is an argument to the module’s function, the module does not need to declare it. This removes a lot of garbage from our example module.

 var names = ["", "", "", "", "", "", ""]; exports.name = function(number) { return names[number]; }; exports.number = function(name) { return names.indexOf(name); }; 

When using such a template, a module usually begins with the declaration of several variables that load the modules on which it depends.

 var weekDay = require("weekDay"); var today = require("today"); console.log(weekDay.name(today.dayNumber())); 

This simple option of require has flaws. First, it will load and execute the module each time it is loaded through require — if several modules have the same dependencies, or the require call is inside a function that is called repeatedly, time and energy will be lost.

This can be solved by storing already loaded modules in the object, and returning the existing value when it is loaded several times.

The second problem is that the module cannot export a variable directly, only through the export object. For example, a module may need to export only the constructor of the object declared in it. This is now impossible, as require always uses the exports object as a return value.

The traditional solution is to provide modules with a different variable, module, which is an object with the exports property. It initially points to an empty object created by require, but can be overwritten with another value to export something else.

 function require(name) { if (name in require.cache) return require.cache[name]; var code = new Function("exports, module", readFile(name)); var exports = {}, module = {exports: exports}; code(exports, module); require.cache[name] = module.exports; return module.exports; } require.cache = Object.create(null); 

Now we have a system of modules that use the same global require variable to allow modules to search for and use each other without going into the global scope.

This style of the module system is called CommonJS, after the pseudo-standard that first described it. It is embedded in the Node.js system. These implementations do much more than what I have described. The main thing is that they have a smarter way to go from the name of the module to its code, which allows loading modules by a relative path to the file, or by the name of the module pointing to locally installed modules.

Slow loading of modules

Although it is possible to use the CommonJS style for the browser, it is not very suitable for this. Downloading a file from the web is slower than from a hard disk. While the script is working in the browser, nothing else happens on the site (for reasons that will become clear to chapter 14). So, if every require call downloaded something from a distant web server, the page would hang for a very long time when loading.

You can get around this by running a Browserify program with your code before posting it on the web. It will look at all calls to require, will handle all dependencies and compile the necessary code into one big file. The website simply loads this file and gets all the necessary modules.

The second option is to wrap the module code in a function, so that the module loader first loads the dependencies in the background, and then calls the function that initializes the module after loading the dependencies. This is done by the AMD system (asynchronous module definition).

Our simple dependency program would look like this at AMD:

 define(["weekDay", "today"], function(weekDay, today) { console.log(weekDay.name(today.dayNumber())); }); 

The define function here is the most important. It takes an array of module names, and then a function that takes one argument for each of the dependencies. It will load the dependencies (if they are not already loaded) in the background, allowing the page to work while the file is being downloaded. When everything is loaded, define calls the function given to it, with the interfaces of these dependencies as arguments.

Modules loaded in this way must contain define calls. As their interface, it uses what was returned by the function passed to define. Here is the weekDay module:

 define([], function() { var names = ["", "", "", "", "", "", ""]; return { name: function(number) { return names[number]; }, number: function(name) { return names.indexOf(name); } }; }); 

To show the minimal implementation of define, pretend that we have a backgroundReadFile function that takes a file name and a function, and calls this function with the contents of this file as soon as it is loaded. (Chapter 17 will explain how to write such a function).

To track the modules while they are loaded, define uses objects that describe the state of the modules, tells us if they are already available, and provides their interface for accessibility.

The getModule function takes a name and returns such an object, and makes sure that the module is queued for loading. It uses the caching object to not load one module twice.

 var defineCache = Object.create(null); var currentMod = null; function getModule(name) { if (name in defineCache) return defineCache[name]; var module = {exports: null, loaded: false, onLoad: []}; defineCache[name] = module; backgroundReadFile(name, function(code) { currentMod = module; new Function("", code)(); }); return module; } 

We assume that the uploaded file also contains a call to define. The currentMod variable is used to inform this call about the module object being loaded so that it can update this object after loading. We will return to this mechanism.

The define function itself uses getModule to load or create module objects for the dependencies of the current module. Its task is to schedule the launch of the moduleFunction function (containing the module code itself) after loading dependencies. To do this, it defines the whenDepsLoaded function that is added to the onLoad array, which contains all the dependencies that are not yet loaded. This function immediately stops working if there are still unloaded dependencies, so that it does its work only once the last dependency is loaded. It is also called immediately from define itself, in the case when no dependencies need to be loaded.

 function define(depNames, moduleFunction) { var myMod = currentMod; var deps = depNames.map(getModule); deps.forEach(function(mod) { if (!mod.loaded) mod.onLoad.push(whenDepsLoaded); }); function whenDepsLoaded() { if (!deps.every(function(m) { return m.loaded; })) return; var args = deps.map(function(m) { return m.exports; }); var exports = moduleFunction.apply(null, args); if (myMod) { myMod.exports = exports; myMod.loaded = true; myMod.onLoad.every(function(f) { f(); }); } } whenDepsLoaded(); } 

When all dependencies are available, whenDepsLoaded calls the function that contains the module, passing dependency interfaces as arguments.

The first thing that define does is to keep its currentMod value, which it had when it was called, in the variable myMod. Recall that getModule, right before executing the module code, saved the corresponding module object in currentMod. This allows whenDepsLoaded to store the return value of the module's function in the exports property of this module, set the loaded property of the module to true, and call all functions waiting for the module to load.

This code is harder to learn than the require function. Its implementation is not in a simple and predictable way. Instead, several operations must be performed at uncertain times in the future, making it difficult to learn how this code is executed.

This AMD implementation is much smarter at turning module names into URLs and more reliable than the example. The RequireJS project provides a popular implementation of this style of module loader.

Interface Development

The development of interfaces is one of the most delicate points in programming. Any non-trivial functionality can be implemented in a variety of ways. Finding a working method requires insight and forethought.

The best way to know the value of a good interface is to use many interfaces. Some will be bad, some will be good. Experience will show you what works and what doesn't. Never take for granted a bad interface. Fix it, or wrap it in another interface that suits you best.

Predictability

If a programmer can predict how your interface works, he will not often have to be distracted and look for a hint on how to use it. Try to follow generally accepted conventions. If there is a module or part of the JavaScript language that does something similar to what you are trying to implement, it would be nice if your interface resembles an existing one. Thus, it will be familiar to people familiar with the existing interface.

Predictability is also important in the behavior of your code. You may be tempted to make the interface too abstruse, ostensibly because it is more convenient to use. For example, you can take any kind of argument type or combination of arguments and do “what you need” with them. Or provide dozens of specialized functions that offer slightly different functionality. This can make the code based on your interface a bit shorter, but it can make it difficult for people working with it to build a clear mental model of how your module works.

Composability

Try to use as simple data structures as possible in interfaces. Make it so that functions perform simple and understandable things. If applicable, make the functions clean (see Chapter 3).

For example, often modules offer their version of a massive collection of objects with their own interface for counting and retrieving elements. Such objects do not have map or forEach methods, and no function that expects a real array can work with them. This is an example of poor composability - the module cannot be easily linked with other code.

An example is the module for spelling verification of text, which can be useful in a text editor. The test module can be made so that it works with any complex structures used by the editor itself and calls the internal functions of the editor to provide the user with a choice of writing options. If you do this, the module cannot be used with other programs. On the other hand, if we define the interface of the validation module, which accepts a simple string and returns the position at which there is a possible error in the line, and in addition the array of proposed amendments, then we will have an interface that can be linked with other systems, because the strings and arrays are always available in javascript.

Multilayer interfaces

Developing an interface for a complex system (for example, sending an email), you often come to a dilemma. On the one hand, it is not necessary to overload the user interface with details. Do not force them to study it for 20 minutes before they can send an email. On the other hand, I don’t want to hide all the details - when people need to do something complicated with the help of your module, they should have such an opportunity.

Often it is necessary to offer two interfaces: detailed low-level for difficult situations, and simple high-level for normal use. The second can be built on the basis of the first. In the module for sending emails, the high-level interface may simply be a function that receives the message, the address of the recipient and the sender, and sends the letter. Low-level should give access to headers, attached files, HTML letters, etc.

Total

Modules allow you to structure large programs, dividing the code into different files and namespaces. If you provide them with well-designed interfaces, they will simply be used, applied in other projects and continue to be used in the development and evolution of the project itself.

Although JavaScript does not help making modules at all, its flexible functions and objects make it possible to make a fairly good system of modules. The scope of functions is used as the internal namespace of the module, and objects are used to store sets of variables.

There are two popular approaches to using modules. One is CommonJS, built on the require function, which calls modules by name and returns their interface. The other is AMD, which uses the define function, accepts an array of module names and, after loading them, executes a function whose arguments are their interfaces.

Exercises

Names of the months

Write a simple weekday module that converts the number of months (starting from zero) into names and back. Give him your own namespace, because it will need an internal array with the names of the months, and use pure JavaScript, without a module loading system.

 //   console.log(month.name(2)); // → March console.log(month.number("November")); // → 10 

Back to e-life

I hope that Chapter 7 has not yet been erased from your memory. Go back to the system developed there and suggest a way to divide the code into modules. To refresh your memory - here is a list of functions and types, in order of appearance:

 Vector Grid directions directionNames randomElement BouncingCritter elementFromChar World charFromElement Wall View WallFollower dirPlus LifelikeWorld Plant PlantEater SmartPlantEater Tiger 

Do not create too many modules. A book in which a new chapter would appear on every page would get on your nerves (if only because the headlines would have eaten up all the space). No need to do ten files for one small project. Count on 3-5 modules.

Some functions can be made internal, inaccessible from other modules. The correct option does not exist here. The organization of the modules is a matter of taste.

Circular dependencies

A tricky topic in dependency management is circular dependencies, where module A depends on B and B depends on A. Many module systems simply forbid this. CommonJS modules allow a limited option: this works until the modules do not replace the exports object, which exists by default, with a different value, and begin to use each other’s interfaces only after the download is complete.

Can you think of a way to enable the implementation of a system for supporting such dependencies? Look at the definition of require and think about what you need to do this function for this.