πŸ“œ ⬆️ ⬇️

Scalable JavaScript applications

More than a month ago in the article FAQ on JavaScript: ask questions the question was β€œTell me examples of a good approach to organizing JS code to a site at a high level. How can I learn more about the practice of implementing for example gmail? ”.

It's time to answer this question. I tightened a bit. I wanted to tell a report on the topic of the same name on J. Subbotnik. The report was very short many important points had to be thrown out. Article - more or less complete version.

This article is about how to make a large web application extensible and supported: architecture, approaches, rules.

If you are working on a web application, then most of the time you spend on writing code, fixing errors. Only a small part is spent on writing new functionality. Any complex web application is constantly changing. Today - 1 line, tomorrow - 20, the day after tomorrow - 300.
')
Let's see what kind of code is on the sites:
$(function () { //     $('#button').click(function (event) { alert(this.innerHTML); }); $('#list').uberScrollerPluginStart({ "theme": "red" }); $('#lazy_thing').click(function () { $.get('/lazy/thing/body.html', doLazyLoad.bind(this)); }); /*       */ }); 

More often it is jQeury, we hang events, connect plugins, execute ajax requests. If any plugin is removed, then everything will break. We get a kind of tangle of code in which frameworks, plugins, our code are mixed. Such a tangle is not very large (lines 100) and, as a rule, is created and maintained by one person. Most sites are created 1 time and not supported at all, so for them something more can be harmful and can increase the cost of the entire site.



If we apply this architecture to GMail, Yandex.Mail, the Yahoo! Twitter portal, we will get a huge tangle of code (10,000+ lines) that creates several people. A huge tangle is very difficult to unravel, and even more confusing to mess up, so as not to break anything. Web applications are constantly evolving, so this tangle has to constantly unravel and confuse.
The site code is not structured, and its architecture is strongly connected. For web applications, this architecture cannot be used.

Architecture


Consider one of several architectures that allows you to easily create scalable applications. The architecture is taken from NC Zakas, I like it very much (it will be great if you watched his presentation and remember what it is about), I will change it a bit in the course, and you will see the end result in the examples.

Framework

In any application, as a rule, the framework enters. All client frameworks be jQuery, Mootools, YUI, dojo - this is just a toolbox. Tools help you hammer in nails, saw boards. Some tools are very necessary, there are and which are gathering dust. If the basic tools are few, then heavier ones are connected, for example, Backbone.js + Underscore.js
In life, replacing the jQuery box with Mootools is easy. Think that you will be abandoned by jQuery or your favorite library now? In order for the replacement to be easy, you need to add a wrapper over the library functions - it can be the Application Core.

Core

In addition to wrapping libraries, the kernel also has other functions: it manages the life of parts of the system, provides a communication interface, controls the errors of the entire application.

Modules

An application consists of modules β€” these are independent parts of the application that are managed by the Core, but have no direct connection with the Core itself. A heavy javascript application is just as complex as a space station. The ISS has an extensible architecture in it dozens of modules, each fulfilling its own role. Station modules were made in different countries, delivered at different times and there are a lot of them, but they all work as a single unit.

Web application modules consist of HTML + CSS + JavaScript + Resources

Module resources - localization, handles and other private data of the module.

Each module must live separately from the entire application. The task of each module is to perform a narrowly targeted function. The modules should be separate and the less the module knows about other modules, the easier it will be to support the entire application. When you create a new code and add dependencies, think about what you need to throw it tomorrow.

To ensure weak connectivity and to restrict the freedom of a module, it is necessary to enclose it with a special medium object β€” a sandbox. Each module must be inside its sandbox and communicate only with it. This is the only object that the module knows about. The role of the sandbox is simple. She acts as a security guard - she knows what the module can do, knows with whom the module can communicate. Sandbox provides communication module with the kernel.

A module can call its methods and sandboxes, use its HTML element. The module must ask permission before performing any action. The module is forbidden to create globals, use non-standard globals, communicate with other modules directly.

Submodules

Each module acts as a micro-application, which can have its own rules, the rules should not contradict the global one. Each module can delegate its sandbox to its parts, connect other modules (during project assembly).

It is not important for the kernel how the module will manage its parts - it is important for it that the global rules for each module are respected.

Subordination


To reduce the number of connections within the system, we need hard subordination.

- Only the library knows about the browser and the existing API
- Only the core knows about the library
- Only the sandbox knows about the core
- Each module knows only about its sandbox

None of the objects should be aware of the entire application.



Our application must be constantly changing. Users want a new functional - we add it. Browsers get a new feature - we fasten it.

Therefore, each object must change
- The library is expanded by plugins - The browser gets a new API - add a plugin
- The kernel is updated due to extensions - Replaced the protocol from XML to JSON, changed the format of the sent data, changed the AJAX transport - added the extension
- The whole application is expanded with modules - Users want a new function - we add a module

Communication


Everyone knows that the HTML DOM is replete with events. Events are everywhere - there are elements, there is an API (XHR, Workers). DOM events allow the document object model to infinitely expand and create extensible applications. I believe that events are the best base for web applications.

Consider an example:
 var Module1 = { "someAction": function () { Module2.getSomeValue(); } }; var Module2 = { "getSomeValue": function () { return 'data'; } }; 

In a conventional pattern, modules communicate with each other directly. Module 1 depends on module 2 and on its getSomeValue() method. If we remove module 2, then everything will break.

If the method call is replaced by an event, the modules will become independent (they will be loosely coupled).
 //   var Module1 = { "init": function ($) { $.on('event', function (e) { // $ -  jQuery,   sandbox console.log(e.data); }); } }; var Module2 = { "someAction": function ($) { // $ -  jQuery $.trigger('event', 'data'); } }; 

In module 1, we listen to the event event, module 2 causes an event event with some data and the event handler draws data to the console

Yes, the architecture is changing a lot, but we get rid of strong connectivity.

Asynchronous functions


Events inevitably entail asynchrony, I think everyone knows that HTML DOM is replete with asynchronous methods. XHR, JSONP, sending data from a frame to a frame, sending data to the worker and back. Asynchronous programming is more complicated and its use is not always justified. But in our case, asynchronous functions can be incredibly useful.

Let's see an example:
 //   var Storage = { "read": function (key) { return localStorage[key]; } }; var data = Storage.read('key'), pData = process(data); $.trigger('data', pData); 

Suppose we store some user data in localStorage , work with it synchronously and everything is fine. Well, as long as we did not want to keep the same data on the server, the probability that we decide to change something is very high. Translation of synchronous code to asynchronous can be a big problem, even with an event model.

Rewrite our code with replacing localStorage:
 //   var Storage = { "read": function (key, cb) { $.get('/read/' + key, cb); } }; Storage.read('key',function(data) { var pData = processData(data); $.trigger('data', pData); }.bind(this)); 

We had a simple data retrieval function and 10 lines of source code. We added 1 line and changed 4 lines. Received almost 50% of changes (and that I considered brackets). If it were not for our events, then we had to change the code that uses Storage. Using the asynchronous approach in the functions of receiving and storing data, we save ourselves from some of the problems in the future.

As a rule, all asynchronous methods are associated with sending and receiving data (XHR, Location API, File API), so all the functions of storing and receiving data are best made asynchronous or connect events.

Advantages of such an architecture

These are excerpts from the report from J.Subbotnik . At a volunteer I promised to illustrate the theory with practice. In theory, everything is beautiful, but theory without practice is of little value, so let's create an application based on our architecture.

Example: Scalable JavaScript application that is easy to maintain.


We will create an application, more precisely, an application stub (the article turns out to be very large, therefore I will shorten this part), demonstrating modularity. The application includes:Part of the examples associated with the assembly of the project, the automatic generation of unit tests and unit testing, I will show on my fingers so as not to overload the article.

Libraries and Technologies

As usual, there will be jQuery. To systematize the layout we will use the simplified BEM technology: Block, Element, Modifier. For dynamic script loading we will use $ script.js. Templates - Modified template engine from Rezig.

Project structure

  /app /css -        /blocks /b-module-name /b-module-name.css ... ... /pages /index.css ... /descriptors -   /ModuleName.json ... /locales -   /ModuleName.json ... /modules -   /ModuleName.js ... /templates -   /ModuleName.html ... /views -   /ModuleName.html ... /build -    /lib -   /Core.js /test -  /lib /qunit.css /qunit.js /ModuleName -   ModuleName /index.html /index.js /TestData.js -     /vendors -   /Script.js /jQuery.js ... /index.html -   /index.js -  js ,    /index.json -   ... 

Modules

Let's start first with the modules. Each part of the module must be connected dynamically and statically (when assembled into a single file), or both. Everything should be as transparent as possible for the developer: there is a module β€” we use, no β€” we load and use.

Our modules will consist of several parts:
  1. Layout sample: HTML file with connected styles. This file will be used in the unit test, it shows how the module looks like. Samples can be taken from it to create a template.
  2. Template: The HTML blocks that the module uses.
  3. Styles and images: simple CSS file or files, images
  4. JavaScript: module code
  5. Descriptor: JSON file containing the name of the module, its settings and a list of those events that can listen and generate. It is used to automatically generate a Unit Test, it is used by the sandbox to differentiate rights.
  6. Localization: JSON file containing texts in different languages
Each module has an adequate name, each part of the module is located in a separate directory and has the same name as the module. We have maximally separated all the logical parts of the module (Markup / Template, View, Logic, Description and Configuration, Texts). Each module exports only 2 methods: init and destroy:

Module Example (DataGenerator Module)

 (function(global){ "use strict"; var intervalId; var DataGenerator = { init: function (sandbox) { intervalId = setInterval(function () { sandbox.trigger('newData', Math.random()); }, sandbox.getResource('interval')); }, destroy: function () { clearInterval(intervalId); } }; //  if (!global) { return DataGenerator; } if (!global.exports) { global.exports = {}; } global.exports.DataGenerator = DataGenerator; }(this)) // ;  ! 

I agree that a lot of garbage in the code. We have exactly this format because of the requirements for the module: The kernel authoritatively connects the modules by itself, can be connected both statically and dynamically. There are no modules in JavaScript as such, so everyone creates their own look. There are some " standard ", but more often - this is a bike for specific tasks.

Descriptor Example (DataGenerator Module)

 { "name": "DataGenerator", "acl": { "trigger:newData": true //     newData }, "resources": { "interval": 1000 } } 

Locale example (MessageView module)

 { "text_label": { "ru": " : ", "en": "He said: " } } 

Template example (MessageView module)

 <div class="b-message-view"> <span class="b-message-view__label">{%=label%}</span> <span class="b-message-view__value">{%=value%}</span> </div> 


Advantages of this format
Each logical part is separated. We can use each part repeatedly. For example, we can use the descriptor for automatic generation of skeleton unit tests.

Core

We need to load and register modules. We need to be able to do it both at assembly and dynamically. jQuery Deffered will help us a lot in this. Since the process of loading one part of the module is practically no different from the other, and we have a lot of parts, we need to allocate a factory for the production of load functions:
  var loaderFactory = function (cacheObject, method, format, methodOwner, type) { return function (name) { var dfd = $.Deferred(), self = this; if (cacheObject[name]) { dfd.resolve(); return dfd.promise(); } function successOrFail(object) { var camelCasedType = type.slice(0, 1).toUpperCase() + type.slice(1); self['push' + camelCasedType](name, object); dfd.resolve(); if (object) { // if fail EventManager.trigger(type + ':loaded', {name: name}); EventManager.trigger(type + ':' + name + ':loaded'); } } var path = Core.descriptor.path[type] + format.replace('$0', name); if (type === 'module') { method.call(methodOwner, path, successOrFail); } else if (type === 'template') { method.call(methodOwner, path, successOrFail, 'html').error(successOrFail); } else { method.call(methodOwner, path, successOrFail).error(successOrFail); } return dfd.promise(); } }; 


ModuleManager

The module manager simply loads parts of the modules and caches them. It has a number of methods for registering a module without loading (static build).
  var ModuleManager = { modules: {}, descriptors: {}, locales: {}, templates: {}, pushModule: function (name, module) {}, pushDescriptor: function (name, descriptor) {}, pushLocale: function (name, locale) {}, pushTemplate: function (name, template) {}, load: function (name) {} }; ModuleManager.getModule = loaderFactory(ModuleManager.modules, require, '$0.js', this, 'module'); ModuleManager.getDescriptor = loaderFactory(ModuleManager.descriptors, $.getJSON, '$0.json', $, 'descriptor'); ModuleManager.getLocale = loaderFactory(ModuleManager.locales, $.getJSON, '$0.json', $, 'locale'); ModuleManager.getTemplate = loaderFactory(ModuleManager.templates, $.get, '$0.html', $, 'template'); 

I left the skeleton of the object to a lot of space did not take. In any case, no one reads. Full version in source code.

Template engine

We will use a simple template from John Resig
 var templateFactory = function(str, data) {} 

Eventmanager

The event manager registers events, deletes, triggers. All global events in the app go through it. We will not reinvent the wheel. EventManager will use jQuery.bind jQuery.trigger jQuery.unbind besides the standard methods it will have an interesting method - hook , which hooks the hook function to the event. The hook function can change the contents of the event parameters, it can also prevent the event from being triggered.
  var EventManager = { $: $('<div/>'), hooks: {}, trigger: function (event, data) { if (this.hooks[event]) { // Update event data var result = this.hooks[event](data); // Don't trigger event if (result === false) { return this; } // Trigger with new data data = result || data; } this.$.trigger.apply(this.$, [event, data]); return this; }, bind: function () {}, unbind: function () {}, hook: function (event, hookFunction) { // One hook for example this.hooks[event] = hookFunction; return this; }, unhook: function (event) { delete this.hooks[event]; return this; } }; 

Using the global event manager has one important plus: We can record the event log, and then reconstruct the course of events from the log (convenient for catching bugs on the user's side).

Core

  var Core = { descriptor: {}, runningModules: {}, //  ,   init: function (descriptorOrFileName, callback) {}, //    _initModules: function (callback) {}, //      initModule: function (name, callback) {}, //   destroyModule: function (name) {}, //  HTMLElement    getBox: function (name) {}, //      getTemplateFunction: function (moduleName, templateSelector) {} }; 

Removed function body and JSDoc blocks.

From the outside, we need only some methods from all of our kernel modules. We will export only them:
  var CorePublic = { trigger: $.proxy(EventManager.trigger, EventManager), bind: $.proxy(EventManager.bind, EventManager), unbind: $.proxy(EventManager.trigger, EventManager), on: $.proxy(EventManager.bind, EventManager), getModule: $.proxy(ModuleManager.getModule, ModuleManager), getDescriptor: $.proxy(ModuleManager.getDescriptor, ModuleManager), getLocale: $.proxy(ModuleManager.getLocale, ModuleManager), getTemplate: $.proxy(ModuleManager.getTemplate, ModuleManager), pushModule: $.proxy(ModuleManager.pushModule, ModuleManager), pushDescriptor: $.proxy(ModuleManager.pushDescriptor, ModuleManager), pushLocale: $.proxy(ModuleManager.pushLocale, ModuleManager), pushTemplate: $.proxy(ModuleManager.pushTemplate, ModuleManager), init: $.proxy(Core.init, Core), destroyModule: $.proxy(Core.destroyModule, Core), initModule: $.proxy(Core.initModule, Core), getTemplateFunction: $.proxy(Core.getTemplateFunction, Core) }; 

Sandbox

Each module has its own sandbox, so we will create a Sandbox constructor that generates sandboxes. The sandbox receives a module handle as an argument. All methods sandbox can use the module.
  var Sandbox = function (descriptor) { this.descriptor = descriptor || {}; }; Sandbox.prototype.getBox = function () {}; //      -  Sandbox.prototype.is = function (role) {}; Sandbox.prototype.bind = function (event, callback) {}; Sandbox.prototype.unbind = function (event, callback) {}; Sandbox.prototype.trigger = function (event, data) {}; Sandbox.prototype.hook = function (event, hookFunction) {}; Sandbox.prototype.unhook = function (event) {}; Sandbox.prototype.getText = function (message) {}; Sandbox.prototype.getResource = function (resource) {}; Sandbox.prototype.getTemplate = function (templateSelector) {}; 


In those functions ( bind, trigger, hook, ...) that can affect other objects, the sandbox checks whether this function can be performed on this module (using the module descriptor).

Kernel build

Each part of the kernel should be placed in a separate file and put together by the preprocessor, in the example I do not use the preprocessor (it is spherical in a vacuum), so I threw it all together.
 (function(global, $, require, undefined){ "use strict"; var templateFactory = function(str, data){}; var loaderFactory = function (cacheObject, method, format, self, type) {}; var ModuleManager = {}; var Sandbox = function (descriptor) {}; var EventManager = {}; var Core = {}; var CorePublic = {}; if (!global) { return CorePublic; } if (!global.exports) { global.exports = {}; } global.exports.Core = CorePublic; }(this, jQuery, $script)); 

Application descriptor

The application also has a descriptor that describes which modules are included in the application, where the parts of the modules are located, determines the current locale, describes the basic markup. There can be several descriptors for different assemblies.
 { "modules": ["MessageView", "DataGenerator", "Logger", "Hook"], "layout": { "MessageView": ".b-message-view" }, "locale": "ru", "path": { "descriptor": "./app/descriptors/", "module": "./app/modules/", "locale": "./app/locales/", "template": "./app/templates/" } } 

Application: index.js

As you understood from the application descriptor, we will have 4 modules. The application turned out to be the simplest without any interaction with the server. I do not attach the code for the modules in the article - it is in the repository.

MessageView - displays a message on the event newData
DataGenerator - generates a newData event once per second with data from Math.random()
Logger - listens for the newData event and writes what came to the console
Hook - puts a hook on the newData event. , . 0.5, 100.

Assembly


(dev, test, prod) ( ). , . , , ( ). . . index.json .

index.js

index.js , : require, buildFrom. , ( JavaScript CSS). index.js , .
 /*$require: ./lib/Core.js */ (function (Core) { "use strict"; /*$buildFrom ./index.json */ Core.on('ready', function () { Core.trigger('newData', 'Pewpew'); }); Core.init(/*$require*/'./index.json'/*$*/); }(this.exports.Core)) 

- :
 //   Core.js (function (Core) { "use strict"; // + descriptors/Logger.json Core.pushDescriptor("Logger", { "name": "Logger", "acl": { "listen:newData": true, "listen:ready": true } }); // - descriptors/Logger.json // + modules/Logger.js Core.pushModule("Logger", (function(global){ // ... }(this))); // - modules/Logger.js // + locales/Logger.js Core.pushLocale("Logger", {}); // - locales/Logger.js // ...  -  .... Core.on('ready', function () { Core.trigger('newData', 'Pewpew'); }); Core.init({ "modules": ["MessageView", "DataGenerator", "Logger", "Hook"], "layout": { "MessageView": ".b-message-view" }, "locale": "ru", "path": { "descriptor": "./app/descriptors/", "module": "./app/modules/", "locale": "./app/locales/", "template": "./app/templates/" } }); }(this.exports.Core)) 


C require , buildFrom . . Logger ( - ). buildFrom ( "en": "He said: " ), .
dev β€” .

index.css

: 1 , :
 /*$buildFrom: ../../index.json */ /* app/css/blocks/b-message-view/b-message-view.css */ .b-message-view { color: green; font-family: monospace; } 

buildFrom css : layout , b-message-view β€” . , , . , json .

index.html

index.css index.js . index.html
 <!DOCTYPE HTML> <html> <head> <meta http-equiv="content-type" content="text/html; charset=utf-8" /> <title></title> <!--$require: index.css--> <link rel="stylesheet" href="app/css/pages/index.css" /> <!--$--> </head> <body> <!--$buildFrom: index.json--> <div class="b-message-view"></div> <!--$--> <script type="text/javascript" src="http://yandex.st/jquery/1.6.1/jquery.js"></script> <!--$require: index.js--> <script type="text/javascript" src="./vendors/Script.js"></script> <script type="text/javascript" src="./lib/Core.js"></script> <script type="text/javascript" src="./index.js"></script> <!--$--> </body> </html> 

buildFrom html ( css): layout div b-message-view . , . xslt, .

:
 <!DOCTYPE HTML> <html> <head> <meta http-equiv="content-type" content="text/html; charset=utf-8" /> <title></title> <link rel="stylesheet" href="/index.css" /> </head> <body> <div class="b-message-view"></div> <script type="text/javascript" src="http://yandex.st/jquery/1.6.1/jquery.js"></script> <script type="text/javascript" src="/index.js"></script> </body> </html> 

Makefile - General build

What the collector does:
  1. Runs the preprocessor to build index.css
  2. Optimizes index.css: data / uri, etc.
  3. Shrinks index.css (gz optional)
  4. Assembles index.css resources: images
  5. Run automated unit tests (below)
  6. Runs the preprocessor to build index.js
  7. Validates index.js
  8. Shrinks index.js (gz optional)
  9. Build a package under your OS .deb .rpm
  10. Puts a package into the repository.
  11. Installs a package

Unit tests


. , , , , , .

, -. ( ) , acl , , . .

, , :
 var TestData = { "newData": function () { var data = [NaN, Infinity, window, Error, 'pewpewpew', '<b>Pewpew</b>', '"', '\'', new Date, Date, Math, 42, 8, -1, 0, false, true]; return data; } }; 

QUnit. MessageView. - () MessageView, β€” , . , .

: index.js

 // MessageView test (function (Core, $, TestData, ok, test, module, equals, expect, asyncTest, start, stop) { "use strict"; //    var ApplicationEnvironment = { "modules": ["MessageView"], "layout": { "MessageView": ".b-message-view" }, "locale": "ru", "path": { "descriptor": "../../app/descriptors/", "module": "../../app/modules/", "locale": "../../app/locales/", "template": "../../app/templates/" } }; Core.on('ready', function () { module("MessageView"); //  1 test("listen:newData", function() { var testItems = TestData["newData"](), $MessageView = Core.getBox("MessageView"), template = Core.getTemplateFunction("MessageView", '.b-message-view'), label = Core.getText("MessageView", "text_label"); expect(testItems.length); // >>> put your code $.each(testItems, function (index, text) { Core.trigger("newData", [text]); // >>> put your code var expected = template({label: label, value: text}); // <<< equals(expected, $MessageView.html(), 'Should be "text_label: value"'); // <<< }); }); //  2 test("trigger:newData:display", function() { var testItems = TestData["newData"](), $MessageView = Core.getBox("MessageView"), template = Core.getTemplateFunction("MessageView", '.b-message-view'); expect(testItems.length); // >>> put your code Core.on("newData:display", function () { // <<< ok(true); // <<< }); // <<< $.each(testItems, function (index, item) { Core.trigger("newData", [item]); // >>> put your code }); }); }); Core.init(ApplicationEnvironment); }(this.exports.Core, jQuery, TestData, ok, test, module, equals, expect, asyncTest, start, stop)) 

5 " <<< ".

index.html β€” .


, 60 ( , ). , ( ). , ( , ):
β€” js-test-driver β€” QUnit, .
β€” TestSwarm ( ) β€” Mozilla Labs
β€” JSCoverage

Do tests so that they really work and perform their tasks. Do not do "what would have happened" tests. If you do not want to introduce automation and code coverage, it is better to think about the need for unit tests.

Code Validation and Documentation Assembly


, . , , Dox, jsdoc-toolkit .. .
. ( ), β€” , . ( ) .


. -.
  1. β€”
  2. ( ). JSLint, JSHint !
  3. .
  4. , , ! $textarea.val($textarea.val())
  5. : . β€” ! β€” . β€” . : foo, temp. , ( tmp), , β€” !
  6. JavaScript, HTML CSS ( ). HTML β€” , . CSS β€” . JavaScript β€” .
  7. . Element.style. , html ( FTW!). CSS Expressions!
  8. . 2-3 !
  9. , . β€” ! (Array.prototype, Function.prototype)
  10. , β€” , !
  11. instanceof, typeof, Object.prototype.toString magic
  12. ! URL ; ; , ,
  13. : (js-test-driver), (JSLint, JSHint), (JSDocToolkit, Dox). , Ctrl+S. : , .


GitHub scalable-js-app ( , )

: , -, ( MessageView), .

/

  1. Andrew Dupont (Gowalla, Prototype.js, S2) β€” Maintainable JavaScript
  2. Nicholas Zakas (Yahoo !, YUI, YUI Test) - Writing Maintainable JavaScript . Slides are new , old
  3. Nicholas Zakas - Scalable JavaScript Application Architecture
  4. My video " Scalable JavaScript applications " from J.Subbotnik slides
PS And so, you decided to rewrite the whole code, clean the code with JSHint / JSLint. Stop! It will take you a lot of time to rework, even more to test and rework reworked because surely at least something will break (if you do not have Unit Tests, then this will surely happen). Create a new code according to the new standard, and modify the old one if necessary, then when you change its part.

I hope it was interesting. , , ! . β€” .

Source: https://habr.com/ru/post/123635/


All Articles