The frontend build system in the CleverStyle Framework or why you may not need a custom

The CleverStyle Framework in every possible way helps the developer not only on the server, but also on the frontend. I mentioned this several times in previous articles, but I never went into the details of exactly how everything worked under the hood.

This article will dive into the details of working with statics for the frontend, starting from how the necessary files are defined on the page and ending with statics delivery optimizations like HTTP / 2 Server Push. Let's not forget why using the CleverStyle Framework you can do without a custom build system and how, if you wish, integrate such a build system into the framework processes.

This article specifically misses the integration of Bower / NPM and RequireJS, this will be the topic of a separate article in the near future.

What to connect

The first task is to determine which files (CSS / JS / HTML) are needed on a specific page, and here there are two conditional groups:

• shared files for all pages of the site
• files that are specific to this page (or several pages)

Shared files

Shared files, in turn, can also be divided into three subgroups.

The first subgroup includes the kernel files from the assets directory. There is that basic minimum of service files that the framework itself needs (in fact, this can be circumvented, but more on that later). Files of this subgroup are loaded first.

The second subgroup contains the files of the current theme of the themes//{css|js|html} interface themes//{css|js|html} , which are connected immediately after the kernel files.

The third group includes files of any installed modules, for which it is not indicated on which page (the page is a bit conditional) they should be connected (about this a little further).

Files for specific pages

Specific pages are generated by installed modules.

In a typical scenario, when the Module_name module is Module_name it will process pages starting with /Module_name . In order to indicate that on the pages of this module specific files should be used, a map of such files associated with the paths in the file with the module meta-information is compiled.

Example for the Blogs module :

 { "package" : "Blogs", "category" : "modules", ... "assets" : { "admin/Blogs" : [ "cs-blogs-admin-*" ], "Blogs" : [ "cs-blogs-*" ] }, ... } 

In the Blogs module there are no CSS and JS files to connect to the page, only web components (which in turn may contain CSS / JS, but this is another story, about which a little further).

In the example above, we see that all pages of the module will include files whose paths begin with the prefix cs-blogs- (the asterisk is discarded and maintained solely for the purpose of improving readability), and files with the prefix cs-blogs-admin- will connect only on administration pages (this key is listed first in the list, so that the files with this prefix are first filtered, and the rest will be included in the next key, despite the same beginning).

At the expense of paths, you need to clarify that they are counted from the following roots: modules/Modules_name/assets/{css|js|html} , from the file format it is clear in which directory it is located, that is, modules/Modules_name/assets/html will be searched only HTML files, but not CSS and / or JS.

Direct dependencies

The files of the module itself are only part of the history, the dependencies between the modules are an important link in this system.

Let's look at the Photo gallery module for an example and one of its dependencies, the Uploader module :

 { "package" : "Photo_gallery", "category" : "modules", ... "assets" : { "admin/Photo_gallery" : "admin.css", "Photo_gallery" : "general.*" }, ... "provide" : "photo_gallery", "require" : [ "System>=6.25", "System<7.0", "Composer", "Fotorama>=4.4.9", "file_upload", "composer_assets" ], ... } 

 { "package" : "Uploader", "category" : "modules", ... "assets" : { "admin/Uploader" : "admin.css", "Uploader" : "script.js" }, ... "provide" : "file_upload", ... } 

Here we see that the Photo gallery mod is directly dependent on file_upload . file_upload in turn, is not a module name, but its functionality ( for more information, see the documentation ). From the point of view of static processing, this means that the files intended for the Uploader module (for the module as a whole, not for the subpages) will also be connected on the pages of the Photo gallery module, with their own files in front of this module (this is important if the dependency has some synchronous code used on dependency further).

It should be noted that not only mandatory ( require key), but optional ( optional key) dependencies are taken into account in this scenario.

Inverse dependencies

For frontend, inverse relationships are also an important concept. These are such dependences, which are indicated not by the target module, but by a third-party module themselves. Together with the ability to redefine web components with alternative implementations, this is quite a powerful tool. In essence, this provides an opportunity to tell the framework that this module provides additional functionality for another module (for example, it changes the appearance of a blog without having to edit the blog module itself). This is a rather subtle point, they should not be abused, but sometimes it is useful.

Nested dependencies

Dependencies can have their own dependencies, including dependencies that can be repeated. The framework understands this and turns the dependency tree into a flat structure, taking into account this feature (do not try to create circular dependencies, the result will be undefined).

It is important to note that the dependence on the system version (the System module) does not affect the cyclical nature of dependencies; you can effectively assume that it will be ignored during the processing of statics.

Integration into the process of collecting files for connection

During the assembly of files for connection, the system generates the System/Page/assets_dependencies_and_map , by subscribing to which it is possible to manipulate the files collected by the system as well as the built dependency structure.

You can add your own files to this structure, organize them into such virtual modules and complete the dependencies. Thus, the Composer assets module integrates the connection of files from Bower / NPM packages into the framework in order to use its internal mechanisms for processing files (which are a little further).

Actually connecting files to the page

The framework does not always simply connect all the files needed for a particular page, but comes in a more advanced way depending on the system configuration.

The following parameters affect how files are attached:

• caching and compression
• optimization of frontend loading (depends on the previous parameter)
• vulcanization (also depends on the first parameter)
• moving javascript and html over body page
• disable web component support

Moving javascript and html for body page

This is the simplest, JS / HTML files will be connected before </body> instead of being placed in a <head> , it is generally recommended to use.

CSS is specifically connected in <head> always, since the basic styling is usually small and the rendering delay in the page is not significant (especially using HTTP / 2 and Server Push).

Disable web component support

Too simple enough setup, which, however, works only for custom themes (so as not to break the admin panel with the default theme). It leads to the complete filtering of any HTML files, also disables the loading of polyfills for web components and JS files from assets/Polymer , which generally disables everything related to web components in the framework if you do not need it. It is important to understand that many ready-made modules will stop working, since their interface is built entirely on web components.

Caching and compression

First, the framework takes all the collected files to connect to the pages with dependencies and packs the files to the cache. For shared files, a triplet of cached files with css / js / html formats is created, similar files are created for each assets key in each installed module + interface interface language translation caches are created separately; the polyfill for web components is separately cached and the new dependency structure is saved, indicating data collected cached files.

Secondly, files do not just go directly to the cache — some processing precedes this. The framework is not trying to catch the stars from the sky in this matter, but it works extremely fast.

CSS processing is to:

• remove extra spaces and line breaks
• optimizing some constructions such as reducing color HEX values ​​from 6 to 3 characters and converting RGB colors to HEX
• embedding small images and similar files (up to 4 KiB) in base64 form, not embedded files are written in a separate list and will be used later
• adjusting relative paths for new cached css target

JS processing is much more scarce, since there is no desire to drag a full-fledged minifier, otherwise it is easy to break the code with a simple search and replace, so we limit ourselves to the following:

• deletion guaranteed safe string translation
• delete single-line comments that start at the beginning of the line
• delete multiline comments that start at the beginning of the line
• Zmenman </script> to <\/script> , since this JS code can later be used to insert into HTML
• Multi-line patterns from ES2015 are ignored (from the first to the last - everything inside is excluded from processing)

The CSS / JS minification data is fairly basic, but they allow you to sort through the entire project in ten milliseconds, while full-fledged minifiers take a few seconds and with Gzip you can win up to 5% only (you can run these minifiers over the built cache).

HTML is processed even less - in fact, all JS code comes together and is run through the mentioned minifiers, CSS for each web component is also run through the mentioned minifiers, in order to avoid problems, <link rel="import" href="../polymer/polymer.html"> from HTML.

After the cache is built, a System/Page/rebuild_cache is System/Page/rebuild_cache - custom minifiers and other things like building a custom cache can be done in the event handler.

In the cache structure, each file is accompanied by a hash in terms of parameters (examples below in the part about HTTP / 2 Server Push). The hash is counted from the contents of the file, so if you change one file and rebuild the cache, the hash of the corresponding cached file will change, but not the entire cache at once.

You can read more about the structure of the auxiliary JSON files with the description of the cache structure, dependencies and other things in the documentation .

Vulcanization

Vulcanization is the processing of an HTML file, in which CSS / JS code is embedded in the resulting HTML. For CSS, this does not matter, since Polymer does not yet support CSP-compatible constructs in all situations, so CSS is always embedded in the resulting HTML.

Implementation in the framework of its own, in general, you can read about it in the relevant project on GitHub .

Frontend load optimization

This is an interesting mechanism, whose vocation is to speed up the initial rendering of the page.

As described above, there is a common CSS / JS / HTML code and specific to individual pages. So this common code can be thought of as an App shell. Optimization of frontend loading is that initially only common JS / HTML is connected to the page, and only after it has been loaded and the asynchronous loading and sequential execution of the rest of the JS / HTML code begins (modules must be ready for HTML code that will be common to all pages may be executed earlier than the JS code of dependencies of this module).

This approach allows you to significantly speed up the first page rendering and thus give the user, if not an interactive page, then something useful and as quickly as possible.

For optimized loading, a separate file is built with an additional optimized cache structure.

HTTP / 2 Server Push

When caching is turned on, the necessary page files will be accompanied by the appropriate 'Link: `headers, which proxy servers (like CloudFlare, nghttpx) are usually turned into Server Push, an example of headers:

 Link:</storage/public_cache/CleverStyle:TinyMCE.html?602ca>; rel=preload; as=document Link:</storage/public_cache/CleverStyle:System.css?fecf7>; rel=preload; as=style Link:</storage/public_cache/CleverStyle:Uploader.js?2167a>; rel=preload; as=script Link:</storage/public_cache/CleverStyle:System.html?14dd7>; rel=preload; as=document Link:</storage/public_cache/CleverStyle:System.js?a0e9d>; rel=preload; as=script Link:</storage/public_cache/CleverStyle:Static_pages.html?a292d>; rel=preload; as=document 

It becomes even more interesting if we add the frontend load optimization to this - then Server Push will be used only for those files that are connected initially, an example of the same page as above:

 Link:</storage/public_cache/CleverStyle:System.html?14dd7>; rel=preload; as=document Link:</storage/public_cache/CleverStyle:System.js?a0e9d>; rel=preload; as=script Link:</storage/public_cache/CleverStyle:System.css?fecf7>; rel=preload; as=style 

Server Push will also get files that were not embedded in CSS due to their size, but which are needed to render the page (see the documentation for details).

And, of course, Server Push is triggered only once, after which a cookie is set in order to avoid performance degradation.

Why you may not need a custom front end build system

If your development process is similar to mine: LiveScript-> JavaScript, SCSS-> CSS, Jade-> HTML - everything is automatically generated by File Watchers with every change and completely goes into Git, then you can do without a custom build system. Any change to any file leads to the generation of the corresponding artifact and while you are switching to the browser to press F5 (without caching enabled in the framework) you are ready to work.

What if you have a custom build system, do you want to add a custom build system or replace what the framework does completely?

Your best friends are two system events:

• System / Page / assets_dependencies_and_map
• System / Page / rebuild_cache

With their help, you can integrate into the work of the framework and enjoy all the mentioned advantages while having / using additional artifacts / tools as you like.
The documentation also describes the cache structure, so you can prepare the artifacts of your build system in a format that the framework will understand and thus not use the framework’s build system, but feed it its own artifacts in an edible format.

At last

Always glad to new ideas and constructive comments.
» GitHub repository