Geekly Articles each Day
📜 ⬆️ ⬇️

Angular 1.x: creeping webpack, hidden grunt

The story of how we changed the project build from grunt to webpack


You come to work, you open IDE, you write npm start , starting the build system, you start working. It is convenient for you to navigate the structure of the project, it is convenient to debug the code and styles, obviously, exactly how and in what order the project is going.

It takes two years. In the development process, you periodically think about where to put the files with the new module correctly, how to deal with shared resources, and it is not always straightforward to answer the Junior's question “how does this file get into the bundle at all?”. Or you answer the sacred “so historically” and yearn for what was two years ago.

As it turned out, this happens if you do not upgrade the assembly system with the growth of the project. The good news is that this is successfully treated! In the summer we confirmed this in battle and want to share our experience.
')


initial situation


We have been developing the office application package MoyOffice since 2013, the web version (which will be discussed later) - since 2014.

There are several related projects (file manager, authorization and profile, web-editor of documents) with common sub-repositories, each of which is a SPA-part of a large application MyOffice. Development is carried out on angular 1.5 , jenkins is used for continuous integration.

The original grunt build system, consisting of complex interdependent tasks, was created at the dawn of the project and has changed little since then. To indicate the scale: the dev-build launched about 30 grunt-tasks, 30% of which collected modules and styles, 70% - shifted images and fonts, updating the links to them. The order of execution was crucial, however, information about interdependence could only be obtained from colleagues.

Why migrate and why webpack


Collecting an angular project is actually not so difficult: you just need to combine all the source files into one, not forgetting that the module was announced before its controller. We made it even easier: we collected all the files from the src folder (using `_` at the beginning of the file name to ensure the correct connection order), added an array of external packages, and then connected the files directly to the head (of course, only for the dev-build, for production code was concatenated into a bundle followed by obfuscation and minification.



The assembly is obviously morally obsolete. The latest critical changes were dated 2015, which in the conditions of the modern frontend can be equated to a web-covered corner, in which, frankly, unfashionable grunt stores its intermediate files.
Plus, she had only one: the reassembly of the project was conditionally free due to the direct connection of files to the head.

Cons are much more:


And the more our project expanded, the more strongly the shortcomings of the build system interfered.

Taking a step back, looking at ourselves, at others, at the trends, recalling the experience of previous projects, we chose a webpack that effectively solves the problems described above.

Work organization




The main secret of successful refactoring is to clearly define the steps in advance and make a plan.

  1. Make a list of all requirements.
  2. Implement the proof of concept on a small project site. The idea is to collect all possible rakes cheaply and in the background, without risks for the main development.
  3. To make a complete transition, taking into account all the fine points identified in paragraph 2. Knowing all the problems and having experience transferring part of the code to new rails, one can fairly accurately estimate the labor intensity.

Reverse engineering requirements


The new system should not only solve the existing problems of assembly, but also support several new and long-desired features (and of course, not lose the old ones). Compiled a list of what should be able to webpack in a finished form:


In this case, grunt cannot be excluded from the process, since it is responsible for assembling styles, working with images and fonts, and generating documentation. For consistency, we even want to run the webpack via grunt, and not via npm-task, in order not to change the command to build the project at all and not to reconfigure anything to CI.

Proof of concept


At the mercy was given one of our applications - SPA, responsible for all manipulations with authentication and user profile. At the end of the work with him could be taken for the rest.

In a good way, all the work was divided into three parts:

1. Create a config for webpack.
2. Prepare files for such an assembly. File Formats:


3. Rewrite unit tests.

Css along with the media temporarily postponed, since they are not integrated into angular and can continue to live their lives.

js modules


For those who have not looked into angular for a long time, let us recall how it looks from the inside:

 // module.js angular.module('moduleName',[ 'dependencyOneName', 'dependencyTwoName' ]) .controller('SomeController', function(){…}) .directive('someDirecive', function(){}); // someService.js angular.module('moduleName') .service('SomeService', function(){…});

The main thing that disturbs us in the case of the webpack: all dependencies are indicated simply by the string-name of the required module. To build a dependency graph in a webpack, you must explicitly specify which file to include.

Over time, such a plan was formed:

 //module.js module.exports = angular.module('moduleName', [ require('path/to/firstDependency'), require('path/to/secondDependency') ]) .controller(...require('controller.js')) //es6 spread syntax feature yay! .name; //controller.js module.exports = ['SomeController', function(){}];

By using es6 spread syntax, we were able to elegantly avoid duplicating the name of the module when declaring a component.

Since the dependency connection format changed critically, it was not possible to touch common sub-repositories within the POC framework in order not to hook on other projects. Therefore, all common files had to be connected manually by a long-long list.

HTML templates


Templates are divided into two categories: index.html and all others. Collecting index.html easy with the html-webpack-plugin . Everyone else used to do grunt-ng-template . I had to search for webpack-plugin to work with templates. There were only two requirements for it:


The first item was easy to handle, and the second one had problems. Until now, there is no suitable solution, and although it is easy to write it, it was faster for us to connect all such templates in js. In the future we want to develop a webpack-loader for this purpose. If you have already written one yourself, share the github link with us in the comments.

With an hit in $templateCache interesting nuance: if you do require within a directive or controller, then it will try to add itself to the cache only in runtime, without getting into the bundle in advance. With the advent of angular components, this was corrected; in other places, templates had to be connected before the controllers were announced.

In order to easily detect missed template connections, we added a webpack-dev-middleware in webpack-dev-middleware that prohibits downloading any nested html .
 function blockLocalTemplatesMiddleware(req, res, next) { var urlPath = parseUrl(req).pathname; if (/[^\/]+\/[^\/]+\.html$/g.test(urlPath)) { res.statusCode = 404; res.end('Request to .html template denied'); } else { next(); } }

Configs


Each of our projects has configurations that are sewn into the project at the assembly stage. Previously, all configs were stored in several json-files, grunt-ng-constant wrapped them in an angular-module and connected to the project at the assembly stage, reducing the transparency of reading and debugging. Using DefinePlugin made it much more convenient and easier.

Unit tests



Final migration


Three weeks of careful POC later, we were ready for the final migration of the entire application.

image

Work with css


We migrated in June-July, thought about the article in August, resolutely started writing in December, and during this time we already had time to get used to the convenience of the modular structure and decided to transfer the sass-style assembly to a webpack.

And although this article initially assumed the story only about the first stage of migration, we cannot but share the experience of giving up grunt-sass .

The process of such migration is in fact quite trivial: just connect all the necessary styles in the modules where they are used. However, without pitfalls also not done.

How did the build work before? Like building js modules. According to the mask, all *.scss were collected and imported in one file. Then sass worked on it alone, all mixins and helpers connected once were available everywhere, there were practically no cross-imports.

To implement the modular structure, we started importing variables, mixins, node-bourbon , lunaparks, blackjack in each style file. Because of this, two troubles happened:


However, the update to the recently released node-sass@4.0.0 accelerated the reassembly by about 1.5 times, and we decided to postpone the massive processing of styles.

Debugging and Testing


The main thing in development is not to write, but to debug, according to the results of debugging, we have compiled a cheat sheet for those who decide to repeat the migration path (by the way, it doesn’t matter where to get off - with gulp or grunt). Basically, all the defects encountered on the way and their diagnostics looked like:


Separate manual testing from the QA-team in this task was not required, it was easy enough to run autotests. The only thing we asked testers to check is that jenkins is successfully and correctly built with all possible flags.

Technical points


There are a lot of possibilities to optimize the process of working angular using a webpack. On github, you can find dozens of loaders (curiously, from the moment we completed the migration to the day when this paragraph was started, there were already some plug-ins that we lacked then). However, a third of them do not have documentation and contain only minified code, so it is not possible to use them, the second third works ambiguously (for example, there are three loaders for templates, they do the same thing, and we have earned correctly only one).

Inevitable difficulties



For example:

 require('ng-file-upload'); angular.module('app', ['ngFileUpload'])

You can cope with this with the help of a pull-requests battery on github, we sometimes send them in spare time.


For example, the project structure is as follows:

 ├── src │ └── module.js └── common └── src └── module.js // webpack.config.js resolve: { root: ['src','common/src'] } //app.js require('module.js');

If you change the file, change detection cannot work correctly and the project is completely unpredictable, it is re-assembled, then no. Since we had several input points with the same internal structure, we had to abandon it.


Unexpected Benefits



 new webpack.optimize.CommonsChunkPlugin({ name: 'vendor', chunks: ['app'], filename: 'vendor.[hash].js', minChunks: function minChunks(module) { return module.resource && module.resource.indexOf('node_modules') > 0; } })

Why divide the code into two parts? To use a DLL , speeding up rebuilding. Plus, with fairly frequent releases (and we strive to increase their frequency) the list of dependencies does not have time to change, while maintaining the same hash. This allows the user not to download an extra file, but simply to take it from the browser's cache.


Did not go to work


Automatic loading of modules


Of course, we didn’t really want to rewrite all module dependencies from strings to require . It would be cool to fasten the loader , which would analyze the code and itself substitute the necessary require !

However, such an approach requires a strict project structure so that it is possible to unambiguously match the path to the module with its name. In fact, the string with the module name would uniquely transform into the file path. At the time of the migration, the project structure did not have this approach, and the reorganization of files would take no less time and threaten a large number of conflicts when merging branches.

Now we are starting the path to strict organization of the source code and, when we are finished, we will be able to take advantage of such opportunities. Although this, most likely, it does not want to, because going through ctrl-click immediately to the dependent module is extremely convenient.

Hot module replacement


Unfortunately, we had to refuse HMR for js-code. There are two plug-ins , but both of them require not only a very strict project structure, but also an exact export format, and also work only with controllers, but not with directives. Even with a suitable structure, using the update only for a part of the code is completely inconvenient. However, for styles HMR works correctly.

Tips for the past


The migration process went quite smoothly and step by step, however, as it usually happens, having completed the work, we figured out how to facilitate it:


About numbers


The most interesting is, of course, the numbers. Interesting logs on tasks:



Re-and underestimations take place, however, in general, we managed to predict the labor intensity rather accurately. The key to this success, we believe a clear wording of the requirements prior to the implementation of the task. The experience of previous and subsequent mass refactorings confirms this: if we take on the task, planning to make a list of requirements in the development process, forgetting something important is easier than easy.

Assembly time


On the old build, when detecting changes in js, the page started reloading right away. If changes were made to the styles, rebuilding css took about 3.5 seconds.
After moving reassembly occurs in 5 seconds regardless of where the changes were made.

The loading time of the page in the dev-version on the old build took about 1.5 seconds due to the large number of js-files connected. After the transition to the webpack, it was reduced to 0.8 seconds. When you change styles, both then and now, no reload is required.

Thus, the following data is obtained. The table shows the time from making changes to applying them on the page:



findings




Minuses:


Pros:


Future plans:


In general, it became easier to navigate the project, the entry threshold for a new employee became lower, refactoring is more accessible, dependency tracking is more convenient. Now you can develop individual modules and not be afraid that part of the code or css will fall into a common bundle.

It would be a shame to read such a long article and not get a bonus at the end! We put ready for you configs for webpack and karma !

Source: https://habr.com/ru/post/321584/

More articles:


All Articles