Geekly Articles each Day
📜 ⬆️ ⬇️

We develop frontend Dnevnik.ru. Part one. Build and validate JavaScript code

Introduction


During its existence, Dnevnik.ru (and this is more than 4 years) has accumulated a huge amount of JavaScript code: some were in a separate project as connected files, some were determined right on the controls layout, and some were collected directly in the code-behind using StringBuilder . To this were added:

Deciding that it’s time to do something, we set ourselves the primary task: to remove all separately connected files from the <head> into one minified package. At the same time, the code was divided into third-party and “our”, which was planned to be checked with some kind of parser.

In this article we will tell you about how you solved this problem.

What to use?


First of all, we had to decide on the means by which we would organize the automatic assembly of this package. Of course, you could use any build system, from Ant to MSBuild; you could write your own simple script — for example, in Ruby or Python. As a result, we decided not to write our bikes and not to hammer nails with a tractor, but to use Grunt . For those who do not know: Grunt is a JavaScript task runner, it works on node.js , and is distributed under the free MIT license . Despite the relative "youth" of this solution, it has already managed to establish itself as a great tool - it is used to build jQuery and QUnit, Tweetdeck on Twitter and Brackets in Adobe. In addition to these recommendations, we also had our own reasons for choosing Grunt:

By the way, it's no secret for anyone that our project works on ASP.NET, therefore we considered the possibility of using the Web Optimization Framework Bundle Transformer derived from it. However, we abandoned these solutions for the following reasons:

However, if in the future these tools rise to the level of sprockets from Ruby on Rails, then I do not rule out that we will return to their consideration.

Go!


So, the system for the assembly has been chosen and it’s time to act, but before further narration you should make a reservation. Since the application is written in ASP.NET, the majority of developers work on Windows (which is not surprising), and the continuous integration process that was built with TeamCity (we wrote about this in a previous article ) also occurs on Windows Therefore, the author asks fans of the Unix-way to forgive him for the fact that the following will be described exactly within the framework of the Windows ecosystem, and to take the entire experience below as a challenge.
')
Installing node.js on Windows has long been a problem. All you need to do is download a binary file from the official site , launch it and poke it in the “Next” button. Together with node.js, npm, the package manager, will be installed, with which we will install both Grunt and everything needed for its operation. To begin with, we will create a package.json file in the project, in which we will write the name of our project, its version, dependencies, and the node.js version. It will look like this:

 { "name": "Dnevnik", "version": "0.1.0", "private": true, "dependencies": { "grunt": "0.4.0", "grunt-cli": "0.1.6", "grunt-contrib-concat": "0.1.3", "grunt-contrib-jshint": "0.2.0", "grunt-contrib-uglify": "0.1.1", "grunt-hash": "0.2.2", "grunt-contrib-clean": "0.4.0" }, "engines": { "node": "0.10.0" } }

In the dependencies, we specify Grunt and its version, as well as the necessary plugins. At the initial stage, we used only six plugins:

To install all packages with their dependencies, you need to execute just one command in the console relative to the directory where package.json is located:

 > npm install

After successful completion, a folder will appear in it .\node_modules , which will contain all the necessary modules (this is the standard folder name for modules installed via npm).
Next you need to create Gruntfile.js in the root directory of the application, it will contain all the logic of Grunt. Its structure is very simple:

 module.exports = function (grunt) { 'use strict'; grunt.initConfig({}); grunt.loadNpmTasks('grunt-contrib-jshint'); grunt.loadNpmTasks('grunt-contrib-concat'); grunt.loadNpmTasks('grunt-contrib-uglify'); grunt.loadNpmTasks('grunt-hash'); grunt.loadNpmTasks('grunt-contrib-clean'); grunt.registerTask('default', ['jshint', 'concat', 'uglify', 'hash', 'clean']); };

In essence, this is a javascript script for node.js, consisting of:

When a task starts, it tries to find an attribute with its name in the object that was passed to the grunt.initConfig() function, and it gets all the settings through the option attribute and the target through the other attributes. There can be an unlimited number of goals in a task, and each goal can override some settings for itself. More information about the configuration of tasks can be found in the official documentation .

In order to start Grunt, you need to run the following command in the console relative to the application root directory:

 > .\node_modules\.bin\grunt.cmd

Optionally, you can pass a parameter with the name of the task and the goal to be executed. If it starts without parameters, the task with the name default will be executed.
Next, we had to split those notorious 11 files containing various libraries and jQuery plug-ins into separate atomic files. Some of them were compressed, and for the convenience of development I wanted to have all the code in a normal, readable form. And if it was easy to find a non-minified version of jQuery, then finding the right version of some ancient plugin was no longer so trivial, it was necessary to tinker with it. However, the result was worth the effort: now there was no minified code in the project, and it was possible to get into a debugger even in the jQuery source code without any problems.

In order for each of the developers to not need to install node.js and build the package, we made a simple mapping file in JSON format (of course, not the most beautiful solution, but we decided to make everything as simple as possible), in which the package name corresponded to from several files:

 {"package.js": ["jquery.js", "foo.js", … "bar.js"]}

When the application started, this file was read, deserialized, and the files were added to the page. And when building, Grunt received information about which packages to build from which files, and at the final stage transformed it. So instead of a set of files in the list there was one name of the assembled package.

After the code for running Grunt was written, and the tasks were prepared, it was necessary to install node.js on all build agents in TeamCity and try it in action by running a script through PowerShell. In order not to download the necessary dependencies from the network each time (do not think it’s not about our stinginess for traffic, we just don’t want to depend on the stability of the Internet or the npm repository), we decided to save them to a separate folder on each build agent in the right place before use. “Cheap and angry,” we thought (for what this led to, read below). However, in this situation it should be .\node_modules in mind that the paths in the folder .\node_modules can be much longer than the maximum 260 characters allowed in Windows (Hi, MS-DOS), so the copy and xcopy commands will crash with an error, can that robocopy with the / e flag.

What problems we faced and how they were solved.


The first Grunt pig slipped to us immediately after launching on TeamCity - we could not get a log of its work. Digging into our PowerShell scripts and realizing that the problem is not on our side, we began to watch the issue tracker in the Grunt repository and found there a remarkable message . It turns out that such a problem appeared not only in our country and it is connected with a bug in node.js, in which the stdin / stdout / stderr threads are not blocked in Windows. They promise to fix it in version 0.12.0, but in order to make Grunt work for us, we had to resort to a not very nice hack: we started Grunt twice - the first time we received the correct exit code, and the second redirected the flow output to file, after which output the contents of this file.

Not so long ago, a patch for Grunt appeared, correcting this error, but it is not yet located in the main repository. So we had to download the fork directly from Github, and then we faced another nuisance. The fact is that when we first started working with Grunt, there were only three cars in our park of build agents. Now there are eight of them, and copying a new package to each of them is tedious. Without thinking twice, we decided to set up a local npm repository, where we could always quickly pick up the packages and where we could put our own, regardless of the connection and availability of the official repository.

The official npm repository works with CouchDB , and to create a local repository, we just needed to create its replication. We quickly picked up the virtual machine (again, running Windows) and installed CouchDB on it - the benefit is no more difficult than installing node.js. Further, in order to be able to access the repository from the local network, in the CouchDB configuration file <CouchDB install directory>\etc\couchdb\local.ini you need to change two values:

 secure_rewrites = false bind_adress = 0.0.0.0

You can check the correctness of the setting by sending a normal GET request to the 5984 port of the virtual machine and receiving approximately the following JSON answer:

 {"couchdb":"Welcome","version":"1.2.1"}

After that, it remains only to obtain information about all the modules used in the project and replicate them. To do this, in the root directory of the project, you can run the following command:

 > npm shrinkwrap

It will create the file npm-shrinkwrap.json , which will contain all the information about the project, including all dependencies. But, since we will only need their names, we will have to work a little more by writing a small recursive script that will get them from the resulting file (I will not give its code, since it is incredibly trivial). Having received the list with the names of the packages, we need to execute the usual HTTP request to CouchDB for their replication. We will use the curl utility for this (although you can use any other) and for convenience we will create a JSON file called deps.json with the following content:

 { "source": "http://isaacs.iriscouch.com/registry/", "target": "registry", "create_target": true, "doc_ids": ["_design/app", "_design/ghost"] }

where the value of the attribute "doc_ids" needs to be supplemented with a list of necessary dependencies (packages "_design/app" , "_design/ghost" organize the work of the database as a repository). And now we just execute the following command in the console:

 > .\curl.exe -X POST http://user:password@npm:5984/_replicate -d@deps.json -H "Content-Type: application/json"

The response from the server will again be in JSON format, and it is worth paying attention to two attributes: "ok" and "doc_write_failures" . If the first value is true and the second is 0 , then the replication of the packets was successful.

All we had to do after that was to publish the fork received from Github with the Grunt patch. To do this, you need to change the version name in the package.json file of the fork, register the user in the local repository with the command:

 > npm adduser --registry="http://npm:5984/registry/_design/app/_rewrite/"

And publish the package:

 > npm publish --registry="http://npm:5984/registry/_design/app/_rewrite/"

Everything is done, the package is published in our local repository, you just need to remember to change its version in the project package.json .
Now to install all the packages you can (and should) use the following command:

 > npm install --registry="http://npm:5984/registry/_design/app/_rewrite/"

By the way, not so long ago we abandoned the use of the PowerShell script to run Grunt on TeamCity and switched to using the plugin for it. The plugin is called TeamCity.Node and allows you to run node.js scripts, npm, Grunt and PhantomJS on TeamCity, while it checks whether node.js and npm are installed on the build agent. So far, we are absolutely satisfied with his work, if only because it was with his help that we learned that we forgot to put node.js on one of the agents.

What's next?


We are waiting for the release of node.js 0.12 and Grunt 0.5, in which the errors described above should be corrected. And our plan for the future is like this: first, we need to abandon the use of the file mapping, second, we need to move all the JavaScript code from the controls to separate files in order to reduce the amount of code and improve its support.

But we will tell about it in our next articles.

Source: https://habr.com/ru/post/181352/

More articles:


All Articles