Automatic build javascript / coffeescript project

When developing at least some large javascript project, you immediately understand that you cannot write all the code in a single file. After that, the code is spread across several files and directories and a simple script is written so that all these files can be easily merged into one large production file. After some time, you begin to notice that the farther you go, the harder it becomes to keep track of the dependencies between files, and the whole mechanism developed looks more like a crutch. And here comes the insight that it would be nice to see what are the solutions to this problem.

The following requirements are put forward to the project assembly management system:

1. Compilation of coffescript to javascript. If the coffeescript file contains an error, the file name and error message should be displayed in the console.
2. Building a project into one javascript file should be based on dependencies.
3. The ability to collect the entire application in one file in several forms (with comments, minimized). In this case, the application itself may consist of several modules.
4. Building test files and executing them in the console (yes, we are developing for the web, without touching the mouse and not getting out of our favorite vim at all).
5. Of course, all this should be convenient to use.

In this article, I will not touch on the issue of testing, but I will consider a version of the project's javascript / coffescript build management system (and the project structure itself) using rake and Rake :: Pipeline ( git ).

Rake :: Pipeline is a file processing system. She is able to read files from a directory according to a given pattern, change files according to a given rule, and write the result.

As it is not difficult to guess, Rake :: Pipeline uses rake, therefore ruby ​​is necessary for its work. All pipeline settings are usually stored in the “Assetfile” file. This file is a ruby ​​script. It may have, for example, the following form:

#  Assetfile #        input "app/assets/javascripts" #         output "public/javascripts" #         , #  input.          #*.js,      "app/assets/javascripts" match "*.js" do #ConcatFilter - ,      . #    *.js    app/assets/javascripts #   application.js,      #public/javascripts. filter Rake::Pipeline::ConcatFilter, "application.js" end 

')
Consider, for example, a project called “application”. This project will consist of 3 coffeescript files: “file1.coffee”, “file2.coffee”, “file3.coffee”. Thus we get the following directory structure:

-application
--src
---file1.coffee
---file2.coffee
---file3.coffee

Suppose that we have the following dependencies:
2nd depends on 1st and 3rd
3rd depends on 1st
Thus in the assembled version of the files should be in the following order: 1-3-2.
For convenience, we will create the main file “main.coffee”. It will contain a list of files used in the project. Now you can start filling in the files:

 # main.coffee (     ): require("file1") require("file2") require("file3") # file1.coffee (    ): # ...  ... file1 = true #   # file2.coffee (  1-  3-): require("file1") require("file3") # ...  ... file2 = true #   # file3.coffee (  1-): require("file1") # ...  ... file3 = true #   

In this case, require (“file1”) is a pseudo-function. More precisely, it is a template, a pointer to the fact that the first file is required for work. You can configure it so that instead of require ("file1") you need to write:

  ! , , file1.   , . 

That is, the file connection syntax can be done in any way. For example, you can specify dependencies in the comments. This allows you to use the pipeline, for example, to process css files.

In our case, since the second file depends on the first and third, then only one line could be entered in the main.coffee file: require (“file2”). The remaining files should connect automatically.

With the structure figured out, it remains to collect all this. To do this, in the root of the project we create a Gemfile with the following content:

 #  Gemfile source "http://rubygems.org" gem "rake-pipeline", :git => "https://github.com/livingsocial/rake-pipeline.git" gem "rake-pipeline-web-filters", :git => "https://github.com/wycats/rake-pipeline-web-filters.git" gem "uglifier", :git => "https://github.com/lautis/uglifier.git" group :development do gem "rack" gem "github_downloads" gem "coffee-script" end 

Here rake-pipeline-web-filters is a helper library that contains, in particular, a class for processing coffe-scripts. uglifier is a library for minimizing javascript.

Now create a rakefile:

 #  Rakefile abort "Please use Ruby 1.9 to build application!" if RUBY_VERSION !~ /^1\.9/ require "bundler/setup" def pipeline require 'rake-pipeline' Rake::Pipeline::Project.new("Assetfile") end task :dist do puts "build application" pipeline.invoke puts "done" end task :default => :dist 

Here Rake :: Pipeline :: Project.new (“Assetfile”) - a new object is created, “Assetfile” is a file with assembly settings, which we do not already have, but now we will create it.

Immediately you can register the root directory for compiled files. The path will be “target”:

 #  Assetfile output "target" 

We will build the project in 2 stages. First, we will compile all coffescript files into javascript, and then we will compile the project itself.

Javascript compilation

Compilation will be carried out in the directory "target / src". In addition, each file ".coffe" will correspond to its own file ".js" (that is, at this stage we will not merge the files). To do this, add the following lines to the "Assetfile"

 #  Assetfile #      "src" input "src" do #    *.coffee (   "src") match "**/*.coffee" do require "rake-pipeline-web-filters" #       javascript filter Rake::Pipeline::Web::Filters::CoffeeScriptFilter do |filename| # ,       ( ) #  js . #         "src"  "target" #       '.coffee'  '.js' File.join("src/", filename.gsub('.coffee', '.js')) end end end 

Now, if you run the rake command, a project version compiled into javascript will be created in the “target / src / lib” directory. If any of the files fail to compile, an error message will be displayed.

Build javascript project

This time we will read the already compiled js files from the 'src / lib' directory:

 #  Assetfile #   ,      name="application" #     "target/src" input "target/src" do #  main.js  match "main.js" do #   NeuterFilter. #      ,  #       . neuter( # ,   . :additional_dependencies => proc { |input| #       ,   main Dir.glob(File.join(File.dirname(input.fullpath),'**','*.js')) }, #     :path_transform => proc { |path, input| #    require("file1")   #   .    . #  require("file1")   require("file1.js") "#{path}.js" }, # ,         js- :closure_wrap => false ) do |filename| "#{name}.js" end end end 

Now, if you run the rake command, the file 'application.js' will appear in the 'src' directory with the following contents:

 #  application.js (function() { var file1; file1 = true; }).call(this); (function() { var file3; file3 = true; }).call(this); (function() { require("file2"); }).call(this); 

But wait! What is the line doing here?

 require("file2"); 

? After all, she had to disappear. This seems to be a neuter filter error. Let's look at the source code of this filter ( code ). We are interested in the line here:

 #  neuter_filter.rb regexp = @config[:require_regexp] || %r{^\s*require\(['"]([^'"]*)['"]\);?\s*} 

As you can see, if the parameters do not specify their own rule for identifying text with the name of the required file, the regular expression is used by default

 %r{^\s*require\(['"]([^'"]*)['"]\);?\s*} 

Unfortunately, I could not understand why such a regular expression processes only the first require, and the second one does not notice. I would be very grateful if you explain what is the matter. I solved this problem as follows:

 #  Assetfile input "target/src" do match "main.js" do neuter( .... :closure_wrap => false, :require_regexp => %r{^\s*require\(['"]([^'"]*)['"]\);?\s*$} ... 

Pay attention to the appeared "$" sign. That is, we have limited the regular expression to the end of the line. After that, the compiled file looks like it should:

 #  application.js (function() { var file1; file1 = true; }).call(this); (function() { var file3; file3 = true; }).call(this); (function() { var file2; file2 = true; }).call(this); (function() { }).call(this); 

Elegant (note the order of the files). If you want to wrap the whole thing in one more javascript function (I don’t know why, but you never know), you can do the following. Create your own filter:

 #  Assetfile class ClosureFilter < Rake::Pipeline::Filter def generate_output(inputs, output) inputs.each do |input| # output.write "(function() {\n#{input.read}\n})()" end end end 

And now this filter remains to be specified after applying the neuter filter.

 #  Assetfile input "target/src" do match "main.js" do neuter( ............. ) do |filename| "#{name}.js" end filter ClosureFilter end end 

Now everything is in order. It remains only to make a minimized version of our application. To do this, you need to write only 5 lines:

 #  Assetfile input "target" do match "#{name}.js" do # uglify -    uglify{ "#{name}.min.js" } end end 

Now, when compiling, in addition to “application.js”, the file “application.min.js” will be created with the contents:

 (function(){(function(){var e;e=!0}).call(this),function(){var e;e=!0}.call(this),function(){var e;e=!0}.call(this),function(){}.call(this)})(); 

Final version of my assetfile

 #  Assetfile require "json" require "rake-pipeline-web-filters" name="application" output "target" input "src" do match "**/*.coffee" do filter Rake::Pipeline::Web::Filters::CoffeeScriptFilter do |filename| File.join("src/", filename.gsub('.coffee', '.js')) end end end class ClosureFilter < Rake::Pipeline::Filter def generate_output(inputs, output) inputs.each do |input| output.write "(function() {\n#{input.read}\n})()" end end end input "target/src" do match "main.js" do neuter( :additional_dependencies => proc { |input| Dir.glob(File.join(File.dirname(input.fullpath),'**','*.js')) }, :path_transform => proc { |path, input| "#{path}.js" }, :closure_wrap => false, :require_regexp => %r{^\s*require\(['"]([^'"]*)['"]\);?\s*$} ) do |filename| "#{name}.js" end filter ClosureFilter end end input "target" do match "#{name}.js" do uglify{ "#{name}.min.js" } end end # vim: filetype=ruby 

It remains only to note that the project structure may contain nested directories. If you need to connect a file from a subdirectory, you need to specify

 require("dir_name/file_name") 

You can also write your own filters, which, for example, will substitute the license text, version number, date and time of the last commit, temperature and humidity in your city at the time of assembly, etc. into the file.

If you're interested, in the next article I can show you how to organize javascript testing using phantom.js (testing from the console itself) and connecting template files at the build stage.