Advanced Gulp and Browserify: interesting tricks
A couple of weeks ago, I started a cycle about how a non-profit musical project was doing (the first post is in “I am PR”, I will not put links), but, unfortunately, I got carried away in the very first article, and instead of telling me about how I did it specifically, I began to recall effective tricks from other projects. Apparently, this, together with the prescribed emphasis on the project itself, led to the fact that a UFO flew behind me and the post.
However, all that was in the article was at least little known, and half of it was unique at all, as I am sure, and each of these tips can significantly facilitate the work with gulp, so I really would be sorry if this material was irretrievably lost. would.
Therefore, I tried to remove all references to the project and re-publish (with revisions and corrections) an article that no one has yet seen. If you are a grunt fan - read at least the second part: just because you don’t like gulp doesn’t mean that you don’t like browserify.
')
Summary:
I switched to gulp while working on one video portal. Now they are doing well, they are developing, and it seems that my build system is still with them.
Then I switched to the stylus .
Why? Extremely fast.
I cite many figures from memory, so I can lie a little, but the proportions are the same, I remember them exactly.
Introductory:
Everything indicates that there is a plug in the work with the hard disk, and in only one of the three tasks, everything can be solved by some kind of cache.
Studying the problem also indicated that styles compiled the longest.
The solution to the problem was as follows:
As a result, the assembly of the entire project began to take 3-4 seconds, some types of files - about a second.
Watchdog was working on it all instantly, of course.
Since then I have been using the gulp + browserify stack.
One of the advantages of browser browsing is wrapping the widgets into a closure, plus, in fact, the simplest validation of the code - it will not miss the code that cannot be parsed.
However, it was hampered by the fact that the failure code forced the collector to crash. This was not the case, of course.
At first the decision was something like:
But, as it turned out, this is not very good: the stream “freezes” and does not restart after the update.
Why is this happening? The fact is that the task is not completed, but it remains to hang in this state, and in the gulp, it seems that restarting is impossible without ending the previous execution.
I have been looking for a good solution for this question for a long time, but over time I wrote my function to solve this problem:
It also closes the stream (this.end ()), causing the completion of the task.
If desired, you can add, for example, growl alerts, but I personally have enough.
The function requires the preset npm package color and gives a very nice output. If you do not want to put extra packages - you can remove the methods from the flowers.
The most important thing here is the last line.
When we execute this.end (), the particular gulp task completes its work. Yes, it is a little shattered in memory, but watchdog-task will be able to re-launch your style assembly when you update them.
It looks like this:
If everything is neatly arranged in daddies like:
So I congratulate you.
But for me personally, everything is lying around like this:
And it is convenient, much more convenient than it was before. Why? Yes, because I had the opportunity to structure the files into each other in any logical way, without changing anything in the collector.
You write @require in styles, layouts and inclusions in templates and browserify for scripts, everything just works.
As a result, all this is going to index.html, app.js and style.css - the very basis for any project.
How did I get it?
In all projects, I try to stick with this scheme:
What is this glob path like that?
This is a selection of all files that do not begin with an underscore. At any depth. Accordingly, if you name the file src / lib / _some_lib.js, it will not be compiled by yourself. But require him to pick up with pleasure.
Now I do not use this technique, because I switched to the scheme with the inclusions of everything and everything in the code, I write mostly from memory, so I can lie a little.
But it is very interesting, and in due time I did not find it anywhere.
When I needed to solve a task like “glue all the CoffeeScript files and js files from the vendor folder, and then from the main one,” at first I was upset because I didn’t know what to do. Why such a sequence - I think it is clear - vendor scripts should be loaded first, and if you do it somehow else, everything will be mixed up.
But I knew that if there was something in memory, then it could be used, and I began to dig. Still, gulp uses native nodejs streams, which means you can do something about it.
Came to a homemade solution:
Please note : judging by the new event-stream documentation, the concat method has been renamed to merge. I did this last half a year ago, so now the method may have new subtleties of use - the code is taken from a real, relatively old project that works with the old version of EventStream.
When you have 10-20 plug-ins, it becomes somewhat tedious to prescribe them manually.
To do this, there is another plugin that creates a plugins object with plug-in methods, but all the same can be made much clearer and simpler:
If someone doesn’t understand what exactly this code does, it opens the contents of devDependencies in package.json and all elements that begin with gulp- in it connects as plugins [pluginName]. If the plugin is called something like gulp-css-base64, it will be available via plugins.css_base64.
Sometimes you need to create something in memory and send it to the stream (but at least with the same gluing). Again, there is a plugin for this, but why? If you can write everything yourself in three lines.
It works all over Vynil FS from gulp-util, but what difference does it make to us?
Why browserify in a post about gulp? Yes, because it can be called a meta-assembly system, which is used in other systems. Its capabilities have long since gone beyond the simple gluing of js-modules, and in the next part of the post everything will come together.
If you use browserify and commonJS modules - tell me honestly, did you ever want to write like that?
This is the real code from the project itself, for the post about which a UFO came flying behind me, by the way.
As it turned out, rip your browserify plugins - elementary.
The real task for building JS looks like this:
What is this ... and how does it work? Yes, very simple.
The simplest wrapper looks something like this:
In the future, I will quote only the compile function - to save space.
If there are browserify Ninjas who already know all the plugins to intrigue, they will ask "so what?"
Yes, nothing.
In this form, plugins already exist.
But the trick is that we can change the syntax.
For example:
And now in the jade template we can write
As a result, we can include templates on jade in js, and styles in templates on jade.
We can connect several collectors at once, for example:
This is what we are doing the JS function of the template on DoT (handlebars-like templating engine over HTML) wrapped in Jade.
And we can even ...
… drumroll…
... use gulp plugins to create browserify plugins that we can connect as a gulp task
and finally the denouement of the whole post. We can turn this data string into a stream (which I just talked about in the middle of the post), which can be used with gulp. We take the function I showed above and get ...
Once again, very carefully:
We just skipped a bunch of gulp plugins that went into browserify.
Yes, it turns out a little hemorrhoids. But the result is worth it.
What for?To the glory of Satan, of course, because it was impossible to simply assemble and configure Stylus-styles in a browserify that would also suck out base64-pictures, and pass through autoprefixer and minification.
gulp is an amazingly elegant system that can be customized for most cases. And the fact that its plugins can be used in browserify (and, therefore, other projects) is generally brilliant. Yes, a little hemorrhoids, but this is something.
I hope you learned something new. More precisely, I am sure of this, but I wanted to say beautifully.
And I hope that the UFO will return me to Habr and give you a talk about neural networks inside Web Workers and algorithms that can produce accurate recommendations on the user's musical preferences based on an extremely small amount of data.
However, all that was in the article was at least little known, and half of it was unique at all, as I am sure, and each of these tips can significantly facilitate the work with gulp, so I really would be sorry if this material was irretrievably lost. would.
Therefore, I tried to remove all references to the project and re-publish (with revisions and corrections) an article that no one has yet seen. If you are a grunt fan - read at least the second part: just because you don’t like gulp doesn’t mean that you don’t like browserify.
')
Summary:
- A simple way to handle errors;
- Universal structure for storing source files;
- Combining multiple threads (for example, compiled coffee and js) into one;
- Creating a stream from text;
- creating your own plugins for Browserify;
- creating plugins from Gulp plugins for Browserify.
I switched to gulp while working on one video portal. Now they are doing well, they are developing, and it seems that my build system is still with them.
Then I switched to the stylus .
Why? Extremely fast.
I cite many figures from memory, so I can lie a little, but the proportions are the same, I remember them exactly.
Introductory:
- grunt;
- There is a large number (73, if not confusing) of style files that are almost unrelated to each other, the assembly is glued together. Basically styles are components and page hooks and layouts, so that they are independent of each other, in almost everyone only a set of variables is connected;
- There are a large number of scripts that stick together in one. CoffeeScript is compiled, normal js does not change. Each file turns into a closure;
- There is an application file that is assembled from a large number of precompiled jade templates through some specialized plug-in that made an object with all the templates from the folder in global space, plus it was glued to the main application engine. In fact, the output is also a js-file;
- The assembly time for each item is about 10 seconds on a Macbook Air 2013, i5, for a total of about 30-40 seconds. I remind you that the MBA is SSD, which is connected via PCIe and gives the speed of work with the hard disk, which seems to be basically unattainable on a regular sata connection. On computers with HDD assembly time may take more than a minute.
Everything indicates that there is a plug in the work with the hard disk, and in only one of the three tasks, everything can be solved by some kind of cache.
Studying the problem also indicated that styles compiled the longest.
The solution to the problem was as follows:
- replace grunt with gulp. The problem is solved with the compilation, and then by gluing together styles and scripts - the step of writing each individual file to disk is removed;
- replace sass with stylus, translate everything in the global file. Compiling styles is accelerated to less than a second. Apparently, the transfer of each file from node to Ruby ate a lot of resources. And ruby-sass is not very fast yet. The transfer, by the way, happened without any problems at all - sass was used basic, without mixins and functions, and from a certain point of view the sass format can be called a subset of the stylus format;
- Translate all coffeescript to JS to speed up compilation - good, these were mostly old widgets;
- Translate js build for application to browserify for caching.
As a result, the assembly of the entire project began to take 3-4 seconds, some types of files - about a second.
Watchdog was working on it all instantly, of course.
Since then I have been using the gulp + browserify stack.
One of the advantages of browser browsing is wrapping the widgets into a closure, plus, in fact, the simplest validation of the code - it will not miss the code that cannot be parsed.
However, it was hampered by the fact that the failure code forced the collector to crash. This was not the case, of course.
Error processing
At first the decision was something like:
gulp.task('build-html', function () { ... .pipe(plugins.jade()).on('error', console.log.bind(console)) ... But, as it turned out, this is not very good: the stream “freezes” and does not restart after the update.
Why is this happening? The fact is that the task is not completed, but it remains to hang in this state, and in the gulp, it seems that restarting is impossible without ending the previous execution.
I have been looking for a good solution for this question for a long time, but over time I wrote my function to solve this problem:
function log(error) { console.log([ '', "----------ERROR MESSAGE START----------".bold.red.underline, ("[" + error.name + " in " + error.plugin + "]").red.bold.inverse, error.message, "----------ERROR MESSAGE END----------".bold.red.underline, '' ].join('\n')); this.end(); } gulp.task('build-html', function () { ... .pipe(plugins.jade()).on('error', log) ... It also closes the stream (this.end ()), causing the completion of the task.
If desired, you can add, for example, growl alerts, but I personally have enough.
The function requires the preset npm package color and gives a very nice output. If you do not want to put extra packages - you can remove the methods from the flowers.
The most important thing here is the last line.
When we execute this.end (), the particular gulp task completes its work. Yes, it is a little shattered in memory, but watchdog-task will be able to re-launch your style assembly when you update them.
It looks like this:
Folders and files
If everything is neatly arranged in daddies like:
- assets
- styles
- scripts
- templates
So I congratulate you.
But for me personally, everything is lying around like this:
And it is convenient, much more convenient than it was before. Why? Yes, because I had the opportunity to structure the files into each other in any logical way, without changing anything in the collector.
You write @require in styles, layouts and inclusions in templates and browserify for scripts, everything just works.
As a result, all this is going to index.html, app.js and style.css - the very basis for any project.
How did I get it?
In all projects, I try to stick with this scheme:
gulp.task('build-js', function () { return gulp.src('src/**/[^_]*.js') ... gulp.task('build-html', function () { return gulp.src('src/**/[^_]*.jade') ... gulp.task('build-css', function () { return gulp.src('src/**/[^_]*.styl') ... What is this glob path like that?
This is a selection of all files that do not begin with an underscore. At any depth. Accordingly, if you name the file src / lib / _some_lib.js, it will not be compiled by yourself. But require him to pick up with pleasure.
Pasting results of different tasks
Now I do not use this technique, because I switched to the scheme with the inclusions of everything and everything in the code, I write mostly from memory, so I can lie a little.
But it is very interesting, and in due time I did not find it anywhere.
When I needed to solve a task like “glue all the CoffeeScript files and js files from the vendor folder, and then from the main one,” at first I was upset because I didn’t know what to do. Why such a sequence - I think it is clear - vendor scripts should be loaded first, and if you do it somehow else, everything will be mixed up.
But I knew that if there was something in memory, then it could be used, and I began to dig. Still, gulp uses native nodejs streams, which means you can do something about it.
Came to a homemade solution:
var es = require('event-stream'); gulp.task('build', function(){ return es.concat( gulp.src('scripts/vendor/*.coffee').pipe(coffee()), gulp.src('scripts/vendor/*.js'), gulp.src('scripts/*.coffee').pipe(coffee()), gulp.src('scripts/*.js') ) .pipe(concat()) .pipe(dest(...)); }) Please note : judging by the new event-stream documentation, the concat method has been renamed to merge. I did this last half a year ago, so now the method may have new subtleties of use - the code is taken from a real, relatively old project that works with the old version of EventStream.
Plug-in connection
When you have 10-20 plug-ins, it becomes somewhat tedious to prescribe them manually.
To do this, there is another plugin that creates a plugins object with plug-in methods, but all the same can be made much clearer and simpler:
var gulp = require('gulp'), plugins = {}; Object.keys(require('./package.json')['devDependencies']) .filter(function (pkg) { return pkg.indexOf('gulp-') === 0; }) .forEach(function (pkg) { plugins[pkg.replace('gulp-', '').replace(/-/g, '_')] = require(pkg); }); If someone doesn’t understand what exactly this code does, it opens the contents of devDependencies in package.json and all elements that begin with gulp- in it connects as plugins [pluginName]. If the plugin is called something like gulp-css-base64, it will be available via plugins.css_base64.
How to create a stream from text
Sometimes you need to create something in memory and send it to the stream (but at least with the same gluing). Again, there is a plugin for this, but why? If you can write everything yourself in three lines.
var gutil = require('gulp-util'); function string_from_src(filename, string) { var src = require('stream').Readable({objectMode: true}); src._read = function () { this.push(new gutil.File({cwd: "", base: "", path: filename, contents: new Buffer(string)})); this.push(null); }; return src; } It works all over Vynil FS from gulp-util, but what difference does it make to us?
Browserify Plugins
Why browserify in a post about gulp? Yes, because it can be called a meta-assembly system, which is used in other systems. Its capabilities have long since gone beyond the simple gluing of js-modules, and in the next part of the post everything will come together.
If you use browserify and commonJS modules - tell me honestly, did you ever want to write like that?
var vm = new Vue({ template: require('./templates/_app.html.jade'), ... This is the real code from the project itself, for the post about which a UFO came flying behind me, by the way.
As it turned out, rip your browserify plugins - elementary.
The real task for building JS looks like this:
gulp.task('build-js', function () { return gulp.src('src/**/[^_]*.js') .pipe(plugins.browserify( { transform: [require('./lib/html-jadeify'), 'es6ify'], debug : true } )).on("error", log) .pipe(gulp.dest("build")); }); What is this ... and how does it work? Yes, very simple.
The simplest wrapper looks something like this:
var through = require('through'), jade = require('jade'); function Jadify(file) { var data = ''; if (/\.html\.jade$/.test(file) === false) return through(); else return through(write, end); function write(buf) { data += buf; } function end() { compile(file, data, function (error, result) { if (error) stream.emit('error', error); else stream.queue(result); stream.queue(null); }); } } function compile(file, data, callback) { callback(null, 'module.exports = "' + jade.render(data, {filename: file})+ '"\n'; ); } Jadify.compile = compile; Jadify.sourceMap = true; // use source maps by default module.exports = Jadify; In the future, I will quote only the compile function - to save space.
If there are browserify Ninjas who already know all the plugins to intrigue, they will ask "so what?"
Yes, nothing.
In this form, plugins already exist.
But the trick is that we can change the syntax.
For example:
callback(null, 'module.exports = "' + jade.render(data, {filename: file}) .replace(/"/mg, '\\"') .replace(/\n/mg, '\\n') .replace(/@inject '([^']*)'/mg, '"+require("$1")+"') + '"\n' ); And now in the jade template we can write
style @inject './_font_styles.styl' As a result, we can include templates on jade in js, and styles in templates on jade.
We can connect several collectors at once, for example:
callback(null, 'module.exports = ' + dot.template(jade.render(data, { filename: file })) + '\n'); This is what we are doing the JS function of the template on DoT (handlebars-like templating engine over HTML) wrapped in Jade.
And we can even ...
… drumroll…
... use gulp plugins to create browserify plugins that we can connect as a gulp task
and finally the denouement of the whole post. We can turn this data string into a stream (which I just talked about in the middle of the post), which can be used with gulp. We take the function I showed above and get ...
function string_src(filename, string) { var src = require('stream').Readable({ objectMode: true }); src._read = function () { this.push(new gutil.File({ cwd: "", base: "", path: filename, contents: new Buffer(string) })); this.push(null) }; return src; } function compile(path, data, cb) { string_src(path, data) .pipe(gulp_stylus()) .pipe(gulp_css_base64({maxWeightResource: 32 * 1024})) .pipe(gulp_autoprefixer()) .pipe(gulp_cssmin()) .on('data', function(file){ cb(null, "module.exports = \""+ file.contents.toString() .replace(/"/mg, '\\"') .replace(/\n/mg, '\\n') + '"'); }) .on('error', cb); } Once again, very carefully:
string_src(path, data) .pipe(gulp_stylus()) .pipe(gulp_css_base64({maxWeightResource: 32 * 1024})) .pipe(gulp_autoprefixer()) .pipe(gulp_cssmin()) .on('data', function(file){ cb(null, "module.exports = \""+ file.contents.toString() .replace(/"/mg, '\\"') .replace(/\n/mg, '\\n') + '"'); }) We just skipped a bunch of gulp plugins that went into browserify.
Yes, it turns out a little hemorrhoids. But the result is worth it.
What for?
Conclusion
gulp is an amazingly elegant system that can be customized for most cases. And the fact that its plugins can be used in browserify (and, therefore, other projects) is generally brilliant. Yes, a little hemorrhoids, but this is something.
I hope you learned something new. More precisely, I am sure of this, but I wanted to say beautifully.
And I hope that the UFO will return me to Habr and give you a talk about neural networks inside Web Workers and algorithms that can produce accurate recommendations on the user's musical preferences based on an extremely small amount of data.
Source: https://habr.com/ru/post/242379/
All Articles