Node.js: Overview of General Library Development Technologies

In this post I want to summarize and share the experience gained in developing the node-queue-lib library. I will talk about the technologies that helped me bring the matter to the final end - workable code that is already running on one of my services. A feature of this library is a cross-platform client, i.e. client running in node.js and browser and based on the same code. The post will describe the following tools, without which the development of this library would turn into hell:

• Testing ( jasmine_node )
• Code Coverage Tests ( Istanbul )
• Build client part of the library ( browserify )
• Automated client testing ( phantomjs )
• Memory leak detection ( memwatch )

If you haven’t heard anything about it yet and you want to write a finished product that you will be sure of, this review article will help you get acquainted with one of the options of the toolkit for the full quality control of the javascript library code.

And additionally, I will repeat that the article is an overview, and does not set a goal to teach you to use all the listed tools masterly. I will only show the door, but you will open it yourself ...

Retreat

All examples will contain code for the grunt build system. If you are not familiar with what it is, it's time to do it, otherwise you will regularly stumble over the console on the way to success. If you are "advanced", do not shout in the comments "gulp" , adapting the code for your build system for you will not be a problem.

Testing

There are now plenty of libraries to test. And you have to make it and choose the testing model you like. I do not impose the library I use to you, because the main thing is to have tests, and the choice of the library is secondary. My choice in this case was made for the following reasons:

1. Some historical moments in which I have already used jasmine.
2. Support for node.js and browser by one library, since there is a reuse of code.
3. The maturity of the library, and as a result, the presence of integration in various tools.

')
Installation

npm install grunt-jasmine-node --save-dev 

Gruntfile.js configuration

 jasmine_node: { options: { forceExit: true, match: '.', matchall: false, extensions: 'js', specFolders : ['./spec/'] }, all: ['spec/'] } 

I will not teach you how to write tests, there is plenty of such material. I'd rather go to the second tool, the purpose of which is to show that you not only write tests, but also which parts of the code your tests are testing. After all, it is completely pointless to write tests without presenting the general picture of the tested code.

Code Coverage Analysis

There is no valid reason for the library code not being 100% covered by tests. Each condition in the code must be covered with a test, otherwise you will encounter unexpected behavior or write code that never works. If the code is not covered by tests for 100% confidence in the library falls. To check the code coverage with tests, you must use the istanbul library. By itself, Istanbul in the "bare" form is not interesting to us. It is necessary to integrate it into the test run system. This is what the grunt-jasmine-node-coverage module does. It will allow you to get a report in html-form (and not only) for comfortable analysis of source code coverage.

Installation

 npm install grunt-jasmine-node-coverage --save-dev 

Since we integrate jasmine with istanbul, you just need to change the configuration of jasmine

 jasmine_node: { coverage: {}, //     options: { forceExit: true, match: '.', matchall: false, extensions: 'js', specFolders : ['./spec/'] }, all: ['spec/'] } 

By the way, the analysis of code coverage by tests made me take a little different look at some fragments. For example, from the analysis of some parts of the code, I found that the behavior of the function in some if constructions is not defined. Simply put, there was no else block. Refactoring one of these functions led me to understand that the whole class and interaction with it was written incorrectly and a decent amount of code was rewritten. It also revealed quite a few weak points in error control. Somewhere they did not exist at all, somewhere they were poorly checked, and so on. So use code coverage analysis - it really makes you look at your code completely different . Moreover, the use of the tool is so simple, and the reports are completely predictable.

These tools are enough to make sure that you control the quality of your code. And further material will reveal more complex aspects of the development of libraries.

Build the client part of the library

If you are creating a cross-platform library (node.js and browser) you will need a tool to build the client part of the library. To turn the code for node.js into code that can be connected to the browser, you need to use the browserify tool.

Installation

 npm install grunt-browserify --save-dev 

Configuration

 browserify: { dist: { files: { 'dist/my-library.js': ['lib/my-library.js'] } } } 

The configuration is quite understandable, provided that you have the final files in the / dist folder, and the sources in the / lib folder. Do not forget to minimize the resulting file using uglify. The topic is so jaded that I will not disclose it in my article. Just in case I will leave only the link .

Automated client testing

No need to rely on transparency, which gives technology from the previous chapter. Test your code in a browser. Especially if it uses external libraries like yours, compiled using browserify. Given that jasmine is available both under node.js and under browser, tests can be reused. Just add a test file to your browserify configuration and configure jasmine to run it. The configuration will look something like this:

 browserify: { dist: { files: { 'dist/my-library.js': ['lib/my-library.js'], 'dist/my-library.spec.js': ['spec/my-library.spec.js'] } } } 

In my case, I used the rather popular socket.io library and when I first started integration tests in the browser, I found that not a single test passed, because I completely in vain set the socket.io option, which for each request creates a new connection. For the browser, it was fatal and I had to pretty well refactor my code under node.js to work without the above option. Test your code in the browser after using various tools .

So, customer testing is required, but it has not yet been automated. jasmine assumes that the browser opens an html page (called TestRunner). But we need to test conducted in automatic mode. For this, a project called phantomjs will help us. The essence of the technology is that it is a full-fledged browser, but only without UI. That is, in general, without any UI, - graphic or text. This browser is controlled by a simple javascript script run in phantomjs.

Installation

 npm install phantomjs --save-dev 

Using. To run our TestRunner.html and get the test results you need the following:
1. Write a node.js specification file that will run phantom.js (to run automatically in jasmine)
2. Write a phantomjs script that will do the following:

• Wait until the end of jasmine.
• Parry the DOM of the resulting page and find out how many tests have dropped. (There are suspicions that there is a better way than parsing the DOM).
• Finish phantomjs.

3. To analyze the result of phantomjs for messages about the fall of the tests.

In general, this part will have to tinker well. I perfectly imagine that the person who hears for the first time about the technology of phantom.js did not understand anything at all from the list above. It does not matter. You only need to open the corresponding pages of the phantomjs documentation, read and understand. And you just have to master this technology, you have pretty good prospects for automating the testing of UI.
For inspiration, you can look at the already implemented scenario in the aforementioned library, on the experience of the development of which this article appeared.


Well, now that your code is 100% tested; tested in various environments, with some small reservations of course, you should rightly be bursting with pride in your brainchild. But ... there is another obstacle on your way without overcoming which your fingers will most likely be cut off.

Memory leak detection

Even if you did everything correctly and your code is 100% tested, it does not give you any guarantee that the library is not “flowing.” And if in the browser, for objective reasons, it was killed, the “flowing” server service, as it is not difficult to guess, is a failure.
A less working tool for finding memory leaks that I could find is memwatch ( article on habre ). Unfortunately, I failed to automate its use. There is no stability of the results (or I missed something).
As stated in the memwatch itself, the heap dump is always removed after running the garbage collector. But nevertheless, some allegedly unclaimed objects still flash in the report. The real test with billions of iterations of sample test-case `s showed that the library does not eat memory and works stably.
But this, of course, is not an argument against using the tool. Perhaps in the future it will work stably. I even hope that these are not his problems, but node.js. And in new versions it can show the best results.

By the way, with the help of this tool, I found that socket.io does not flow. And I found a decent amount of leaks in my code. And these leaks were exclusively in the use of the same class: EventEmitter. In principle, a lot has been written about this, that EventEmitter is “gifts bringing ...”. But as they say, experience is an invaluable thing. Purchase it, use automation tools in your work and we will see a lot of quality work.