A week passed from the moment of
PR on the project “
What to do? ". I remind you that this project began as an experiment on the development of an average WEB-project entirely in JavaScript (Node.JS). Now I want to share with the community the results of this experiment, obtained useful experience, as well as a detailed map with a rake marked on it.
Episode 1: The Beginning of the Road
I set myself the following goals:
- Understand how convenient it is to develop conventional WEB-projects on Node.JS;
- Compare the speed of development on Node.JS with other technologies used (in my case it was PHP and Java);
- To get to the pitfalls that you do not come across on numerous synthetic examples;
- Assess the work of the finished project: stability, resistance to stress, the complexity of the support and development of such a project.
This project was my first experience in developing a full-fledged site on Node.JS, before that I used it only to write supporting services for working projects. I started with an introduction to the WEB frameworks for this platform. At that time there were several of them and there was a choice. At this stage, you need to understand that if you write in JavaScript in the same way as in PHP, or as in Java, or in any other language, this will not lead to anything good. You need to write in JavaScript as in JavaScript, otherwise why do we need it at all (thanks, Cap)? Therefore, we immediately pass by clouds of frameworks offering various implementations of classical inheritance, synchronous programming, etc. crutches, not peculiar to the language. I was looking for a "true" -JavaScript path and wanted to evaluate it, and not the PHP port in JavaScript.
Of all the frameworks, the choice fell on
Express . It seemed to me the least overloaded with excesses and at the same time it most fully corresponded to my requirements and could be easily supplemented with third-party modules.
')
From the framework, I wanted the following:
- MVC architecture;
- Routing;
- Work with the DBMS;
- Template engine;
- Multi-language support;
- Multithreading support;
- Minimal overhead projector.
Also, it should have been easy to understand and be a handy tool, and not another technology that requires additional study.
Episode 2: The Way of the Samurai
Having played a bit with Express on a test case, having understood its architecture and studied the source code, I decided to make my own framework that would satisfy all my needs and would not be overloaded with functionality for me. Bike - say you. Experience - I will say. One of the best ways to understand a system is to develop its analogue yourself. The framework was developed, debugged, and the first version of the site was based on it. The site was launched, and I began to monitor its work, simultaneously correcting pop-up errors and adding new features.
Episode 3: The Great Flood
Naturally, the first time on the site there were very few visitors to at least somehow objectively evaluate his work. However, after a few days, I discovered that the site was lying. He fell silently, there was nothing suspicious in the logs, but the fall was accompanied by a series of http requests quickly following each other - it was someone's robot. These requests could not be called a load, many robots scan sites with such speed, and it should not fall because of this. I picked up the site and tested it using the siege utility. And really - the site went even on a very small number of parallel requests, which completely did not meet my expectations. Logs were silent. The sadness grew. The excavation began.
Episode 4: The First Stone
Repeating the experiment, I watched what was going on with the server and discovered a problem - the node process was surviving all the available memory and was either cut down or hung up (stopped accepting http requests). I’ll make a reservation that during development I tried to keep track of memory consumption, since This is one of the most dangerous moments in long-running applications. I did not make extra closures, I did not lose var, in general I behaved carefully. Once again, I reviewed my code, I was convinced that the error was not in it and, frankly speaking, I was upset because there it would be easiest to fix it.
The debugger for Node.JS exists , it is quite convenient, but debugging Node.JS application is more difficult due to its asynchronous operation. I repent, I did not reach the 80th level in working with this wonderful tool and could not calculate the leakage. So ended my vacation. The time for learning the power of Node.JS has become much less, and I decided to postpone this matter until better times. Not satisfied with the result, I wrote a small utility that restarted the process every day. On such crutches the site worked for 4-5 months. Meanwhile, the development of Node.JS did not stand still. New versions came out one after another, and third-party modules also developed, getting rid of childhood diseases.
Episode 5: New Hope
And so, after a rather long break, in the
change log of Node.JS there appeared records about fixed memory leaks (versions 0.6.6 - 0.6.7). Also, starting with version 0.6.0,
cluster api hit the stable release, which allowed to get rid of unnecessary dancing to run several node processes. All this added enthusiasm and forced to look at the project from a new angle. By this time I have already seen which parts of my framework are not used, what is missing in it, which is not convenient. Plus, I decided to replace MySQL (it was originally used) with MongoDB.
Once again, after analyzing the popular frameworks, I realized that all of them, by and large, are assemblies of various modules and minor improvements “from myself”, and also dictate their own rules for building an application. All this drives in a rigid framework at a doubtful prize. If you need to update a separate module, and the framework conflicts with it, you will have to wait for the updated version to be added to it by the developers. Or if you want to use a module that this framework does not support, for example, your favorite template engine. What to do? Fork and add support, and then manually update the update while waiting for the framework developers to accept your pull request? Not!
The modular system Node.JS completely allows not to use frameworks (read - ready-made assemblies of modules), but to work with modules directly. In this additional functionality, instead of wrapping in the framework, you need to make a separate module. This approach greatly increases the flexibility of the system and improves code reuse.
So I had a project refactoring plan. It consisted of the following points:
- Abandon the use of the framework, go to the direct use of modules;
- Choose from a huge number of modules the most suitable for solving my problem;
- Replace MySQL with MongoDB;
- To organize the launch of several node processes, use the now native cluster api .
A few more weekends, and the plan was implemented. Everything went quite smoothly, the amount of code decreased, the structure of the application became more understandable. Everything has been tested and ready to run. Then I published a PR article on Habré and waited.
Episode 6: Epic Wines
Not to say that "habraeffekt" was huge. Just a couple of thousand people came. After a couple of hours the site fell, but not because of the load, but because of my mistake. Yes, in one hard-to-reach place, I made a typo in the variable name, which was the reason for the fall. Otherwise, no problems. The pages were given very quickly, the processor did not strain, the memory did not flow. For a week now I have not restarted the node, and the site continues to work stably. This result, after the previous sad experience, is very good.
findings
You cannot write to JavaScript just for JavaScript. Choosing a language for the reason that it is cool / new / popular (underline the necessary) is stupid. We need to clearly understand what we want from the technology and what it can give us, and also what it will ask in return. My main task was to identify all these “what” and get an answer to the question: “Is Node.JS suitable for developing average WEB projects”. My answer is yes, it does. But, as in any other technology, there are difficulties that will have to face. I begin, perhaps, with the minuses, to finish on a positive note.
Houston, we have problems
-one
The first thing you will encounter when developing on Node.JS is the lack of a full-fledged IDE that works out of the box. Of course, there are plugins for Eclipse and NetBeans, there is partial support in other IDEs. But it’s impossible to call them full-fledged solutions at the moment. There is a very promising development of
Cloud9 IDE . It is an IDE for JavaScript development right in the browser. This IDE is developing rapidly and is already actively used by many Node.JS developers, but at the same time, in my subjective opinion, it is not yet ready to work on large projects, although it is very convenient to use it for writing small modules. During the work on the project, I changed more than one IDE. I started with notepad ++, used Cloud9 IDE for a long time, tried others, I don’t remember their names, but eventually returned to NetBeans. Although there is no support for the Node.JS API, it is quite convenient to work with pure JavaScript there. It remains to wait and hopes for the emergence of full support. The development of
such a plugin is already underway.
-2
The second unpleasant moment will be the need to restart the application after making each change. Despite the existence of utilities that automate this process, it still takes some time. Just as quickly as in PHP, make a change and immediately see the result will not work. However, the application restarts fairly quickly - from one to several seconds, depending on the modules used. In this Node.JS, undoubtedly, wins over Java - there restarting a serious WEB application takes much longer.
-3
The next problem that node.JS beginners will encounter is most likely a memory leak. Memory should be monitored always and everywhere, but fast-living PHP scripts or small client JavaScript are very relaxing. In such applications, many developers do not particularly betray the value of losing a few kilobytes of memory, and some do not follow this indicator at all. Node.JS does not make such concessions. The application on Node.JS works for a long time, and in the event of a leak, each request to the site will take with it a piece of memory that will end very quickly, leading to known consequences. The specificity of JavaScript also contributes to the appearance of this type of error. One extra closure in a large field of view or one lost var can give you unforgettable debug hours. But there is a plus in this - this experience will force any developer to appreciate memory and control its use when programming in other languages. Personally, I do not consider this feature a minus.
-four
Another feature that causes inconvenience is the return of errors from the asynchronous code. For comparison, in PHP all the code is executed synchronously, so you can catch the error at any nesting level using the try-catch construct. We can wrap the controller's work into it and, in the event of an exceptional situation, make a throw, and the error “pops up” to the waiting handler, which will show the user a beautiful page of shame. Despite the fact that JavaScript also has a try-catch construct, it will not help us, because Most of the code works asynchronously (I / O operations). At the same time, exceptional situations, as a rule, arise not when a method is called, but when its callback, which is executed already outside the try-catch structure, works. To transmit information about errors in Node.JS, it is customary to use the first parameter of the callback function. Those. if we have an error, we call the callback function either with one single parameter describing it, or we set the first parameter to undefined / null, and in the following we pass the results of our functionality. In real applications, call nesting can be quite large and it is very inconvenient to send an error to the top each time. I believe that the problem can still be solved by applying the
AOP , but this topic is beyond the scope of this article.
-five
They also make nervous the simplest mistakes that quite often get to the production server. A typo in the name of a variable in a very rarely executed block of code can lead to stopping the entire site or one of the processes if cluser api is used. To avoid such troubles, you can use utilities that monitor the operation of the application and restart it if necessary, but real ninjas just test their code well (although no one is immune from an error in a third-party module).
-6
The last thing I would like to add is complex mathematical calculations not for Node.JS. On this topic on the hub,
there was already a very flame topic and there’s no point in repeating everything that is written in it and I don’t see comments to it. I can only say that the author of the topic (of course, I mean the author of the original, not the translation) has problems with the choice of technology, or he is just a Troll. In Node.JS, it is still possible to perform complex mathematical calculations, by splitting the problem into short iterations, performed in several turns of the event loop, but this is already a perversion and fanaticism. It is much easier to move such tasks outside the event loop or even choose another technology for development.
And for what all this?
If by this time you do not have such a question, then you are either an experienced JavaScript developer, or a search robot, or Chuck Norris (hello, Chuck!). For everyone else, I will try to describe the benefits that the development of a WEB project on Node.JS gives us.
+1
I consider the most significant advantage of Node.JS is asynchronous I / O and transparency of working with it. In most web projects, the most frequent operations are reading data from the database and saving it. These operations are usually the slowest. But they are not always dependent on each other, on the contrary, in most cases they are atomic. For example, when adding a new comment to an article, we need:
- Save the comment itself;
- Update user (for example, the number of comments from the user and the date of his last comment);
- Update article (similar to updating user);
- Log the result of the query.
All these operations are independent of each other and the DBMS is able to perform them simultaneously. When performing synchronously, for example in PHP, these operations will be performed one after the other sequentially, each time waiting for the completion of the previous one. In Node.JS we have the opportunity to send all 4 of these requests to the DBMS “in parallel”. In fact, requests are still sent sequentially (so I quoted the word "parallel" in quotes), because Node.JS works in the same process. But Node.JS does not wait for the result of the previous request to send the next one. The time of sending the request, compared with the time of its work, while can be neglected. When requests are executed, callback functions will be called to process their result. These calls will occur as consistently as sending requests, and the result will be processed incomparably faster than their execution time. Thus, the total time to work with the data will be approximately equal to the time of the longest query + a small overhead to send queries and process their results. But in any case, it will be faster than the sum of the execution time of all queries during their sequential processing (the overhead to send queries and process their results in this case also does not go anywhere).
+2
The second very pleasant possibility is that the answer to the client can (and should) be sent immediately, as it will be ready, without waiting for the completion of the work of the entire query logic. Let's go back to the previous example. When adding a comment, we write a log in the database. We need this operation, but we don’t need the user to get an answer. With Node.JS we can form and send the answer, and then finish what we need: write logs, clear the cache, etc. The user timeout is reduced accordingly.
+3
The third positive factor is that we have the same language on the server and on the client, this allows you to reuse some code (validation of forms, building patterns on the client, etc.). And in general, it greatly simplifies the development, especially when the client part of the application is complex and requires serious work, and not the processing of a couple of events on the buttons.
+4
The Node.JS process lives long and handles all http requests within itself. This means that for each new request, initialization is not performed, as, for example, in PHP. The settings are loaded, connections to the database and the cache are open, the code is compiled and ready to go. Thanks to this architecture and the flexibility of JavaScript, there is a huge scope for various optimization techniques. For example, once you have parsed a template, you can store it as a function that takes data as input and returns ready-made HTML. Or you can easily organize local (for the process) caching of the most frequently used data, which will give an increase in the speed of working with them even compared to memcached. Yes, there are solutions for PHP that allow to partially speed up the initialization process - APC for op-code, support for persistent connections in FastCGI, etc. But when comparing the accelerated initialization process with its absence in principle - the gain will always be the last.
+5
Around the Node.JS for its relatively short life a solid ecosystem has already formed, including hundreds of modules and, accordingly, their developers. Largely due to the convenience of github, the community has an excellent tool for the development of this ecosystem, which is taking leaps and bounds, and anyone can make a contribution to this development. Using tools such as
The Node Toolbox and
npm , the process of finding, selecting, and installing the necessary modules becomes simple and fast.
+6
The JavaScript language itself and the Node.JS API are very flexible and concise. Programs are compact and easy to read. You are unlikely to see in them classes consisting almost entirely of getters and setters, or dozens of files with interface descriptions. Such things as closures and lambda functions allow you to write very beautiful code, but with the special talent of a developer they can turn it into a branch of hell.
I laughed for a long time, seeing the
performance comparison of Java and Node.JS. Here you can talk for a long time and give various arguments, but I will say simply how I think. In the event loop, I spit such performance if the program code increases tenfold. I have experience in developing a large WEB-project in Java + Spring Framework + Hibernate. There are 10 lines of code, beautifully describe what the 11th will do. Crude, of course, but it roughly reflects the situation. Perhaps for some class of problems this is also relevant, but not for average WEB-projects. It is worth adding that such a comparison on the “Hello world” does not reflect the real state of affairs. With the appearance of application logic, work with the database, caching and other components of the system, the probability of catching the brakes due to the non-optimal use of the technology is greatly increased than due to its imperfection. In addition, we must remember that the server resources may be missing in two cases:
- The code is written poorly;
- Your site has a huge attendance.
In the first case, only straightening the arms with the subsequent rewriting of the code and technology will do nothing to do with it. In the second - you already have enough money to buy another server, because it is economically more profitable than the support of tens of times more code.
Conclusion
Node.JS is not a magic wand. This is a powerful tool with its pros and cons, which should be used only when you need it. I am satisfied with the result of my experiment.
The project is fast, has a simple and clear code that is easy to develop and maintain. Despite some shortcomings, Node.JS is convenient to use for the development of WEB-projects and I believe in its development in this direction.
To be continued
In the second part I will focus on the application architecture. I'll tell you what modules I used and how the interaction between them is organized.
Thanks to everyone who read it. I will be glad to see your feedback and opinions in the comments.