On the eve of the launch of the Node.js school from Yandex. Money, I would like to tell a little more about why this platform has taken root in our front end.

A few years ago, in Yandex.Money, there was a change of platform for the front-end server layer: the one that was available was an internal proprietary development and was gradually dying from weak support and sagging performance speed. Together with the slow work in runtime and the lack of development of XSLT, on which the API worked, this “black box” with many restrictions has come it’s time to replace.

By "frontend" we understand not only the code executed in the browser, but also the server layer for data collection and HTML generation. Node.js became a good substitute for the logic available.

Why Node.js

Instead of the traditional thread-based concurrency model, Node.js uses the principles of event-oriented systems — compared to the “one thread per connection” approach, the code is simpler and faster. Node.js became popular thanks to the extensive NPM repository, an impressive developer community, and the ability to use JavaScript on the client, on the server, and for developing tools. In general, our team liked all these advantages, so the choice was obvious.

Since the Yandex.Money frontend is a collection of the client and Node.js, everything that the user interacts with or can get to gets into view of the front-end developer. At the same time, the “thick” business logic lives in separate backend components, which are provided by the public HTTP-based API for Node.js applications. Backends know nothing about page entities and interface components, but they manipulate user entities, payments, settings, etc.

From additional bonuses for the transition to a new technology - there is no need to retrain developers, and now not only one person can be engaged in supporting tool code.

How is the new framework settled in our front end?

Our payment system is characterized by exactly the load for which Node.js was originally developed: many I / O operations (incoming requests and backend calls) and little work in one cycle, without complex synchronous calculations. The result is a “live” event cycle in which the execution of operations is never blocked for long.

Consider the logic of processing user requests in our Node.js applications:

1. The HTTP server on Node.js listens for incoming requests from users;

   
   

2. it also processes the data from the request: parsing cookies, parsing the body of the post request, logging information about the request, etc .;

   
   

3. then the URL is redirected to the processing logic:

   
   

   1. in the process, the data required for the page is requested, they are aggregated and the subsequent execution of business logic;

      
      
   2. the server renders HTML from the collected data or generates another suitable response to the client (for example, json, binary file or redirect);
   
   
4. Node.js server places general response headers and sends the response to the client.

To implement all the processing logic of the request, we use Express , the popular Node.js framework. And for internal applications, Koa 2 is used , which allows you to write the entire request processing flow using Async Functions , a new JavaScript feature that greatly simplifies writing asynchronous JavaScript code.

Moreover, Koa 2 will most likely appear on our external applications when Node.js 8 becomes LTS, which means it is suitable for installation on production servers. It is in the eighth version that the V8 engine is used with the new TurboFan compiler, which optimizes Async Functions and allows them to be used in production.

You can learn more about V8 and the new optimizing compiler in the translation of the article by V8 developer Benedikt Meurer .

Static files, such as js, css and images, are not distributed to Node.js by the server - for this there are more suitable web servers like Nginx. In addition, more flexible caching control is available in Nginx. Therefore, it is better to install a special web server in front of the Node.js application.

Although Node.js executes the JavaScript code of your application in a single thread, it would be rash to start the process of processing all user requests for production into one stream, especially if you have multiple cores. Therefore, we use clustering, that is, we generate a whole pool of individual Node.js processes — workers — from the main master process.

Metrics and graphs

Like all our other services, the frontend is connected to the monitoring, and all its key indicators are displayed in real time on numerous team dashboards. Supervision over schedules especially fascinates in days of releases or, for example, at a configuration of iron.

Graphite and Prometheus are great for collecting metrics, and Grafana is responsible for a visual display of everything collected.

Teams hang right in the office with teams displaying graphs and metrics of interest to them.

The dynamics of the key indicators of the frontend system.

The frontend team also has its own dashboard - the following graphs are displayed in real time:

• HTML rendering time across all pages;

  
  

• number of incoming requests;

  
  

• processing time for incoming requests;

  
  

• the number of 5xx and 4xx status codes in the responses;

  
  

• execution time of outgoing requests to backends for all calls;

  
  
• Number of unsuccessful outgoing requests to backends.

Why for all this a separate TV? First of all, you don’t always look into the mail and follow the alerts, but here everything is in front of your eyes and you don’t even need to turn on the laptop. Secondly, it is possible to “stick” in the morning with the results of yesterday’s labor.

What are we faced

The choice of Node.js as a platform for the server layer, in addition to many advantages, also entailed a certain amount of headaches for developers.

For example, such large applications as ours, even with a good coverage of tests and documentation, each update is not easy: the constant presence in the change process comes not only from the desire of developers to please themselves or improve their product, but also from the realities of the modern world of JavaScript development .

We use Node.js in conjunction with the binding of C ++ code in JavaScript, which we cannot abandon due to the existing infrastructure. With each update of Node.js, we are faced not only with the standard difficulties of updating the migration guides, but also with updating our binding to support the next version of Node.js: libxml, libxslt and our internal C ++ libraries.

In general, dependence on binders introduces additional overhead costs for support, so they should be avoided - for example, we try to translate our internal binders into separate HTTP services.

A separate story is the asynchrony of Node.js, which still needs to be able to "prepare." For example, at some point we noticed that from time to time the working worker fell due to a business logic error and restarted, although all the logic code in Express and Koa is wrapped in error catching and should be intercepted. That is, the business logic described in the handlers of the route should not stop the whole Node.js process, but this happened to us all the same.

An analysis of the situation revealed a curious feature. When sent to the callback function in process.nextTick , it will start in the current clock cycle of the asynchronous cycle after the rest of the current clock code, including the error trapping code, is executed. If Exception occurred inside such a function, it was not caught, because process.nextTick is not at all the “next clock”, but the end of the current one.

There are still some nuances with ES2015: all our client code goes through Babyl transfiguration, which allows us to use any innovations of the language and not only the ES2015 standard. But Node.js does not have such good support and a number of features are missing in it, and running the server code through Babel is a big headache when supporting and building code. Thus, when writing JavaScript code, we need to always keep in mind the environment where this code will be executed, as well as adjust the linter rules in different ways.

A pleasant cherry on the cake of difficulties can be considered the “complicated” process of selecting modules for use. Of course, there are recognized leaders in one or another area of ​​application, but even among them one can reasonably look for more suitable for each specific case. Yes, the number of logic-reviews has increased, but Node.js and NPM are so good that we have a wide choice of open modules and the opportunity not to adapt to the technology, but to use the convenient one for us.

Why not share it all

In the process of transition to modern technologies, we, of course, faced difficulties, stuffed bumps, but all of this paid off with invaluable experience and advantages that did not end exclusively in a single language for the client and server.

To accumulate all this from scratch is long and unproductive, so our team decided to launch the Node.js development school and share experiences . Training will begin on September 18, 2017, and you can apply now.

During training, we will go through the following topics:

• Web servers: variations and principles of operation;

  
  

• debugging and logging in Node.js;

  
  

• template engines and server HTML rendering;

  
  

• testing in Node.js: unit tests, integration tests;

  
  

• working with databases in Node.js;

  
  
• streams in Node.js.

In fact, the list is longer, you can see it on the school page . But the salt itself will be in practice - over the course of 13 sessions we will together develop a server application for an electronic wallet for managing bank cards.

Sign up and feel yourself as one of the developers of the whole Yandex.Money service.