Support for HTTP / 2 Server Push technology in Node.js

In July 2017, HTTP / 2 implementation appeared in Node.js 8. Since then, it has gone through several stages of improvement, and now the Node.js Foundation says that they are almost ready to deduce HTTP / 2 support from the level of experimental features. If you want to experience HTTP / 2 in the Node.js environment, the best thing to do is to use Node.js 9 - here you have all the latest bug fixes and improvements.

The material, the translation of which we are publishing today, is devoted to working with HTTP / 2, and, in particular, with Server Push, in Node.js.

The basics

In order to experience HTTP / 2, it is easiest to use the compatibility layer, which is part of the new http2 kernel http2 :

 const http2 = require('http2'); const options = { key: getKeySomehow(), cert: getCertSomehow() }; //   https,     //     const server = http2.createSecureServer(options, (req, res) => { res.end('Hello World!'); }); server.listen(3000); 

The compatibility layer provides the same high-level API (request listener with familiar request and response objects), which can be used by connecting the http module to require('http') to the project module. This makes it easy to convert existing projects to HTTP / 2.

The compatibility layer also provides a convenient way to migrate to HTTP / 2 for framework authors. Thus, the Restify and Fastify libraries already support HTTP / 2 using the HTTP / 2 Node.js compatibility layer.
')
Fastify is a new web framework that focuses on performance and is designed to make it easier for programmers to work with. It has a rich ecosystem of plugins. Recently released version 1.0.0 .

Using HTTP / 2 with fastify pretty simple:

 const Fastify = require('fastify'); //   https,     //     const fastify = Fastify({ http2: true https: {  key: getKeySomehow(),  cert: getCertSomehow() } }); fastify.get('/fastify', async (request, reply) => { return 'Hello World!'; }); server.listen(3000); 

While the ability to run the same application code on top of HTTP / 1.1 and on top of HTTP / 2 is important during the implementation phase of the protocol, the compatibility layer itself does not give access to some of the most powerful HTTP / 2 features. The http2 kernel http2 allows http2 to work with these additional features through the new kernel API ( Http2Stream ), which can be accessed through the stream listener:

 const http2 = require('http2'); const options = { key: getKeySomehow(), cert: getCertSomehow() }; //   https,     //     const server = http2.createSecureServer(options); server.on('stream', (stream, headers) => { // stream -    // headers -  ,    //  respond    // -     (:) stream.respond({ ':status': 200 }); // ,  ,   stream.respondWithFile() //  stream.pushStream() stream.end('Hello World!'); }); server.listen(3000); 

In fastify, you can access Http2Stream through the API request.raw.stream . It looks like this:

 fastify.get('/fastify', async (request, reply) => { request.raw.stream.pushStream({ ':path': '/a/resource' }, function (err, stream) { if (err) {   request.log.warn(err);   return } stream.respond({ ':status': 200 }); stream.end('content'); }); return 'Hello World!'; }); 

HTTP / 2 Server Push - opportunities and difficulties

Compared to HTTP / 1, HTTP / 2 gives, in many cases, a huge performance improvement. Server Push technology is one of the HTTP / 2 features that is relevant to this.

Here is how, simplified, looks like a typical HTTP communication session.


Hacker News session

1. Browser requests an HTML document from server
2. The server processes the request and sends the document to the browser, possibly generating it beforehand.
3. The browser receives the server response and parses the HTML document.
4. The browser identifies the resources needed to render an HTML document, such as style sheets, images, JavaScript files, and so on. The browser then sends requests for these resources.
5. The server responds to each request, sending the browser what it requested.
6. The browser displays the page using HTML document code and associated resources.

All this means that during a typical browser-to-server communication session, in order to output one HTML document, the browser needs to perform several independent requests and wait for answers to them. The first request loads the HTML code, the rest - additional materials, without loading of which the document cannot be correctly output. It would be great if all these additional materials could be sent to the browser along with the original HTML document, which would relieve the browser from having to download them separately. As a matter of fact, the HTTP / 2 Server Push technology is intended for the organization of similar work scenarios.

When using HTTP / 2, the server can automatically, on its own initiative, send additional resources along with the response to the original request. These are the resources that, in the server's opinion, the browser will necessarily request later. When the browser needs these resources, instead of sending additional requests for receiving them, it is enough to use the data that the server sent to it in advance.

For example, suppose that the /index.html file /index.html following content is stored on the server:

 <!DOCTYPE html> <html> <head> <title>Awesome Unicorn!</title> <link rel="stylesheet" type="text/css" href="/static/awesome.css"> </head> <body> This is an awesome Unicorn! <img src="/static/unicorn.png"> </body> </html> 

Having received the corresponding request, the server will respond to it by sending this file. At the same time, the server knows that for the correct output of the /index.html file, the files /static/awesome.css and /static/unicorn.png . As a result, the server, using the Server Push mechanism, will send these files along with the /index.html file.

 for (const asset of ['/static/awesome.css', '/static/unicorn.png']) { // stream -  ServerHttp2Stream. stream.pushStream({':path': asset}, (err, pushStream) => {   if (err) throw err;   pushStream.respondWithFile(asset); }); } 

On the client side, as soon as the browser parses the code of the /index.html file, it will understand that to render this document you need the static/awesome.css and /static/unicorn.png . In addition, the browser will become clear that these files have already been sent to him at the initiative of the server and stored in the browser cache. As a result, he will not have to send two additional requests to the server. Instead, it will simply take from the cache the data that has already been loaded there.

Until now, it all looks very good. However, if you look closely, in the above scenario, you can detect potential difficulties. For a start, the server is not so easy to find out what additional resources can be sent on its initiative in response to the original request of the browser. The logic of making this decision can be brought to the application level, placing the responsibility on the developer. But even the site developer may not be easy to make such decisions. One way to do this is as follows: a developer looks at the HTML code and makes a list of additional resources necessary for the correct display of the page in the browser. However, as the application develops, keeping such a list up to date is laborious and fraught with errors.

Another possible problem lies in the fact that the internal mechanisms of the browser are engaged in caching resources that have recently been loaded. Let's return to the above example. If, for example, the browser downloaded the /index.html file yesterday, it would also /static/unicorn.png file, which usually gets into the browser cache. When the browser loads /index.html again and then tries to load the /static/unicorn.png file, it knows that this file is already in the cache. Therefore, the browser does not fulfill the request to download this file, instead getting it from the cache. In this case, sending the /static/unicorn.png file /static/unicorn.png browser at the initiative of the server will be a waste of network resources. It would be good for the server to have some kind of mechanism that allows it to understand if the browser has already cached a certain resource.

In fact, other nontrivial tasks are associated with Server Push technology. If you're interested, read this document .

Automate the use of HTTP / 2 Server Push

In order to simplify support for the Server Push feature for Node.js developers, Google has published an npm package for its automation: h2-auto-push . This package is designed to solve many difficult problems, among them those we have mentioned above and those mentioned in this document .

The package identifies patterns in requests coming from browsers, and finds out what additional resources are associated with the source resources that browsers are looking for. Later, when requesting the same source resources, additional resources are automatically sent to the browser at the initiative of the server. In addition, the package evaluates the possibility that the browser already has some resources in its cache, and if it turns out that this is the case, it does not send these resources to it.

This package is designed for use in the middleware layer of various web frameworks. In particular, we are talking about tools for maintaining static files. As a result, with the use of this package, the development of auxiliary means for automating the sending of materials to browsers by servers is facilitated. For example, take a look at the fastify-auto-push package. This is a fastify plugin designed to automate sending materials to browsers initiated by servers and using the h2-auto-push package.

This middleware is quite simple to use and from applications:

 const fastify = require('fastify'); const fastifyAutoPush = require('fastify-auto-push'); const fs = require('fs'); const path = require('path'); const {promisify} = require('util'); const fsReadFile = promisify(fs.readFile); const STATIC_DIR = path.join(__dirname, 'static'); const CERTS_DIR = path.join(__dirname, 'certs'); const PORT = 8080; async function createServerOptions() { const readCertFile = (filename) => {   return fsReadFile(path.join(CERTS_DIR, filename)); }; const [key, cert] = await Promise.all(     [readCertFile('server.key'), readCertFile('server.crt')]); return {key, cert}; } async function main() { const {key, cert} = await createServerOptions(); //    HTTP/2  https. const app = fastify({https: {key, cert}, http2: true}); //     AutoPush.       //   . app.register(fastifyAutoPush.staticServe, {root: STATIC_DIR}); await app.listen(PORT); console.log(`Listening on port ${PORT}`); } main().catch((err) => { console.error(err); }); 

Results

According to the results of tests conducted in the Node.js Foundation, it was found that using h2-auto-push improves performance by about 12% compared to using HTTP / 2 without using Server Push technology, and gives a performance increase of about 135% compared to HTTP / 1.

Dear readers! How do you feel about Server Push technology?