Nginx variables with njs: easy, painless and through javascript
njs is a JavaScript interpreter in a lightweight web server, with which you can create new nginx variables and request stage handlers. What is njs good for? What can not? And why did they make it at all? Dmitry Volyntsev ( xeioex ), the developer of nginx and the main developer of the njs interpreter, will answer these and other questions.
- The first reason is the
')
The second reason is what nginx has come to now. Previously, it was used for caching statics and queries, as well as load balancing - a classic proxy set. The distribution of microservices blurred the scope of nginx. If earlier configuration configuration ended on a pair of location on several backends in some languages, then with microservice architecture we have more moving parts. The backend has become a bunch of small components. The authorization logic, for example, needs to be duplicated on each microservice or put it, say, to the frontend. To implement advanced authorization, the built-in solution mechanisms in nginx are not always sufficient.
Thirdly, in nginx many directives accept dynamically calculated expressions, for example:
You can concatenate variables with each other or with literal strings. But this is not enough, and I would like to have more powerful tools, for example, to calculate the hash, to work with numeric data, to reduce to upper and lower case.
To expand all the bottlenecks in nginx, you need to either develop your own syntax or use something ready. We concluded that it is best to take an existing scripting programming language. Thus, developers do not need to learn a new language, which will also save time and lower the entry threshold. We chose javascript.
- We chose JavaScript for several reasons:
- There is already a ready third-party project OpenResty. If you do not go into details, it is, in fact, nginx + Lua, but it has an architecture that goes against nginx. We wanted to avoid intersections with this ecosystem. In addition, there are several reasons:
- We appreciated the njs in comparison with the well-known engines - V8 and SpiderMonkey. They are ineffective for tasks inside nginx, because they are sharpened for browsers and are very heavy, and nginx requires high speed. In addition, both of these engines are rapidly evolving, their API is unstable. Finally, njs can be more effectively built into nginx:
Number of contexts created per second
- At the moment, almost all the main elements of the ECMAScript 5.1 specification have been implemented with some interspersing of the elements of specifications 6 and 7. That is, standard objects such as Object, Array, String, Number, Date, RegExp, JSON. Fully supported closures, anonymous functions, work with exceptions.
We do not set as our first priority the full compliance of the language specification. So at the moment there is no support for eval () , and so far we are not planning to add it. But we plan to add support for keywords const and let, as well as switch functions.
What can and what can not njs at the moment
It is important to mention one more thing: lack of garbage collection. Most modern languages independently monitor the lifetime of objects. If an object is no longer used, it is automatically deleted. This mechanism cannot be dispensed with, but usually it is necessary to sacrifice something for it - the work of the program slows down or stops altogether. In njs, memory is not released until the request object is released.
This approach has its pros and cons. The main disadvantage is that it does not allow to work effectively with long queries. Therefore, in the future we plan to add garbage collection as an option to enable it as needed.
- Before answering this question, I would like to repeat once again that the main task of njs is to expand the possibilities for flexible configuration of nginx and solving tasks on the proxy side.
Now the question itself. What should be considered in advance?
If this topic is extremely relevant for you and you are eager for more details, we recommend watching the video recording of Dmitry Volyntsev's report on HighLoad ++ Siberia 2018, where he opened it from all sides.
We also invite all pros to submit their reports at the November conference HighLoad ++ 2018 , which will be held in Skolkovo on November 8 and 9. If you have a unique and interesting experience and you are ready to share it - register and fill in the form by September 1.
If you are afraid to speak in public - we have a so-called school of speakers , where we help to pump these skills for free.
- Dmitry, why did you need scripting in nginx configurations?
- The first reason is the
if directive. People who saw her for the first time think that you can use it imperatively. In fact, this is not the case - the nginx configuration is declarative. In the example below, you might think that the response will include two headers: X-First and X-Second. But only the second header will be answered, because nginx is so arranged: if we write two if-directives, the last one will be chosen.')
location /only-one-if { set $true 1; if ($true) { add_header X-First 1; } if ($true) { add_header X-Second 2; } The second reason is what nginx has come to now. Previously, it was used for caching statics and queries, as well as load balancing - a classic proxy set. The distribution of microservices blurred the scope of nginx. If earlier configuration configuration ended on a pair of location on several backends in some languages, then with microservice architecture we have more moving parts. The backend has become a bunch of small components. The authorization logic, for example, needs to be duplicated on each microservice or put it, say, to the frontend. To implement advanced authorization, the built-in solution mechanisms in nginx are not always sufficient.
Thirdly, in nginx many directives accept dynamically calculated expressions, for example:
proxy cache bypass $cookie_nocache $arg_nocache; You can concatenate variables with each other or with literal strings. But this is not enough, and I would like to have more powerful tools, for example, to calculate the hash, to work with numeric data, to reduce to upper and lower case.
To expand all the bottlenecks in nginx, you need to either develop your own syntax or use something ready. We concluded that it is best to take an existing scripting programming language. Thus, developers do not need to learn a new language, which will also save time and lower the entry threshold. We chose javascript.
- Why JavaScript?
- We chose JavaScript for several reasons:
- A modern dialect that is good for developers migrating from other languages.
- C-like style. This is important because curly brackets are used in the nginx config, and in the future we want to add the ability to write JS code right inside the config. Braces will help us with this. In Lua, for example, begin and end curly braces are inconvenient.
- The JavaScript model fits well with the nginx architecture.
“So Lua was also considered?” Is it because of begin and end?
- There is already a ready third-party project OpenResty. If you do not go into details, it is, in fact, nginx + Lua, but it has an architecture that goes against nginx. We wanted to avoid intersections with this ecosystem. In addition, there are several reasons:
- Lua has a pascal-like syntax.
- Arrays are indexed from 1.
- Lua is still a niche programming language.
- How does njs work compared to competitors?
- We appreciated the njs in comparison with the well-known engines - V8 and SpiderMonkey. They are ineffective for tasks inside nginx, because they are sharpened for browsers and are very heavy, and nginx requires high speed. In addition, both of these engines are rapidly evolving, their API is unstable. Finally, njs can be more effectively built into nginx:
Number of contexts created per second
- What standards does njs support?
- At the moment, almost all the main elements of the ECMAScript 5.1 specification have been implemented with some interspersing of the elements of specifications 6 and 7. That is, standard objects such as Object, Array, String, Number, Date, RegExp, JSON. Fully supported closures, anonymous functions, work with exceptions.
We do not set as our first priority the full compliance of the language specification. So at the moment there is no support for eval () , and so far we are not planning to add it. But we plan to add support for keywords const and let, as well as switch functions.
What can and what can not njs at the moment
It is important to mention one more thing: lack of garbage collection. Most modern languages independently monitor the lifetime of objects. If an object is no longer used, it is automatically deleted. This mechanism cannot be dispensed with, but usually it is necessary to sacrifice something for it - the work of the program slows down or stops altogether. In njs, memory is not released until the request object is released.
This approach has its pros and cons. The main disadvantage is that it does not allow to work effectively with long queries. Therefore, in the future we plan to add garbage collection as an option to enable it as needed.
- What is njs not?
- Before answering this question, I would like to repeat once again that the main task of njs is to expand the possibilities for flexible configuration of nginx and solving tasks on the proxy side.
Now the question itself. What should be considered in advance?
- njs is not a replacement for Node.js.
- The nginx + njs bundle is not an application server.
- njs does not fully implement ECMAScript standards, since there is no support for eval ().
If this topic is extremely relevant for you and you are eager for more details, we recommend watching the video recording of Dmitry Volyntsev's report on HighLoad ++ Siberia 2018, where he opened it from all sides.
We also invite all pros to submit their reports at the November conference HighLoad ++ 2018 , which will be held in Skolkovo on November 8 and 9. If you have a unique and interesting experience and you are ready to share it - register and fill in the form by September 1.
If you are afraid to speak in public - we have a so-called school of speakers , where we help to pump these skills for free.
Source: https://habr.com/ru/post/420465/
All Articles