Server javascript, 1ms per transformation

What for?

The question “Why?” Is the most important when making any decision. In our case, there were several reasons.

First, the people. The current templating engine was processed by C. All questions about his changes were not resolved quickly. And the most important thing is that some people wrote the templating engine, but used completely different ones.
')
In general, this is a frequent and, in my opinion, not very good practice of writing tools for web designers. It is clear that they need tools, but these tools are implemented by people who very remotely imagine the daily tasks of layout designers. Rather, on the contrary, the decisions of the plan are often made “we will let them write the conditions and cycles, but nothing more can be needed in the layout”. Perhaps it is the blame of the designers and their qualifications.

But Mail.Ru Group has a whole team of highly qualified people who know JS, who are able to write a tool on their own, and most importantly, they will use it.

Secondly, the task. Take the project Mail.ru. We can’t refuse to use templating on the server - we need a fast download on the first login We can’t refuse the standardization on the client - people should see a high reaction rate to their actions, which means that AJAX and the standardization on the client are required.

The problem is obvious: two sets of completely different templates on the server and on the client. And the most annoying that they solve the same problem. Duplication of logic simply exhausted us.

v8 is a JavaScript interpreter, which means we can get one template that works both on the server and on the client.

Third, speed. After reading a lot of articles, in which they praise the speed of v8, they decided to check their fairness. But first it was necessary to understand how we want to see the new template engine.

What do you need

I will say right away that we are very limited in terms of server time for transformation, so there was no way to implement something very functional. However, leaving the old functionality with the only difference in adding a layer from v8 is a strange idea.

I have been using a lot of XSLT for transformation for a long time (not the fastest transformation option), although, if used correctly, it shows good numbers (I'm talking about libxslt). But XSLT has a very powerful templating tool - redefining templates. We decided to implement something similar, but much easier.

/head.xml … <title><fest:get name=”title”/></title> <fest:set name=”title”>Mail.ru</fest:set> /mail.xml … <fest:include src=”head.xml”/> <fest:set name=”title”> Mail.ru</fest:set> 

It would be strange to use v8 and not to allow access to JavaScript in templates.

 <fest:script> var text = “mail.ru” </fest:script> <fest:value>text</fest:value> 

And a lot of small things, in addition to standard conditions and cycles.

XML

We took XML as the syntax for the template engine.

Support for basic functionality in the IDE. One hundred percent of popular IDEs know what XML is. Hyphenation, highlighting, basic autocompletion you get for free.

Validation at IDE level. Valid HTML is obtained through valid templates. Again, all IDEs can validate xml as such.

Modularity out of the box (name spaces). The basic capabilities of the template engine very quickly want to expand. For example, add a tag that allows you to do projects in multiple languages. The Name Spaces system makes it easy to do.

A wide range of ready-made tools. Over the years, many tools have been accumulated for processing XML, for example, libraries for processing XML via XSL or a whole class of SAX parsers, XSD and DTD schemes for validation, etc.

Matching syntax processing structures and resulting structures. In other words, creating XML in XML is convenient, fest-templates are easy to read. Plus resolved all data screening issues.

Implementation

Cowboy I myself recently learned that there is such a term.

Before that, I was sure that the most reliable way to accomplish a task was pair programming and tests. Tests, as always, justified themselves, but there were alternatives to pair programming.

The difference from the typical task: the template was known and the resulting HTML, which this template should produce. Kostya and I (frontend mail developer) started writing our implementations. Once a week we compared the transformation speed measurements.

We chose two different approaches: he compiled the template into a function, and I into the structure. An example of the simplest structure:

 01 [ 02 {action:"template"}, 03 "<html>...", 04 {action:"value"}, 05 "json.value" 06 ] 

The second line marks the beginning of the pattern. The third is just to give the browser. The fourth says that the fifth should be executed as JavaScript and the result of the execution should be given to the browser.

 01 [ 02 {action:"template"}, 03 "<html>....”, 04 {action:"if"}, 05 "json.value", 06 "<span>true</span>", 07 "<span>false</span>" 08 ] 

The option is a bit more complicated. The fourth line means that the fifth must be executed, and if the result is true or false, then give the sixth or seventh line, respectively.

The option with the function does not need to be specifically explained.

 01 function template(json){ 02 var html = ""; 03 html += "<html>…"; 04 html += json.value; 05 return html; 07 } 

Looking ahead, I will say that his version was faster. For some time we walked almost evenly, but the variant with the structure rested on the ceiling much earlier.

To understand what gave such an approach: my first implementation carried out the task in 200ms. When we squeezed everything we could and then put together the best of our two programs, we got 3ms.

If we describe the current implementation briefly, we translate the cycles into cycles, conditional operators into conditional operators, etc.

 fest:foreach for(i = 0; i < l; i++) {} fest:if if(value) {} fest:choose if(value) {} else {} 

No context narrowing. Yes, this is a restriction, but there is no overhead for restricting the context, and most importantly, as soon as you narrow the context, the problem immediately arises to get something from the global context or from the context a higher level.

It is important that the templates are translated into the JS-function, which works in strict mode. This does not give designers a chance to write code that will lead to memory leaks.

Wherever a logical work with data is needed, JavaScript is available.

 <fest:if test=”_javascript_”></fest:if> <fest:value>_javascript_</fest:value> 

All constructions that assume the execution of JavaScript are wrapped in a try catch.

All constructions that assume output in HTML after the execution of JavaScript, by default, pass HTML escape.

 <fest:value>json.name</fest:value> 

 try { html += __escape(json.name) } catch(e) {} 

From the very beginning, the development of a template engine is carried out in the open form.
https://github.com/mailru/fest

Integration capabilities

On the one hand, v8 is only a library that allows you to interpret JavaScript. By itself, it seems useless - no access to the system. But, on the other hand, it is easily screwed to other languages.

Having zero programming experience in C and Perl, I made test examples in both languages. Plus, at the moment we have a bunch of Python.

And, of course, NodeJS for prototypes and browsers are environments where JavaScript templates work out of the box.

Conditions close to combat

After receiving 3ms, I went to the server-side programmers. When they asked how much time I had for a request that gave the list of letters to the user, they said: no more than 4ms. I already had 3ms to transform, I had to try.

The list of letters we give our own http-server written in C. Data acquisition - operations that do not compete for the processor, so they were not measured. We stopped on the preparation of data for transformation and on the transformation itself.

For historical reasons, our http server stores data in a flat hash.

 msg_length = 5 msg_1_title = “letter” msg_1_Unread = 1 

As we are talking about javascript, the first thing that comes to mind is JSON

 msg = [ {title: “letter”, Unread: true} ] 

We took a string with a flat hash, put it in memory and began to achieve a result when, when transforming a template into v8, JavaScript operated on JSON.

Sifted through a lot of options. To forward an object, forward a string and parse it in JavaScript, forward the string and pass it through JSON.parse.

Oddly enough, the fastest is to convert a flat hash into a string that matches JSON, and in v8 to give the string

 “template([ {title: \“letter\”, Unread: true} ])” 

But, despite everything, we rested on 6ms with the transformation of 2ms. Everyone was ready to give up. I nevertheless decided to take the source data, a string with a flat hash and, using the same compiled template, get the desired HTML on NodeJS.

Got 4ms. When I came to our sishnik with this number, to be honest, I expected the phrase “It's great, but we have no resources to write NodeJS” But instead I heard “If NodeJS can in 4ms, then we can also!”.



It was at that moment that I realized that we would bring it to production. There was a second wind!

The solution was simple. Since we lose 67% of the time on data preparation, and in principle we already have data, we need to throw out data preparation.

We threw the function __get ('key') into v8. Thus, we took data from v8 directly from the hash of our http server. There is no data conversion in the desired format. There is no conversion of this string to an object within v8. We went out at 3ms and had a stock of 1ms.

Almost production

So, everything looks great, but we have not been close to the production yet. Itching to try.

We take a separate server, pick up on it the http version of the server that works with v8, and duplicate the real requests for it. We leave for 30 hours one 2.2 GHz Xeon core.

 10,000,000+ hits
 1.6ms average transformation time

 992 422 10% between 2 and 5ms
 208 464 2% between 5 and 10ms
 39,649 0.4% more than 10ms

Only 12% were over 2ms. v8 behaves consistently from memory.

Production

I came with the latest figures to the deputy technical director, saying that v8 is ready for production, you need to do a separate small project, which, if anything, you can forget in case of failure. In response, he received “the numbers are good, a separate project, the failure of which is not terrible - this is correct, but do you really want to run v8? Start with the main Mail.Ru page. ” The question is posed correctly - either we are doing business or having fun in the sidelines.

The layout of the main page at the fest took three days. Turned off one server from the balancer, filled in the version with v8 and duplicated requests. All calculations will occur in the context of a single daemon / kernel.

Then I will tell a very instructive story with a good ending.

Always find out what your graphics show. We allowed half of the load on the test server. Processor consumption was three times higher than usual. It looked like a failure, losing resources by six times compared with the current template engine.

Began to watch. Here I will talk a little about the main architecture. It collects information from different projects. Gathers its internal development, we call it RB. Of the 165kb, which are generated for the return of the main, 100kb collects RB. And the following happens: RB gives HTML chunks via http server in v8, v8 concatenates them with its own strings, and the result returns it all back to the http-server.

There is a double probros data. Did the optimization. Now, v8 instead of building one big line, which includes data from the RB, sends the data immediately to the http-server.

 __push_string('foo'); __push_rb(id); __push_string('bar'); 

As a plus, there is no string concatenation on v8, there is no RB double forwarding from server to v8 and back, and most importantly, any data forwarding is the conversion from utf-8 to utf-16 and back. V8 stores everything in utf-16.

There was a profit, resources were consumed twice as much as usual, and not three. Those. we still lost four times, although everything seemed to be squeezed to the last drop.

And now the instructive part. I took the interest for the sake of the load that we tested, multiplied by two, by the number of demons on the machine and by the number of machines. Got 440 million hits. In this case, we have 110 000 000 hits per day. Vague doubts crept in.

Let's go watch logies. It turned out that for each request with a load, we received three requests with log reports for statistics! The actual load on one http-server is four times lower than the one on which we are testing!

The next morning, we rolled out a version of the main page with v8.

Data for today:
The size of the HTML generated, which generates v8 65kb.
Time, work v8 on request 1ms.
On average, v8 requires 40MB per context.

A pair of refinements

Everyone who thinks about v8, stumbled upon an article by Igor Sysoev sysoev.ru/prog/v8.html

For all the time we've been working on this task, V8 Vyacheslav Egorov (http://mrale.ph) has helped us a lot.

Fears about memory are justified. If correct work is critical for you (provided that the lack of memory is a normal situation), then you will have problems. It is possible to intercept the allocation error correctly, but all that can be done about this is to correctly restart.

But to be honest, we have only one product where this is critical. As for the main page, we have multiple memory reserves and we monitor it very well.

It turned out that our v8 trunk is flowing. Vyacheslav did not succeed in reproducing this, but I think we will put together a test case that will help developers find a leak. Version 3.6 behaves perfectly from memory.

useful links

- github.com/mailru/fest template engine
- code.google.com/p/v8 v8 API
- sysoev.ru/prog/v8.html article by Igor Sysoev


Andrei Sumin, Head of Client Development, Mail.ru