Iterator in doT.js template engine for objects with filtering

The compact (3.5 Kb) and fast doT.js template for browsers and nodeJS so far (v.1.0.1) has iteration over arrays only. It is not always convenient to adjust the control object to the presence of arrays in it. It is better to adjust the template engine to the presence in it of an iterator on objects with condition check . It is often necessary to check conditions in loops on objects — this includes hasOwnProperty (), and checking for a DOM object, and taking a part of the hash by filtering indexes.

How do you write an array iteration in a template? Like that:

{{~it.elemsArray:property:i}} ... {{~}} 

Inside the loop, you can use {{= property}} (the value of the current property), {{= i}} (array index) and expressions based on them. Nothing prevents you from adding a check in the template engine not only to the array, but also to the object in order to correctly select the further construction of the loop. To do this, you need to get inside a rather simple template engine and rewrite the loop, already working on arrays, into a loop that will work on objects. Leaving, of course, the original possibility in the general code.

Add another iterator type to the template engine. (The number of teams in the arsenal will increase by one - {{@ ...}}, so the template engine will begin to work a little slower - while you search for another regexp using a template.)
')

 {{@ it.elemsObject : property : i : condition}} ... {{@}} <!--    --> 

" condition " here is just an executable part of an expression. The peculiarity of this template engine, like many others, is that it synthesizes the JS code (as they say - compiles the template), turning it into an executable function, and immediately executes it. We use this to simply attach part of the expression to the property value, which is calculated in a loop. As you know, the loop for objects in JS often looks like this:

 if(it.elemsObject) for(var i in it.elemsObject){ var property = it.elemsObject[i]; if(property condition){ ...   ... }}; 

This is exactly what the iterator on the object in our appendix will do. The syntacically incomprehensible “property condition” is our ploy that fully exploits the already existing drawback of this template engine — the implicit execution of eval () in new Function (). In order not to violate the syntactic integrity of the code, it will be necessary to comply with the requirement that condition be an expression that does not change its meaning when concatenated with property .

Here are the 3 most popular examples when it is needed and how it will look.

1. Cycle by own properties
{{@ it.elemsObject: propName: ii: .hasOwnProperty (ii)}} ... {{@}}

2. Loop through DOM elements

 {{@ it.elemsObject : elName : ii :.attributes}} ... {{@}} 

Will give a cycle:

 var arr1 = it.elemsObject; if(arr1) for(var ii in arr1){ var elName = arr1[ii]; if(elName.attributes){ out += '...' }} 

3. Cycle of the main object

 {{@ it.elemsObject : elName : ii :, /y\d+/.test(ii) }} ... {{@}} 

For iterations, selects only those objects whose indices look like “y <number>”.

The last example greatly expands the flexibility of using templates — you can place one or more collections of items in the root of a shared object that differ in the format of the index or the properties of their values. (For example, values ​​can be an object with a property — a collection attribute. It also demonstrates how easy it is to drop the default propName element in a template — simply by putting a comma, turning the expression into an enumeration of expressions.

In the existing template engine, such cycles, in fact, could be implemented by using the “evaluate” construction: {{<arbitrary operators>}} . Then I would have to completely describe my cycle in the template. But instead, we create a small addition to the loop construction of arrays, in order to have a brief notation of the conditional expression. It would be a little wise (for large systems) to wrap an expression in a function (so as not to have a composite syntax), which, by the way, would slow it down and require explicitly writing the name property in the condition.

For reference, the loop that makes the existing template array array looks like this:

 var arr1 = it.elemsObject; if(arr1){ var vname, ii =-1, l1 = arr1.length -1; while(ii < l1){ vname = arr1[++ii]; out += '...' }} 

Therefore, he could not work with objects - he uses a counter. Which is probably a bit faster than the universal for-in loop, but the loss of time does not go on cycles. Time is wasted interpreting the synthesized pattern. We will not improve it in this extension in order to fully preserve the efficiency and behavior of existing templates.

There is in our new type of cycle, c condition at the end, a small but natural limitation - it cannot use 2 "}" characters in a row. In all cases, the code is easy to get around. It is permissible to write multi-line expressions with definitions of functions, for example, not forgetting only that "}}" is the end of the iterator header. Example:

 {{@ it :year:i:, aa = function(){ return 1234; }, /^y\d+$/.test(i + aa) }} 

With such features, the template easily turns into a description of functions with small HTML inserts (and not vice versa). This is not necessarily convenient, but we have removed the restrictions on JS in the iterator.

Related improvements

The propName: ii values ​​can be omitted - the parsing in the template engine is corrected so that in the absence of values ​​the default names are substituted. The default property name is arrI1, arrI2, and further, depending on the sequence number of the iterator header. The default object index is i1, i2 or later. Such strange names were chosen for historical reasons — the same exist in doT.js for array indices, and the implicit array names looked like arr1, arr2, and so on. We simply keep the tradition: if the name of the hash is arr1, then the name of its property is arrI1 (equal to arr1 [i1]). For example,

 {{@ it.elemsObject :::.attributes}} 

tantamount to

 {{@ it.elemsObject : arrI1 : i1 :.attributes}} 

(if this is the first iterator header in the template)
or

 {{@ it.elemsObject : arrI2 : i2 :.attributes}} 

(if this is the second iterator header in the template, etc.)

If you think about the quality of reading the template, then the default name would be better to choose as the last in the compound name of the object. In this example, elemsObject. It would be like this: {{@ it.elemsObject}} is {{@ it.elemsObject: elemsObjectI1: i1}}. But for the sake of historical conformity, we will not introduce such a rule for the iterator.

* The second important addition. Not to delete elements with the value 0 || '' || false || null - the loop filter becomes true (the filter is not valid) if the expression in the iterator header is absent.

And the third - multiline partner in retexp for the first parameter (the name of the array or object) is replaced with a single line (". +"). This leads to a slight acceleration of parsing (3% by measurement).

4. Added the ability to write inline single-level (multi-line) hashes, as can be seen in the example on jsfiddle . It is possible to write inline arrays (in an iterator over objects), as seen in the same example.

Speed ​​comparison

How much will we pay for the execution time for including in the template engine the possibility of looping over objects?

The answer to the question will be given by the test, which lies in the doT.js project on Github. It will need to be rewritten, because it measures the speeds of the compiled templates, and the difference in compilation speeds is primarily of interest. For simplicity, we will consider tests in Chrome (version 30). On other browsers, they are easy to repeat and draw similar conclusions and comparisons.

It was originally made for comparing strongly shortened doU.js and full doT.js, and shows how much slower the executable part of the full function doT.compile () is compared to the much shorter one. It also proves that using “it” instead of this makes almost no difference on long patterns, so the tests are replete with lines of measurements of various simple variants. The template, the speed of which is measured, looks like this:

 <h1>Just static text</h1>\ <p>Here is a simple {{=it.f1}} </p>\ <div>test {{=it.f2}}\ <div>{{=it.f3}}</div>\ <div>{{!it.f4}}</div>\ </div> 

Or its modifications, or repetition many times (32-128-256). We are interested in these tests to show that an extra check on the iterator header did not slow down the work of simple templates.

For our purposes, we are also interested in cycle tests. In order for their speed to be approximately equal to the speed of simple templates, the following test template was written (it is without line breaks):

 <h1>Text from hash</h1><div>\ {{@it::i}} <div>{{=i}}: {{=it[i]}} : </div> {{@}}\ {{@it:val:i}} <div>{{=i}}: {{=val}} : </div> {{@}}\ {{@it:a}} <div>{{=a}}: {{=a}} : </div> {{@}}\ </div> 

For testing in the old doT.js template engine of version 1.0.1, the characters "@" were replaced with "~". The same - for testing by arrays - in the new. And the data was used for one case - hashes, for another - arrays with the same number of elements. According to the results at the bottom of the diagram we will see which of them is faster.

It will be of interest both uncompressed (doT11.js) and compressed (doT11m.js) new version. To pass the tests, the doT variable in each version had to be made unique. The pictures show the test speed diagrams (the number of calculations per second): more is faster, and therefore better. The first picture is the result (attention) of the compiled templates. Therefore, they are so fast - 500-700 K per second, and the difference between the minified version and the uncompressed version does not have the meaning that we want to see. On the other hand, this test shows how much slower the full cycle (lines “5. doT.js”, 6, 7) compared with the compilation results (the higher the indicator, the better).



The results show that the template compilation time takes a huge share of the total time - from 99% for short templates to 90% for long ones (more than 10KB), and for a reloaded browser - 92% and 75% under the same conditions. This is a fee for implicit eval (), which must be performed during de-template.

By numbers it can be seen that the minified code is 1-1.5% faster. And the execution of compiled templates, in theory, should not depend on minification at all. Dependence - or the manifestation of random errors, or compilation errors of functions, or errors of the measuring tool.




Separate attention deserves a comparison of 2 types of cycles. As it was supposed, the array loop works faster on almost the same templates, but with slightly different data. The difference is 20% for short templates, 25% for templates of medium length (2-5 KB with the number of cycles up to 300 in the template). With 100 or more cycles, an interesting effect appears in Chrome, apparently related to the number of variables in the compiled function. The performance of the cycle on objects remains at the usual level, and the cycles on arrays suddenly begin to sink sharply with an increase in the number of cycles (and variables) in the pattern. For example, at 190 cycles, the pattern test becomes 10% slower, and at 750 cycles, it almost stops at all - 6% of the pattern speed for objects.

The following figure illustrates the beginning of the collapse of loops over arrays on long patterns with the number of loops 386 (the higher the indicator, the better).


An interesting question is whether there is a collapse threshold for a loop on objects? Yes. Increasing the volume of the test pattern by a factor of 2 (up to 512 copies of the short pattern), we get the browser to hang on iterating through the array and every 10 times lower speed of the cycles on the objects. (Perhaps, in this mode, something is not optimized and some stack overflows. Or so the test script used in this project, which, besides standardization, does thousands more and hundreds of thousands of tests, affects it, and it could also Something "to flow. In the meantime, we conclude that the limit on the number of cycles in a template and in JS functions in general is. It is better not to do more than 1000 cycles on objects in one template (for Chrome).)

The effect of a general slowdown of the template engine operation with the open browser operation time and measurements in it has also been noticed. At the same time, the speed of work on arrays starts to look not so much higher than the speed of objects.

Pattern Interpretation Rates

The initial interest in the question of speeds was in the speed of interpreting an iterator on objects. The fact that the compiled template for the array is faster - have already figured out. But developers are often interested in not compiling templates, but in their interpretation, even if it is 10-100 times slower than the execution stage. After all, if you need to render a page in 30 ms, you usually don’t think about compiling for subsequent rendering in 3 ms. We will do tests to interpret our templates using the same function and data stubs. This is where the minification template template will play its role - in the first series of measurements it did not matter.

In the existing tests there is also a separate page for a separate compilation (without execution). We transform the tests on it into complete ones by adding the pattern drawing execution. This, in practice, will not change the picture, but will give realistic tests, more similar to practical tasks.

Run the doT / benchmarks / compileBench.html file .

This group of tests showed everything that interested us:
1) the minified code shows stable acceleration for single short templates - by 5%;
2) for the same, but uncompressed functions, short patterns demonstrate a slowdown compared with the old code in tests with measurement cycles, by 4%;
3) for long patterns, the new code is always steadily faster by 3%;



Intuitively, it seemed that the comparison would show a slowdown of 10%, especially where it is necessary to parse the arrays - the work on regexp passes along the pattern increases by about so much. Changes show compliance with these assumptions, but not so significant. Firstly, it is compensated for by improving the array pattern without losing properties (". *"), Secondly, the pattern has become slower on short patterns by 4% (for all, not just arrays), but the difference disappears for long ones - in this and is the expected slowdown.

A separate group are measurements on other templates, with cycles (the last 3 measurements). The third is the pattern by arrays in the old code. It had to be slightly corrected so that there were no implicit variables ("~ it: val: i", and not "@it :: i"). Perhaps that is why it is a little slower, 10% in short patterns. Other features are visible:

4) Interpreter on objects slower than interpreter on arrays by 3-5%;
5) the performance of both of them was faster relative to other dimensions, and interpretation was 2-3 times slower; the longer the pattern, the slower;
6) the old code by arrays is not faster than the new code by objects - faster by the indicated 3-5% only the new code by arrays. (This effect is obtained by a small rewrite of the compilation code - using "++" instead of "+ = 1".)
7) the speed of work of templates and speed of work on arrays noticeably (at times) depends on the browser reload. Arrays are faster than cycles for objects by 20% at the beginning and by 5% after some time of the test page. Perhaps this is not only the effect of the function, but also the influence of the testing environment (tested in Chrome 32bit, Win).

We received answers to the question whether the code had become worse. In general, it does not require further improvements to make its work look like the work of the old code. Replacing the old with the new in existing projects is possible. Even if the question includes the speed of compilation or execution.

Once again, it doesn’t bother to remind : interpretation (see tests) is much slower than the execution of compiled templates. If speed is important in the project and there is more than one template execution, use a separate compilation (doT.compile ()).

Hackney template further?

If the described solution is well suited for cautious extension of functions as nothing that changes for the old code, only adding a new iterator, then for new projects you can use the option in which the loop over the while arrays is replaced with the universal for-in loop, and the loop over the array filter expression added. Two types of iterator are combined into one, and the speed of parsing practically does not suffer, as well as the speed of the cycle. There may be side effects, so a replacement with the second option would be a good idea to test in an existing project to determine if there would be any abnormal situations with erroneous templates. The cycles change, which means that the side effects of errors change.

For a for-in loop over an array for some older browsers, you need to ensure that the iterator only works on numeric indexes and does not capture the length property in the iteration. The requirement is perfectly fulfilled by the existing loop filtering mechanism, so it only remains to write a valid check if the object turned out to be an array. Such a check will not be superfluous for collections of type arguments or attributes. And certainly not superfluous - for objects that inherit the properties of the array. Array-based objects will by default be iterated only by numeric indices.

However, this is a slightly different task. In this article we will limit ourselves to the cautious extension made, which is tested and works on one project.

results

The addition of the doT.js template maker to the possibility of iteration over objects, made in order to obtain an extension that does not violate the other properties of the existing version, is investigated.
Got:
1) an iteration in the template by objects ({{@ ...}}) with the ability to filter a part of the elements by any condition;
2) the possibility not to specify unused parameters and colons at the end of the iterator header;
3) for the array iterator, a similar opportunity has been added not to specify parameters in the array iterator header;
4) there is no loss of speed in the work of the template engine compared to the old version (there is a slight loss on the short templates);
5) — 250 , ;
6) — 3% ( + ).

. — «global» «doT», . 1.0.1 , , (doT ).

GitHub ; jsFiddle .