Not so long ago I had the opportunity to speak at the FFConf conference with the report “You should use <write the necessary library / framework>, this is the best!”. And I decided to set out the main ideas in a publication, in the hope that this will provoke a wider discussion in the professional environment about the “cost” of modern frameworks on mobile devices.

For those who want to refer to the original of my speech here is the video:


')
and presentation slides: speakerdeck.com/paullewis/framework- here-its-the-bestestest .

Framework Benefits

At the beginning of the year I wrote about the dependence of the performance of the React framework on increasing the size of the tree with which it operates (TL; DR: the larger it is, the more computing power is used). I received a lot of feedback on this publication, varying in a very wide range - from appreciative and constructive to bright negative. Thanks to this feedback, I collected additional information, which to a greater or lesser extent can be attributed to all frameworks:

• The frameworks are pleasant and convenient to use. I do not know of a single developer who would have looked for ways more difficult and boring. Yes, ergonomics and comfort are definitely important.
• The frameworks allow you to create MVP faster. This is especially important for small, inexperienced companies and organizations. Quick start can play a decisive role in the success of the product.
• The frameworks help to bypass features and platform bugs. Today it is not so important, because in recent years browsers have become much more consistent with the standards. But remember what a pain it was in the days of IE5 and 6, as well as earlier versions of Firefox, Safari and Chrome.
• If I know [insert the name of the framework], then I will pay for it. Your ownership of new and better frameworks can play a big role in your job search.

And the most important factor was the ease of use - most developers explicitly or implicitly noted it. This collective opinion can be formulated as follows: "If it will be easier for me to live, I can do something better for users." It seems to sound good. But, as it seems to me, at such a position many important points are overlooked.

Developer comfort and user needs

We must not forget that users have their own needs. In particular, such:

• Sites and applications should load quickly.
• Their interfaces should work smoothly.
• They should not be too "heavy" for the phone.
• They should not "break".
• They must be a necessary set of functions.

There is a problem: on the one hand - the convenience and comfort of developers, on the other - user needs. And since frameworks are not free in terms of resource use, all this needs to be somehow linked to each other, whether we like it or not.

Small retreat

Here I propose to exclude individual libraries from consideration. After all, if there are any problems with them, then they can be replaced by others. Don't like how the library sets the date format? Just plug in some more. But changing the framework is not so easy. Often this requires rebuilding the entire application. In addition, frameworks occupy much more space and are involved in applications much wider and deeper.

"Cost" code

Each development has its own "price". But I believe that each framework makes a unique contribution to the total “cost”.



That magic moment, when the framework informs you about something not recommended (unacceptable). Probably, the message gave Java. Strange.

• Study of. You have to spend time studying the framework: how to write “idiomatic” code under it, which templates, practices and methods of performing various operations are used here. No difference compared to the web platform itself. There, too, you can not just take and start to use, you have to solve one problem after another.
• To study again. One day during development, you’ll see in the console a warning about something unacceptable, some obsolete operation and a suggestion to update the code. In such cases, it is usually necessary to calculate what the new version of the framework did not like in your actions. And this may entail updating projects released with older versions.
• Debugging As with any software, frameworks contain bugs. You need to be ready for this, even if you write your own version. And the complexity of debugging the framework usually falls in the range from “difficult” to “impossible”. Having discovered some problem, you can try to patch the patch yourself, until the bug is officially fixed in the next version of the framework.

Users also "pay" their price:

• Time. How long will it take to download what has been forwarded by wire?
• Bandwidth. What channel width is needed for work?
• CPU (battery) resources. How intensive is the processor? Do computing power consumption and battery charge correlate with each other?
• Frame frequency. How often is the screen updated (because users like the smoothness of the interface )?
• Memory. How much memory does your product consume? Do you have to unload other applications from memory? Maybe even at the cost of their fall?

Data on the table

Given all this, I propose to discuss the measurement of some of these values. Perhaps by joint efforts we will manage to figure out how best to reach a compromise on the chosen points. I will start from the time of the initial loading of frameworks on smartphones. For testing, I chose TodoMVC. For me, this is the MVP web application, and from the user's point of view, all the options for functionality are identical.

Initial load

I measured the following parameters: time, bandwidth and CPU utilization. Tests were conducted on the Nexus 5 and iPhone 5S.



I decided to measure how long each framework takes:

• to evaluate and execute a useful javascript load,
• processing and placement in the model of the primary data set.

I ignored the cost of applying styles, layouts, fills and other things, since it should not change much in different implementations of TodoMVC. I also did not measure the duration of the data transfer.

Methodology

To load pages on Nexus 5, I applied WebPagetest. For each run, a timeline file was created, which was then processed in Big Rig . I took the results from the iPhone and processed them myself, since profiling JavaScript in Safari does not seem to support the export of traces and timelines.



Using Big Rig

By the way, that's why I created Big Rig - I wanted to make it easier for all of us to process this kind of data.

Reproduction test

If you want to test it yourself, for example, on a device running Chrome, then follow this algorithm:

• Install the Big Rig CLI: npm install -g bigrig.
• Go to the WebPagetest .
• For your location, select Dulles.
• For the browser, select Nexus 5 - Chrome (or Beta / Dev).
• On the tab with the settings of Chrome check the box “Capture Dev Tools Timeline”.
• Run the test.
• Download the timeline file to the left of the results.
• Run bigrig --file /path/to/timeline.json --pretty-print.

Here is an example of how I do it:

results

I received the following data:

Framework	The size	Nexus 5 1, 3 boot time	Boot time for iPhone 5S 2, 3
Polymer v1.1.4	41 Kb 5	409 ms	233 ms
Polymer v1.2.2	47 Kb 5	155 ms	232 ms
AngularJS v1.4.3	324 Kb	518 ms	307 ms
React v0.13.3 [JSX not converted]	311 Kb	1,201 ms	1,463 ms
React v0.13.3 [JSX converted by Babel] 4	162 Kb	509 ms	282 ms
React v0.13.3 [JSX converted; production build] 4, 6	160 Kb	174 ms	118 ms
Backbone v1.2.2 [including jQuery and Underscore]	139 Kb	248 ms	168 ms
Ember v1.10.0-beta.3	580 Kb	1,992 ms	1,440 ms
Vanilla	16 Kb	50 ms	33 ms

Notes:

1. Tests are performed on Nexus 5 running Chrome 47.
2. Tests are performed on iPhone 5S running Safari 9.
3. Bootstrapping involves processing the data in the primary task list.
4. Cut JSX Transformer. JSX files are converted using Babel.
5. With the exception of Web Components Polyfill (12 Kb).
6. Used minifitsirovannaya production-assembly React.

From my point of view, the conclusions are unambiguous: on mobile devices, the use of frameworks is associated with a serious increase in load, especially in comparison with the writing of regular JavaScript. Polymer 1.2.2 showed the best results, but even it turned out to be three times slower than the “vanilla”. React is comparable in speed to Polymer, but I have questions about its scalability .

To really clarify the situation, I note:

• In the case of React, I converted JSX myself, because TodoMVC does not. Instead, it includes the JSX Transformer library. So that the implementation of this library did not affect the result, I drank it and tested the resulting "custom" version. Unfortunately, in this case I used a non-minified version of React, so I had to test the third option.
• The results presented in the table do not include the duration of the data transfer. I measured only the initial load time of the framework in JavaScript, right up to the final rendering of the interface. Please note that some frameworks in TodoMVC are not minified. If you have the desire, you can explore the discussion about the amount of data transfer on the site of the Filament Group .

Possible objections

Probably, some moments of the test can cause objections, which need to be answered:

• "TodoMVC is not idiomatic . " I checked this statement. As far as I understand, each implementation is idiomatic. And when this is not the case, the framework analyzes the necessary computing resources for each file.
• "TodoMVC does not reflect my script . " Maybe. But I bet that in your case the “cost” is even higher than that of TodoMVC. Paul Irish recently investigated the performance of the mobile version of the Reddit site and found, among other things, that on mobile devices the download of React components takes 22 seconds!
• "Our users do not have Nexus 5 / iPhone 5S . " This is also quite real. I had the opportunity to test on good devices, and I can imagine that many users do not have expensive and powerful smartphones. So in reality, the situation with loading times is even worse.
• "In the next version of <insert the name of the framework> the situation will be better . " I admit. But this is unlikely to help those who already use your products on previous versions of the framework.

$ 64,000 question

And inevitably the question arises: is it necessary to use the framework? I can not give an answer. This should be your own decision. There are a million reasons why you desperately need a framework. Here are my own thoughts on this:

• Frameworks facilitate the implementation of ideas and concepts. It is thanks to them that one can understand which approaches work and which do not. This stimulates the process of improving software products at the platform level. From this point of view, frameworks seem to me to be some kind of test sites where you can experience future innovations within the framework of platforms. With their help, you can understand what opportunities the mobile web of the near future will find.
• The frameworks are the inverse of control. Above, I excluded the library from consideration, because they can be easily switched. The frameworks control the life cycle of the application, give you an entry point for starting your code. Yes, you are responsible for the code itself, but you do not control the situation entirely.
• The frameworks are expensive when used on mobile devices. At least compared to the base languages. And from my point of view, the road is unacceptable. But everyone has their own idea of ​​the limits of admissibility.

Yes, frameworks offer the developer a lot of comfort. But I can't help thinking that it is better to invest in my own knowledge and skills within the web platform itself. The frameworks come and go, they are like the ebb and flow of the Internet, helping to implement and hone ideas and patterns. But if one day you find out that your favorite framework is no longer working or some important bug has not been fixed, the ability to understand the platform device will provide you with an invaluable service.

In the dry residue

I once wrote that developer comfort is less important than user needs. I still believe in it. Although I am also attracted by the easy life, I do not want to create poorly working products. And I don’t want users to pay a high “price” for them. Now I’m worried about the speed of initial loading of frameworks on mobile devices. But this is only the first step. There are other metrics: memory usage, long load on the CPU, frame rate. In general, we need to look for less costly solutions for users.

If we manage to implement fast-loading frameworks that consume little memory and work with a large frame rate, and even ensure the comfort of development, this would be ideal. But until then, I would prefer to use do without frameworks, at least in the mobile segment.