Hardware acceleration in the life of the coder. Yandex Workshop
Hello! My name is Alexander Zavyalov. In Yandex, I develop interfaces. Recently, I spoke to my colleagues with a report on hardware acceleration in the life of the layout designer, where I also touched on related topics. He told about the performance of web pages, how it is measured and what it can strive for.
Based on the report, I prepared this post. I will talk about how browsers optimized the process of drawing: where to start and what has come to. What can be done now so that the life of web designers and users becomes a little better. I hope someone bump into any improvements. I would have been nice.
I will start with an example: one of the Yandex . Browser promo pages . On this site we see a full page slider. Now it has already been altered, but if you look at the saved copy, not everything is so smooth there. We go in the timeline, turn on the record and start clicking on the slides. We see that everything is not very good - even at 30 fps we do not always fit. It is enough to add one CSS property - the famous “zero hack” among the designers of the transform: translateZ (0), in order to speed up the rendering twice.
')
I could stop at this, but I saw the
I have already mentioned fps - frames per second several times. This is the frequency with which the display changes. On most screens, it is 60 Hz, i.e. the picture changes 60 seconds in a second. Through complex mathematical calculations we come to the conclusion that every frame is created every 16.6 milliseconds. If we have any animation on the page, we need the browser to create and send a new frame in 16 ms. If he stops doing this (and we skillfully know how to stop him from doing so), the browser starts to skip frames. The number of fps decreases, the smoothness decreases.
Here are examples from the timeline. Each color has its own process here. I think they are familiar to many: script, render, paint / composite. The culprit of the problems we have most often the last one, who is responsible for rendering and compiling the page.
Before we talk about this in more detail, we will briefly recall what the browser does to render the site.
Suppose we have a simple page with this html-code:
It will look like this:
The first thing that a browser does is select tags and build a DOM tree from them:
Parallel parsing CSS, superimposed on the DOM, turns RenderTree. This is a display tree containing Render Objects — the objects that we need to show the user. The render tree for this page looks like this:
It looks like a DOM tree with some differences. It may contain pseudo-elements that are not in the DOM tree. Some nodes from the DOM are also missing here: the head section, link tags, script tags, elements with display: none — everything we don’t need to show. We can add some code to CSS, from which the DOM will not change, and new elements will appear in the Render Tree.
After this, the layout starts (layout / reflow). When the browser has arranged the elements on the page, it remains to fill it all with pixels. Here you need to tell a little about rasterization: transfer the image from the vector to the final grid of pixels, which the browser will show the user. In Chrome, this is the Skia graphics library.
In the process of the site may need to somehow interact with the user. Suppose we need to move the picture. The browser creates a dirty area and starts rasterizing and drawing everything with which it intersects.
The whole process of displaying the page by the browser looks like this: HTML parsing, CSS parsing, DOM Tree, Render Tree, layout and rendering. Moreover, the viewport was originally a single raster. This is how the Chromium developers illustrate this process:
In shared memory lay the final bitmap of the page. With the help of the windowing API, this bitmap was drawn into the required browser window. In the process of scrolling pages appeared previously invisible Render Objects that need to be shown. The browser created and rendered a new viewport raster. This is not optimal, because When scrolling, you do not want to wait until everything new is rasterized. I want to move on the already finished page. The browser developers thought about this earlier than me and introduced a new process - compositing.
Now the page is divided into layers. Initially, it was one layer that matched the entire document. This layer was rasterized, saved in memory and displayed on the screen. In this case, when scrolling, the browser did not need to deal with rasterization, it operated with an already prepared image. The process of placing on the screen a finished raster is called compositing. This process was performed by the CPU. Performance noticeably improved, but the developers decided to optimize the process a bit more by transferring compositing to the GPU.
And they did it. Microsoft was the first to talk about this, they promised this feature in IE9. In fact, the first shot Firefox in the fourth version. Then IE9 came out, and during the year all the other browsers were pulled up. To date, all major browsers, including mobile, display pages with accelerated compositing using the GPU. What happens in this case? The page is also divided into layers that are rasterized into textures and sent to the GPU, where they are stored in the video memory. Here comes the new part of the system - compositor. She instructs the GPU on how to build the final image.
A layer (composite layer) is part of a page, a subtree of the DOM. It is drawn independently and assembled in the GPU. It can stretch, move, hide (through transparency) without rendering.
Layers are created for a reason. There are conditions that the browser encounters the element in a new composite layer.
Based on the report, I prepared this post. I will talk about how browsers optimized the process of drawing: where to start and what has come to. What can be done now so that the life of web designers and users becomes a little better. I hope someone bump into any improvements. I would have been nice.
I will start with an example: one of the Yandex . Browser promo pages . On this site we see a full page slider. Now it has already been altered, but if you look at the saved copy, not everything is so smooth there. We go in the timeline, turn on the record and start clicking on the slides. We see that everything is not very good - even at 30 fps we do not always fit. It is enough to add one CSS property - the famous “zero hack” among the designers of the transform: translateZ (0), in order to speed up the rendering twice.
')
I could stop at this, but I saw the
background-size: cover property in the CSS slide. This property stretches the image over the entire area of ​​the block. Why this was done remains a mystery, because block height is fixed. So the property did nothing but resize the image from 600 to 540 px in height. Resizing images is a very expensive operation, so I turned off this CSS property and everything became absolutely good. So in just a couple of minutes we increased the fps 4 times. All these changes are included in the new version of the page.I have already mentioned fps - frames per second several times. This is the frequency with which the display changes. On most screens, it is 60 Hz, i.e. the picture changes 60 seconds in a second. Through complex mathematical calculations we come to the conclusion that every frame is created every 16.6 milliseconds. If we have any animation on the page, we need the browser to create and send a new frame in 16 ms. If he stops doing this (and we skillfully know how to stop him from doing so), the browser starts to skip frames. The number of fps decreases, the smoothness decreases.
Here are examples from the timeline. Each color has its own process here. I think they are familiar to many: script, render, paint / composite. The culprit of the problems we have most often the last one, who is responsible for rendering and compiling the page.
Before we talk about this in more detail, we will briefly recall what the browser does to render the site.
From markup to image
Suppose we have a simple page with this html-code:
<!DOCTYPE html> <html> <head> <title> </title> <meta charset="utf-8"> <link rel="stylesheet" href="main.css"> </head> <body> <section> <h1> !</h1> <img src="i.jpg" alt=""> <p> .</p> <p> !</p> </section> <script src="main.js"></script> </body> </html> It will look like this:
The first thing that a browser does is select tags and build a DOM tree from them:
Parallel parsing CSS, superimposed on the DOM, turns RenderTree. This is a display tree containing Render Objects — the objects that we need to show the user. The render tree for this page looks like this:
It looks like a DOM tree with some differences. It may contain pseudo-elements that are not in the DOM tree. Some nodes from the DOM are also missing here: the head section, link tags, script tags, elements with display: none — everything we don’t need to show. We can add some code to CSS, from which the DOM will not change, and new elements will appear in the Render Tree.
After this, the layout starts (layout / reflow). When the browser has arranged the elements on the page, it remains to fill it all with pixels. Here you need to tell a little about rasterization: transfer the image from the vector to the final grid of pixels, which the browser will show the user. In Chrome, this is the Skia graphics library.
In the process of the site may need to somehow interact with the user. Suppose we need to move the picture. The browser creates a dirty area and starts rasterizing and drawing everything with which it intersects.
The whole process of displaying the page by the browser looks like this: HTML parsing, CSS parsing, DOM Tree, Render Tree, layout and rendering. Moreover, the viewport was originally a single raster. This is how the Chromium developers illustrate this process:
In shared memory lay the final bitmap of the page. With the help of the windowing API, this bitmap was drawn into the required browser window. In the process of scrolling pages appeared previously invisible Render Objects that need to be shown. The browser created and rendered a new viewport raster. This is not optimal, because When scrolling, you do not want to wait until everything new is rasterized. I want to move on the already finished page. The browser developers thought about this earlier than me and introduced a new process - compositing.
Now the page is divided into layers. Initially, it was one layer that matched the entire document. This layer was rasterized, saved in memory and displayed on the screen. In this case, when scrolling, the browser did not need to deal with rasterization, it operated with an already prepared image. The process of placing on the screen a finished raster is called compositing. This process was performed by the CPU. Performance noticeably improved, but the developers decided to optimize the process a bit more by transferring compositing to the GPU.
Hardware acceleration
And they did it. Microsoft was the first to talk about this, they promised this feature in IE9. In fact, the first shot Firefox in the fourth version. Then IE9 came out, and during the year all the other browsers were pulled up. To date, all major browsers, including mobile, display pages with accelerated compositing using the GPU. What happens in this case? The page is also divided into layers that are rasterized into textures and sent to the GPU, where they are stored in the video memory. Here comes the new part of the system - compositor. She instructs the GPU on how to build the final image.
Layers
A layer (composite layer) is part of a page, a subtree of the DOM. It is drawn independently and assembled in the GPU. It can stretch, move, hide (through transparency) without rendering.
Layers are created for a reason. There are conditions that the browser encounters the element in a new composite layer.
- Elements with 3D transformation.
Source: https://habr.com/ru/post/239169/
All Articles