Adventures in a separate stream. Yandex report
How to work with images on the client, while maintaining smooth UI? Interface developer Pavel Smirnov talked about this based on his experience in developing photo search on the Market. From the report you can learn how to properly use Web Workers and OffscreenCanvas.
- During this half an hour we will talk about adventures. I will tell you about my adventure and I really hope that my report will inspire you and you will take and do the same with you.
')
First I wanted to talk about some new or not so new technologies that our browsers give us and which allow us to do cool things. But it seems to me that it would not be very fun, because everyone can go to the MDN and read something. Therefore, I will tell the story of one feature that I did with the Market team.
Let's introduce myself first. My name is Pasha, I am an interface developer for the Market team.
I mainly deal with mobile interfaces - search by map, offer card. I also rewrite the code from the old stack to the new one, and then from the new one to an even newer stack. And I try to make my interfaces good. It is worth saying what a good interface is.
Good interfaces have different characteristics. Firstly, it is convenient, secondly, it is beautiful, thirdly, it is available. But one of the characteristics that I want to talk about today is speed. And the speed often manifests itself in the smoothness of his work. Even small friezes can greatly change the user experience of our interfaces.
Let us turn to the plan of my talk today. First we will talk about the task I did: search for a picture on the Market. Further I will tell you what problems I had to solve in order to implement this functionality. Here we will remember a little how your script works in the browser, and look at the technologies that helped me. A small spoiler: this is Web Workers and OffscreenCanvas.
Let's return to the task. A few months ago Luba, our product manager, approached me. Lyuba deals with the problems of product selection on the Market. Now we have several options for finding the goods. One of them is to enter something into the search box.
For example, “buy a red iPhone X ² of Samara”. And we will find something. Or we can use the directory tree. In this directory we have categories and subcategories.
But what if I want to find something on the Market, not knowing what it is called, but either I have a photo of this thing, or I see it at someone’s place?
I'll tell you the real case. I once went with my friends to a cafe. We ordered lemonade there, you know, in such a jug, and this jug had such a strange thing. I even have a photo. It was designed so that when you pour lemonade into a glass, ice does not get into it. We thought - the thing is cool, but we disagreed on how this thing is called and, in general, what it was intended for. Therefore, we found it on Yandex.Mapk ...
But I thought it would be cool if I could not only search for this thing, but buy it right away or at least find out the price, read reviews, features, etc. At this point, our dreams coincided with Anyone, and we decided make this functionality on the Market.
What does this functionality represent? It allows the user to upload a photo or image, you can even immediately take a photo and send it to the Market. We analyze this picture with the help of Yandex search technologies, find a product on it and show the user the issue with these products. It sounds simple, but if it were so simple, I would not have made my report. So that you can see what this feature is, let me show it.
Watch the first demo
I will show on the production. Let's first load the very thing we were looking for and see what happens.
We found some goods and specifically this thing. This thing is called a strainer. In order to search for something else, yesterday, at a colleague on the table, I photographed one book, let's look for it. Here is a book, perhaps someone read it. Called "Perfect Code". Also finds it somehow, and for some reason with a 18+ limit. This is probably a little strange.
Let's return to our report. What problems have I encountered? The first problem is that the user starts downloading anything, including huge pictures. For example, my phone makes pictures of three to four megabytes, it is quite a lot. Such pictures to send to the backend is inefficient. This is a long time to analyze them, so you need to do something about it. But everything is simple - we will cut, compress, somehow resize this picture on the client.
How are we going to do this? We have a file. And we will somehow read this file. We will read using the FileReader API. I will briefly tell you what it is.
This is such a browser API that allows us to read the downloaded file and do something with it. You can read in different ways, we now look at it. Here are its capabilities, and we have some object that came back from input on the change event. Let's try to read it.
The code will look like this. There is nothing complicated here. We have a Reader object created from the FileReader constructor, to which we hang the developer of the load event. Further we will read this file as DataURL. DataURL - a string that represents the contents of the file, encoded via Base64. Like we read, we must somehow cut it off. First let's load it all into a picture. We have a tag or img element, and we’ll upload it right there.
The code will look something like this. We create an img element, upon the load Reader event in the src attribute we load our string and we will execute everything further after loading our string into img.
We will do what we wanted - to crop the image. We will compress it, and here such a thing as Canvas will help us, a very powerful tool. It allows you to do a lot. But here we will simply draw our image on this Canvas, and if the size of the image exceeds the maximum allowed, we will fit them a little bit. We will also be able to pick up this picture with the Canvas of the desired degree of compression.
Like that. Another small disclaimer: the code is very simplified here, I am not pointing out everything. We have error handling and other things, but so that everything fits on the slide and is clear and understandable in the report, I omitted some of the details.
We have the size of the picture, we just look at them. There are some constants allowed to us. If the sizes of the images exceed our constants, we simply equalize them and specify our Canvas to our Canvas.
Next we draw our picture on this Canvas.
Take a 2d context, we need a 2d image, and try to draw using the drawImage method. DrawImage is an interesting method that takes, if I'm not mistaken, nine parameters. But they are not all required, we will use only five. We will take Image and those two zeros, is the offset or indent of the image. We need a left upper point. Let's draw with the sizes necessary to us.
Next, from this Canvas, we also take our BaseUR-encoded DataURL and turn it into a blob - a special object that is convenient for us to send to the server. It seems to be all. Everything is working. The picture is cropped, the picture is sent, the picture is recognized.
But then I began to notice something. When I tested this solution, then when I loaded the image, especially on weak devices, my interface slowed down a bit. That button was not pressed, the element is not so scrolled. Did you get the feeling that your code works 99% of the time and works well, but sometimes a little bit does not work? And you can give for testing, and probably no one will notice. Yes, and users probably will not notice, especially on weak devices.
I have never had such a thing, and I decided to fix it. This turned out to be a problem. If the image is large, then with the manipulation of trimming, compression, it took us some time, and in this small-small time, our interface was non-responsive.
First, I figured out why this is happening. Here it is worth a little bit to remember how JavaScript works in the browser. I will not go into details, this is a topic for a large report. Just remember some moments.
We have javascript running in one thread, let's call it the main one. And we have such a thing in the browser as an event loop. We will immediately say that this is a model. In some browsers, the event loop is organized differently, but as the name implies, in general it is a cycle. It processes certain tasks in the queue strictly in order.
Unpleasant moment: until he handles one task, he will not go to the next. I will show the demo that I’ve written down, it demonstrates it. She is classic.
Watch the second demo
I have a GIF image and CSS animation made in different ways: one with translatex, the other with position: relative left, the third with JavaScript, namely requestAnimationFrame. This is where the hedgehog is spinning. What will i do?
I'll block the main thread for five seconds. You know, usually cool guys calculate a certain Fibonacci number, but I wrote an infinite loop with a break after five seconds.
What will happen? You immediately noticed that the hedgehog stopped spinning, and the lower cat, which is animated using translatex, also stopped driving. But let's see the same demo in another browser, for example Safari. The cat on the gif stopped running.
Why am I showing all this? First, browsers are different, you need to take this into account. Secondly, when our flow is blocked by something, some things will stop working. For example - JavaScript animation. Or let's show that the text will no longer stand out from us, the buttons will no longer be pressed.
This is a very abstract example. Let's not block the stream for five seconds, but take our task, load the photo, cut it off, compress it and draw it here. We will not send it anywhere, it will not be very revealing.
Watch the third demo
I have a powerful enough MacBook here, and so that everything looks more convincing, we will slow down the processor six times. This allows you to do DevTools. Download our photo. “Perfect Code” will help us again. As we can see, the same thing happens as when blocking the main thread.
Let us then return to our task and think about how we will deal with it.
By the way, if you look at the profiler, we will see this. In the red frame - our microtask, which blocks the main thread. We see that he is blocking him for almost five seconds. This is on a fairly powerful computer, and on weaker devices it will be even more noticeable.
We turn to the solution. I will immediately say what I used and what I did, and then we will sort all these things out. First, I used Web Workers. They allow us to bring some tasks to a separate thread. And secondly, in the context of Web Workers, the DOM is not available to us. To deal with this situation, we will use other tools. Image will not be available to us, the classic Canvas will be available, and therefore we use Canvas and some other tricks.
Let's quickly recall what Workers are and what they are for. They allow you to run JavaScript in a separate thread, not mostly. And the Workers thread does not interfere in any way with the rendering stream of the main interface. Therefore, we can perform some complex computational tasks without slowing down our interface.
We have a tool that allows you to transfer something to Workers and return something from Workers. Let's see an example.
So we create our Worker with the help of a constructor. There you need to pass the path to the file. We can even pass on a blob. And we have a Message event handler. In this case, it will simply display something on the screen. Next we can send some data to our Worker.
What with support? Everything is good here. Workers is a well-known tool, not new, but many of my friends think that they are not supported everywhere. This is not true.
Now look at the OffscreenCanvas. As we have already seen, Canvas is a very powerful tool, but, unfortunately, in the context of Web Workers it is not available to us, so we will use the alternative. This is already a fairly new thing called OffscreenCanvas. It allows you to do about the same things as Canvas, only now off-screen, that is, in the context of Web Workers. Of course, we can do this in the context of a window, but for now we will not.
What is there with support? As you can see, there is a lot of red here. OffscreenCanvas is normally supported in Chrome only. There is also an option with Firefox, but there is still under the flag, and the Canvas works only with the WebGL context. Here you can ask - why am I talking about such a cool thing like OffscreenCanvas, which does not work anywhere?
A small digression. We have some levels of browser support in the Market. And we have two quantities. One value characterizes a browser that we don’t support at all. This is about half a percentage point of browser popularity.
And there is a second value. It includes those browsers that we support, but only critical functionality. Here, without Workers, all the search functionality works, but with a few friezes. I think it's okay, and our team thinks it's okay. Let's see how we will implement it.
Here is a diagram of what we will do. We even will have files that we will read through FileReader. But in the main thread, we will send it to Web Workers, where it will be cut off, shrink and return to us, and we will send it to the server.
Let's see the code of our Worker. First, we create an OffscreenCanvas instance with the width and height we need.
Further, as I said, the Image element is not available to us in the context of Workers, so here we use the createImageBitmap method, which will make us the data structure that characterizes our image.
From the interesting: we see self here. Whoever is not familiar with Web Workers, this thing points to the execution context. We don't care here, window or this, use self. This method is asynchronous, I used await here for both compactness and convenience, why not?
Next we get the same image and do the same thing we did before. Draw on the canvas and return.
From simple. Previously, we took DataURL and converted everything into a blob. But here we immediately available method convertToBlob. Why haven't I used it before? Because support was worse. But since we’ve gone all bad, we’ll use OffscreenCanvas, what prevents us from using convertToBlob?
This blob we will return in the main stream, from where we send it to the server. Or, as in the demos, draw it.
So we create the Worker in the main thread, listen to some messages from it and draw or send to the server. There is nothing important here. Worker will accept our files.
Let's go back to our demo.
Watch the fourth demo
All the same demo, all the same three cats and a hedgehog. I'll turn on throttling again, slowing the processor six times. Upload all the same picture. As we see, at the moment when the photo was drawn, the animation did not stop, the hedgehog kept spinning, the interface remained, and we achieved what we wanted.
But can this solution be improved?
Here, by the way, profiler. We do not see here the huge Microtasks for the five seconds that we saw before.
Improve - can be. Using Transferable objects. Here it is worth going back again. When we passed our DataURL or blob through the postMessage mechanism, we copied this data. This is probably not very effective. It would be cool to avoid this. Therefore, we have a mechanism that allows you to transfer data to Web Workers as if in a package.
Why do I say "as if"? When we transfer this data to Workers, we lose control over them in the main thread — we cannot interact with them in any way. Here is the second limitation. We are not all types of data ... can transfer to Web Workers. With a string this will not work, we will do otherwise.
Let's look at the code. First, we transfer data in a slightly different way. Here is our postMessage. See, there is such an array with loadEvent.target.result. Such an interface allows us to transfer our data as Transferable objects, losing control over them.
By the way, whoever writes on Rust will probably hear something familiar. And we will read our file no longer as a string, but as an ArrayBuffer. This is a stream of lidar binary data that is not directly accessible. So we have to do something else with them.
Let's go back to our imageworks. It has become much more interesting. First, we take our buffer and do such a terrible thing as Uint8ClampedArray. This is a typed array. As the name implies, the data in it are numbers of the sign, that is, numbers from zero to 255, which will represent the pixel of our image.
With the third argument we pass on such a strange thing as width multiplied by height multiplied by four. Why exactly four? Right, RGBA. We have three values ​​per color and one per alpha channel.
Next, we will make ImageData from this array, a special data type that can be easily drawn on the canvas. There is nothing interesting here. We just take an array and pass it to the constructor. Next, we draw our image on the canvas in the same way, but using a different method, under ImageData. Then everything is the same as before.
We turn to the conclusions. I told you today about a task that I did not long ago. What did I notice in her?
The smoothness of the interface is very important. When a user lays a little bit, frisks a little, the button is not pressed, this can lead to a severe deterioration of the UX. Browsers work differently. We looked at a spherical example with Safari and with Yandex Browser. We see that if you checked your interface for smoothness in one browser, you should look at the others.
Something needs to be done with blocking scripts if they continue for a long time. In my case, I brought it to Web Workers. But there are probably other approaches, you can somehow divide them into smaller ones, then you have to think. , Web Workers, .
What next? . . . , 200 , .
Web Workers . , , .
:
Thank you very much.
- During this half an hour we will talk about adventures. I will tell you about my adventure and I really hope that my report will inspire you and you will take and do the same with you.
')
First I wanted to talk about some new or not so new technologies that our browsers give us and which allow us to do cool things. But it seems to me that it would not be very fun, because everyone can go to the MDN and read something. Therefore, I will tell the story of one feature that I did with the Market team.
Let's introduce myself first. My name is Pasha, I am an interface developer for the Market team.
I mainly deal with mobile interfaces - search by map, offer card. I also rewrite the code from the old stack to the new one, and then from the new one to an even newer stack. And I try to make my interfaces good. It is worth saying what a good interface is.
Good interfaces have different characteristics. Firstly, it is convenient, secondly, it is beautiful, thirdly, it is available. But one of the characteristics that I want to talk about today is speed. And the speed often manifests itself in the smoothness of his work. Even small friezes can greatly change the user experience of our interfaces.
Let us turn to the plan of my talk today. First we will talk about the task I did: search for a picture on the Market. Further I will tell you what problems I had to solve in order to implement this functionality. Here we will remember a little how your script works in the browser, and look at the technologies that helped me. A small spoiler: this is Web Workers and OffscreenCanvas.
Let's return to the task. A few months ago Luba, our product manager, approached me. Lyuba deals with the problems of product selection on the Market. Now we have several options for finding the goods. One of them is to enter something into the search box.
For example, “buy a red iPhone X ² of Samara”. And we will find something. Or we can use the directory tree. In this directory we have categories and subcategories.
But what if I want to find something on the Market, not knowing what it is called, but either I have a photo of this thing, or I see it at someone’s place?
I'll tell you the real case. I once went with my friends to a cafe. We ordered lemonade there, you know, in such a jug, and this jug had such a strange thing. I even have a photo. It was designed so that when you pour lemonade into a glass, ice does not get into it. We thought - the thing is cool, but we disagreed on how this thing is called and, in general, what it was intended for. Therefore, we found it on Yandex.Mapk ...
But I thought it would be cool if I could not only search for this thing, but buy it right away or at least find out the price, read reviews, features, etc. At this point, our dreams coincided with Anyone, and we decided make this functionality on the Market.
What does this functionality represent? It allows the user to upload a photo or image, you can even immediately take a photo and send it to the Market. We analyze this picture with the help of Yandex search technologies, find a product on it and show the user the issue with these products. It sounds simple, but if it were so simple, I would not have made my report. So that you can see what this feature is, let me show it.
Watch the first demo
I will show on the production. Let's first load the very thing we were looking for and see what happens.
We found some goods and specifically this thing. This thing is called a strainer. In order to search for something else, yesterday, at a colleague on the table, I photographed one book, let's look for it. Here is a book, perhaps someone read it. Called "Perfect Code". Also finds it somehow, and for some reason with a 18+ limit. This is probably a little strange.
Let's return to our report. What problems have I encountered? The first problem is that the user starts downloading anything, including huge pictures. For example, my phone makes pictures of three to four megabytes, it is quite a lot. Such pictures to send to the backend is inefficient. This is a long time to analyze them, so you need to do something about it. But everything is simple - we will cut, compress, somehow resize this picture on the client.
How are we going to do this? We have a file. And we will somehow read this file. We will read using the FileReader API. I will briefly tell you what it is.
This is such a browser API that allows us to read the downloaded file and do something with it. You can read in different ways, we now look at it. Here are its capabilities, and we have some object that came back from input on the change event. Let's try to read it.
The code will look like this. There is nothing complicated here. We have a Reader object created from the FileReader constructor, to which we hang the developer of the load event. Further we will read this file as DataURL. DataURL - a string that represents the contents of the file, encoded via Base64. Like we read, we must somehow cut it off. First let's load it all into a picture. We have a tag or img element, and we’ll upload it right there.
The code will look something like this. We create an img element, upon the load Reader event in the src attribute we load our string and we will execute everything further after loading our string into img.
We will do what we wanted - to crop the image. We will compress it, and here such a thing as Canvas will help us, a very powerful tool. It allows you to do a lot. But here we will simply draw our image on this Canvas, and if the size of the image exceeds the maximum allowed, we will fit them a little bit. We will also be able to pick up this picture with the Canvas of the desired degree of compression.
Like that. Another small disclaimer: the code is very simplified here, I am not pointing out everything. We have error handling and other things, but so that everything fits on the slide and is clear and understandable in the report, I omitted some of the details.
We have the size of the picture, we just look at them. There are some constants allowed to us. If the sizes of the images exceed our constants, we simply equalize them and specify our Canvas to our Canvas.
Next we draw our picture on this Canvas.
Take a 2d context, we need a 2d image, and try to draw using the drawImage method. DrawImage is an interesting method that takes, if I'm not mistaken, nine parameters. But they are not all required, we will use only five. We will take Image and those two zeros, is the offset or indent of the image. We need a left upper point. Let's draw with the sizes necessary to us.
Next, from this Canvas, we also take our BaseUR-encoded DataURL and turn it into a blob - a special object that is convenient for us to send to the server. It seems to be all. Everything is working. The picture is cropped, the picture is sent, the picture is recognized.
But then I began to notice something. When I tested this solution, then when I loaded the image, especially on weak devices, my interface slowed down a bit. That button was not pressed, the element is not so scrolled. Did you get the feeling that your code works 99% of the time and works well, but sometimes a little bit does not work? And you can give for testing, and probably no one will notice. Yes, and users probably will not notice, especially on weak devices.
I have never had such a thing, and I decided to fix it. This turned out to be a problem. If the image is large, then with the manipulation of trimming, compression, it took us some time, and in this small-small time, our interface was non-responsive.
First, I figured out why this is happening. Here it is worth a little bit to remember how JavaScript works in the browser. I will not go into details, this is a topic for a large report. Just remember some moments.
We have javascript running in one thread, let's call it the main one. And we have such a thing in the browser as an event loop. We will immediately say that this is a model. In some browsers, the event loop is organized differently, but as the name implies, in general it is a cycle. It processes certain tasks in the queue strictly in order.
Unpleasant moment: until he handles one task, he will not go to the next. I will show the demo that I’ve written down, it demonstrates it. She is classic.
Watch the second demo
I have a GIF image and CSS animation made in different ways: one with translatex, the other with position: relative left, the third with JavaScript, namely requestAnimationFrame. This is where the hedgehog is spinning. What will i do?
I'll block the main thread for five seconds. You know, usually cool guys calculate a certain Fibonacci number, but I wrote an infinite loop with a break after five seconds.
What will happen? You immediately noticed that the hedgehog stopped spinning, and the lower cat, which is animated using translatex, also stopped driving. But let's see the same demo in another browser, for example Safari. The cat on the gif stopped running.
Why am I showing all this? First, browsers are different, you need to take this into account. Secondly, when our flow is blocked by something, some things will stop working. For example - JavaScript animation. Or let's show that the text will no longer stand out from us, the buttons will no longer be pressed.
This is a very abstract example. Let's not block the stream for five seconds, but take our task, load the photo, cut it off, compress it and draw it here. We will not send it anywhere, it will not be very revealing.
Watch the third demo
I have a powerful enough MacBook here, and so that everything looks more convincing, we will slow down the processor six times. This allows you to do DevTools. Download our photo. “Perfect Code” will help us again. As we can see, the same thing happens as when blocking the main thread.
Let us then return to our task and think about how we will deal with it.
By the way, if you look at the profiler, we will see this. In the red frame - our microtask, which blocks the main thread. We see that he is blocking him for almost five seconds. This is on a fairly powerful computer, and on weaker devices it will be even more noticeable.
We turn to the solution. I will immediately say what I used and what I did, and then we will sort all these things out. First, I used Web Workers. They allow us to bring some tasks to a separate thread. And secondly, in the context of Web Workers, the DOM is not available to us. To deal with this situation, we will use other tools. Image will not be available to us, the classic Canvas will be available, and therefore we use Canvas and some other tricks.
Let's quickly recall what Workers are and what they are for. They allow you to run JavaScript in a separate thread, not mostly. And the Workers thread does not interfere in any way with the rendering stream of the main interface. Therefore, we can perform some complex computational tasks without slowing down our interface.
We have a tool that allows you to transfer something to Workers and return something from Workers. Let's see an example.
So we create our Worker with the help of a constructor. There you need to pass the path to the file. We can even pass on a blob. And we have a Message event handler. In this case, it will simply display something on the screen. Next we can send some data to our Worker.
What with support? Everything is good here. Workers is a well-known tool, not new, but many of my friends think that they are not supported everywhere. This is not true.
Now look at the OffscreenCanvas. As we have already seen, Canvas is a very powerful tool, but, unfortunately, in the context of Web Workers it is not available to us, so we will use the alternative. This is already a fairly new thing called OffscreenCanvas. It allows you to do about the same things as Canvas, only now off-screen, that is, in the context of Web Workers. Of course, we can do this in the context of a window, but for now we will not.
What is there with support? As you can see, there is a lot of red here. OffscreenCanvas is normally supported in Chrome only. There is also an option with Firefox, but there is still under the flag, and the Canvas works only with the WebGL context. Here you can ask - why am I talking about such a cool thing like OffscreenCanvas, which does not work anywhere?
A small digression. We have some levels of browser support in the Market. And we have two quantities. One value characterizes a browser that we don’t support at all. This is about half a percentage point of browser popularity.
And there is a second value. It includes those browsers that we support, but only critical functionality. Here, without Workers, all the search functionality works, but with a few friezes. I think it's okay, and our team thinks it's okay. Let's see how we will implement it.
Here is a diagram of what we will do. We even will have files that we will read through FileReader. But in the main thread, we will send it to Web Workers, where it will be cut off, shrink and return to us, and we will send it to the server.
Let's see the code of our Worker. First, we create an OffscreenCanvas instance with the width and height we need.
Further, as I said, the Image element is not available to us in the context of Workers, so here we use the createImageBitmap method, which will make us the data structure that characterizes our image.
From the interesting: we see self here. Whoever is not familiar with Web Workers, this thing points to the execution context. We don't care here, window or this, use self. This method is asynchronous, I used await here for both compactness and convenience, why not?
Next we get the same image and do the same thing we did before. Draw on the canvas and return.
From simple. Previously, we took DataURL and converted everything into a blob. But here we immediately available method convertToBlob. Why haven't I used it before? Because support was worse. But since we’ve gone all bad, we’ll use OffscreenCanvas, what prevents us from using convertToBlob?
This blob we will return in the main stream, from where we send it to the server. Or, as in the demos, draw it.
So we create the Worker in the main thread, listen to some messages from it and draw or send to the server. There is nothing important here. Worker will accept our files.
Let's go back to our demo.
Watch the fourth demo
All the same demo, all the same three cats and a hedgehog. I'll turn on throttling again, slowing the processor six times. Upload all the same picture. As we see, at the moment when the photo was drawn, the animation did not stop, the hedgehog kept spinning, the interface remained, and we achieved what we wanted.
But can this solution be improved?
Here, by the way, profiler. We do not see here the huge Microtasks for the five seconds that we saw before.
Improve - can be. Using Transferable objects. Here it is worth going back again. When we passed our DataURL or blob through the postMessage mechanism, we copied this data. This is probably not very effective. It would be cool to avoid this. Therefore, we have a mechanism that allows you to transfer data to Web Workers as if in a package.
Why do I say "as if"? When we transfer this data to Workers, we lose control over them in the main thread — we cannot interact with them in any way. Here is the second limitation. We are not all types of data ... can transfer to Web Workers. With a string this will not work, we will do otherwise.
Let's look at the code. First, we transfer data in a slightly different way. Here is our postMessage. See, there is such an array with loadEvent.target.result. Such an interface allows us to transfer our data as Transferable objects, losing control over them.
By the way, whoever writes on Rust will probably hear something familiar. And we will read our file no longer as a string, but as an ArrayBuffer. This is a stream of lidar binary data that is not directly accessible. So we have to do something else with them.
Let's go back to our imageworks. It has become much more interesting. First, we take our buffer and do such a terrible thing as Uint8ClampedArray. This is a typed array. As the name implies, the data in it are numbers of the sign, that is, numbers from zero to 255, which will represent the pixel of our image.
With the third argument we pass on such a strange thing as width multiplied by height multiplied by four. Why exactly four? Right, RGBA. We have three values ​​per color and one per alpha channel.
Next, we will make ImageData from this array, a special data type that can be easily drawn on the canvas. There is nothing interesting here. We just take an array and pass it to the constructor. Next, we draw our image on the canvas in the same way, but using a different method, under ImageData. Then everything is the same as before.
We turn to the conclusions. I told you today about a task that I did not long ago. What did I notice in her?
The smoothness of the interface is very important. When a user lays a little bit, frisks a little, the button is not pressed, this can lead to a severe deterioration of the UX. Browsers work differently. We looked at a spherical example with Safari and with Yandex Browser. We see that if you checked your interface for smoothness in one browser, you should look at the others.
Something needs to be done with blocking scripts if they continue for a long time. In my case, I brought it to Web Workers. But there are probably other approaches, you can somehow divide them into smaller ones, then you have to think. , Web Workers, .
What next? . . . , 200 , .
Web Workers . , , .
:
- How Browsers Work: Behind the scenes of modern web browsers
- WebGL/Three.js OffscreenCanvas -
- OffscreenCanvas — Speed up Your Canvas Operations with a Web Worker
Thank you very much.
Source: https://habr.com/ru/post/453626/
All Articles