How we learned to draw texts on Canvas
We are developing a platform for visual collaboration . To display the content, we use the Canvas: everything is drawn on it, including texts. There is no ready-made solution for displaying texts on Canvas one-to-one as in html. Over several years of working with text rendering, we studied various options for implementation, stuffed a lot of bumps, and seem to have found a good solution. I'll tell you in the article how we moved from Flash to Canvas and why we refused SVG foreignObject.
We created the product in 2015 in Flash. Inside Flash, there is a text editor that can work well with texts, so we didn’t have to do anything extra to work with texts. But at that time, Flash was already dying, so we moved from it to HTML / Canvas. And we faced the task - to display the text on the Canvas as in the html-editor, without breaking the texts created in the Flash version when moving.
')
We wanted to make it so that the user could edit the text directly in our product, without noticing the transition between editing and drawing modes. We saw this solution: when you click on a text area, a text editor opens in which you can change the text; the editor can be closed by removing the cursor from the text area. In this case, the display of text on the Canvas should 1 in 1 correspond to the display of text in the editor.
As an editor, we used an open library, but the ready-made libraries for rendering from html to Canvas did not suit us with the speed of work and insufficient functionality.
We considered several solutions:
How does SVG foreignObject work: we have HTML from the editor → put HTML into foreignObject → some magic → we get an image → we add an image to the canvas
About magic. Despite the fact that most browsers support the foreignObject tag, each has its own peculiarities for using the result with canvas. FireFox works with a Blob object, in Edge you need to make Base64 for an image and return a data-url, and in IE11 the tag does not work at all.
After working with SVG, we had some interesting bugs that we didn’t notice in Flash. Text with the same size and font was displayed differently in different browsers. For example, the last word in a line could be transferred and run into the text below. It was important for us that users get the same type of widgets, regardless of the browsers in which they work. On Flash, this was not a problem, because he is the same everywhere.
We solved this problem. Firstly, for all single-line texts, the width was always assumed, regardless of the browser and data from the server. For height, the difference remains, but in our case it does not interfere with users.
Secondly, experimentally, we came to the conclusion that it is necessary to add several unusual css-styles for the editor and svg in order to reduce the display difference between browsers:
What we ended up with thanks to the SVG <foreignObject>:
Everything worked well, but once designers came to us and asked to add font support to create mockups.
We wondered if we could do this with foreignObject. It turned out that he had a peculiarity, which, when solving this problem, becomes a fatal flaw. It can display HTML within itself, but cannot access external resources, so all the resources it works with must be converted to base64 and added inside svg.
This means that if you have four texts that are written by OpenSans, you need to download this font to the user four times. This option did not suit us.
We decided that we would write our Canvas Text with ... good performance, support for the vector image, let's not forget about IE 11
Why is vector image important to us? In our product, any object on the board can be zoomed in, and with a vector image we can create it only once and reuse it, regardless of the zoom. Canvas.fillText draws a bitmap image: in this case, we need to redraw the image at each zoom, which we thought would have a significant effect on performance.
First we created a simple prototype to test its performance.
The principle of operation of the prototype:
The prototype had several tasks: check that the redrawing of the Canvas at the scale will take place without delay and that the time for turning the html into an object will be no more than creating an svg image.
The prototype coped with the first task, the scale almost did not affect the performance when drawing texts. With the second task, problems arose: processing large volumes of text takes enough time and the first performance measurements showed poor results. To draw text from 1K characters, the new approach required almost 2 times more time than svg.
We decided to use the most reliable way to optimize the code - “replace the test with the one we need” ;-). But seriously, we went to the analysts and asked how long the texts are most often created by our users. It turned out that the average text size is 14 characters. For such short texts, our prototype showed significantly better performance results, since the dependence of speed on the volume of the text is linear, and the wrapper in svg is almost always executed at the same time, regardless of the length of the text. It suited us: we can lose in performance on long texts, but for most cases our speed will be better than svg.
After several iterations of the work on updating Canvas Text, we had the following algorithm:
Stage 1. We break into logical blocks
Stage 2. We collect in one object with coordinates and styles
The advantage of this approach is that we can cover all the code from HTML to a text object with unit tests. Thanks to this, we can separately check the rendering and the parsing itself, which helped us to significantly speed up the development.
As a result, we made support for fonts and IE 11, covered everything with unit tests, and the rendering speed in most cases was higher than that of foreignObject. Checked on beta users and released. It seems a success!
So far, the guys with the right-hand letter system have not written to tech support. It turned out that we forgot about the existence of such languages:
Fortunately, it was easy to add support for the right-hand letter system, since the standard Canvas.fillText already supports it.
But while we were dealing with this, we met even more interesting cases that fillText could not support. We are faced with bidirectional texts in which part of the text is written from right to left, then from left to right and again from right to left.
The only solution we knew was to go to the W3C browser specification and try to repeat it inside Canvas Text. It was difficult and painful, but we were able to add basic support. More about bidirectional: one and two .
If you have experience in solving such problems - share them in the comments.
Moving from Flash
We created the product in 2015 in Flash. Inside Flash, there is a text editor that can work well with texts, so we didn’t have to do anything extra to work with texts. But at that time, Flash was already dying, so we moved from it to HTML / Canvas. And we faced the task - to display the text on the Canvas as in the html-editor, without breaking the texts created in the Flash version when moving.
')
We wanted to make it so that the user could edit the text directly in our product, without noticing the transition between editing and drawing modes. We saw this solution: when you click on a text area, a text editor opens in which you can change the text; the editor can be closed by removing the cursor from the text area. In this case, the display of text on the Canvas should 1 in 1 correspond to the display of text in the editor.
As an editor, we used an open library, but the ready-made libraries for rendering from html to Canvas did not suit us with the speed of work and insufficient functionality.
We considered several solutions:
- Standard Canvas.fillText. Able to draw text as in html, can be styled, works in all browsers. But he does not know how to draw links as in a html-editor multiline texts with different formatting. These difficulties are solvable, but require a large amount of time;
- Draw DOM over Canvas. The option did not suit us, because we have in the product each created object has its own z-index on canvas. And mixing it with the DOM z-index will not work.
- Convert html to svg. He can turn html into an image thanks to the foreignObject element. This allows you to bake html inside svg and work with it as with an image. We chose this option.
Features of SVG foreignObject
How does SVG foreignObject work: we have HTML from the editor → put HTML into foreignObject → some magic → we get an image → we add an image to the canvas
About magic. Despite the fact that most browsers support the foreignObject tag, each has its own peculiarities for using the result with canvas. FireFox works with a Blob object, in Edge you need to make Base64 for an image and return a data-url, and in IE11 the tag does not work at all.
getImageUrl(svg: string, browser: string): string { let dataUrl = '' switch (browser) { case browsers.FIREFOX: let domUrl = window.URL || window.webkitURL || window let blob = new Blob([svg], {type: 'image/svg+xml;charset=utf-8'}) dataUrl = domUrl.createObjectURL(blob) break case browsers.EDGE: let encodedSvg = encodeURIComponent(svg) dataUrl = 'data:image/svg+xml;base64,' + btoa(window.unescape(encodedSvg)) break default: dataUrl = 'data:image/svg+xml,' + encodeURIComponent(svg) return dataUrl } After working with SVG, we had some interesting bugs that we didn’t notice in Flash. Text with the same size and font was displayed differently in different browsers. For example, the last word in a line could be transferred and run into the text below. It was important for us that users get the same type of widgets, regardless of the browsers in which they work. On Flash, this was not a problem, because he is the same everywhere.
We solved this problem. Firstly, for all single-line texts, the width was always assumed, regardless of the browser and data from the server. For height, the difference remains, but in our case it does not interfere with users.
Secondly, experimentally, we came to the conclusion that it is necessary to add several unusual css-styles for the editor and svg in order to reduce the display difference between browsers:
- font-kerning: auto; manages font kerning. Read more
- webkit-font-smoothing: antialiased; responsible for anti-aliasing. More details .
What we ended up with thanks to the SVG <foreignObject>:
- We can draw any html: text, tables, graphics
- The tag returns a vector image
- The tag works in all modern browsers except IE11
Why we abandoned foreignObject
Everything worked well, but once designers came to us and asked to add font support to create mockups.
We wondered if we could do this with foreignObject. It turned out that he had a peculiarity, which, when solving this problem, becomes a fatal flaw. It can display HTML within itself, but cannot access external resources, so all the resources it works with must be converted to base64 and added inside svg.
This means that if you have four texts that are written by OpenSans, you need to download this font to the user four times. This option did not suit us.
We decided that we would write our Canvas Text with ... good performance, support for the vector image, let's not forget about IE 11
Why is vector image important to us? In our product, any object on the board can be zoomed in, and with a vector image we can create it only once and reuse it, regardless of the zoom. Canvas.fillText draws a bitmap image: in this case, we need to redraw the image at each zoom, which we thought would have a significant effect on performance.
We create a prototype
First we created a simple prototype to test its performance.
The principle of operation of the prototype:
- Give the function "text";
- From it we get an object in which there is every word from the text, with coordinates and styles for drawing;
- Give the object to the Canvas;
- Canvas draws text.
The prototype had several tasks: check that the redrawing of the Canvas at the scale will take place without delay and that the time for turning the html into an object will be no more than creating an svg image.
The prototype coped with the first task, the scale almost did not affect the performance when drawing texts. With the second task, problems arose: processing large volumes of text takes enough time and the first performance measurements showed poor results. To draw text from 1K characters, the new approach required almost 2 times more time than svg.
We decided to use the most reliable way to optimize the code - “replace the test with the one we need” ;-). But seriously, we went to the analysts and asked how long the texts are most often created by our users. It turned out that the average text size is 14 characters. For such short texts, our prototype showed significantly better performance results, since the dependence of speed on the volume of the text is linear, and the wrapper in svg is almost always executed at the same time, regardless of the length of the text. It suited us: we can lose in performance on long texts, but for most cases our speed will be better than svg.
After several iterations of the work on updating Canvas Text, we had the following algorithm:
Stage 1. We break into logical blocks
- We break the text into blocks: paragraphs, lists;
- We break blocks into smaller blocks by styles;
- We break blocks into words.
Stage 2. We collect in one object with coordinates and styles
- We count the width and height of each word in px;
- We connect the separated words, since paragraph 2 some words were divided into several;
- From the words we collect strings, if the word does not fit into the string, we trim it until it fits;
- We collect paragraphs and lists;
- Calculate x, y for each word;
- We get the finished object to draw.
The advantage of this approach is that we can cover all the code from HTML to a text object with unit tests. Thanks to this, we can separately check the rendering and the parsing itself, which helped us to significantly speed up the development.
As a result, we made support for fonts and IE 11, covered everything with unit tests, and the rendering speed in most cases was higher than that of foreignObject. Checked on beta users and released. It seems a success!
Success lasted 30 minutes
So far, the guys with the right-hand letter system have not written to tech support. It turned out that we forgot about the existence of such languages:
Fortunately, it was easy to add support for the right-hand letter system, since the standard Canvas.fillText already supports it.
But while we were dealing with this, we met even more interesting cases that fillText could not support. We are faced with bidirectional texts in which part of the text is written from right to left, then from left to right and again from right to left.
The only solution we knew was to go to the W3C browser specification and try to repeat it inside Canvas Text. It was difficult and painful, but we were able to add basic support. More about bidirectional: one and two .
Brief conclusions we made for ourselves
- To display HTML in picture use SVG foreignObject;
- Always analyze your product for decision making;
- Make prototypes They can show that complex solutions can only seem so at first glance;
- Write the code immediately so that it can be covered with tests;
- In international product it is important not to forget that there are many different languages, including biderectional.
If you have experience in solving such problems - share them in the comments.
Source: https://habr.com/ru/post/458624/
All Articles