“Img” evolution: Gif without GIF format

tl; dr

• GIFs are cool, but in terms of quality and performance, they are terrible .
• Replacing GIF with video good idea, but there are drawbacks: they are not preloaded , they use range queries.
• Today you can use img src =".mp4" in the Safari Technology Preview.
• Preliminary results show that mp4s in tags are displayed 20 times faster and are decoded 7 times faster than the GIF equivalent - in addition to the fact that the file size is 1/14!
• Background CSS videos and responsive videos can be a cool thing.
• Finally, sinegraphs will be without GIF flaws.
• Now we are waiting for other browsers to follow: this post weighs 46 MB on Chrome, and only 2 MB in Safari TP.

Special thanks to: Eric Portis, Jack Noble, John Davis, Doron Sherman and Yoav Weiss.

---

The translation was made with the support of the company EDISON Software , which is professionally engaged in development for large customers, for example, developed the client-side of GameStars for the game “League of Legends” and developed the component of displaying objects on a three-dimensional map on the Unity engine , as well as created a service for tracking analyzes about 200 online stores (more than a million items).

---

Introduction

I both love and hate gif animations

Safari TP radically changed the approach to them. Now I love animated “gif”.
')


Animated GIFs are a feature. Quote from the GIF89a specification :

The graphics exchange format is not intended as a platform for animation, although it can be organized with limitations.

But they became a cool tool for sinegraphs , memes and creative expression . However, all this coolness is expensive. Animated GIFs are terrible for web performance. They are HUGE in size, affect cellular data, require more memory and processor performance, cause redrawing and are battery killers. As a rule, GIF-files are 12 times larger than H.264 videos and eat 2 times more energy when loaded and displayed in the browser . And we spend all these resources on things that don't even look very good - GIF is limited to 256 colors, which often makes GIF files awful (although there are some cool ways to get around ).

My daughter loves them, but she does not understand why her battery is always dead.

GIFs have many advantages: they are requested immediately using the browser's preloader, they are automatically played and looped. They are short. Market research has shown that users have a higher level of interaction with them and usually prefer both short videos (<1 minute) and sinegraphs (a photo on which minor repetitive movements occur) than longer videos and still images. Animated GIFs are great for users to use directly.



So how did I come from love / hate for GIF to love for "Gifs"?

In the latest Safari Tech Preview, thanks to the hard work of Jer Nobl , we can use MP4 files in tags. The proposed example is not a long-format video, but a microformat, looping video without sound — like a GIF. Take a look for yourself:

img src="rocky.mp4"



Cool! This will be useful In many areas - for business, for ease of use and especially for working on the Internet.

... but we already have video tags

As already noted , using the video tag is much better for performance than using animated GIFs. That is why in 2014, Twitter noticeably added animated GIF support without adding GIF support. Twitter instead recoded GIF to MP4 on the fly and showed them inside the video tags. Since all browsers now support H.264, this was a very simple transition.



Transcoding animated GIFs to MP 4 is pretty straightforward. You just need to run ffmpeg -i source.gif output.mp4

However, not everyone can rebuild their CMS and convert img to video . Even if it is possible, there are three problems with this method of delivering a GIF-like (Gif) microformat video:

1. Slow performance of the browser with the tag "video"

As Doug Sillars recently pointed out in an HTTP Archive post , there is a huge performance loss in a visual presentation using the video tag.



Unlike img tags, browsers do not load video content. Usually, preloaders only preload JavaScript, CSS, and image resources, because they are crucial to the page layout. Since the video content can be of any length - from microform to long form - the video tags are skipped until the main stream is ready to analyze its content. This delays the loading of video content for many hundreds of milliseconds.



For example, the video at the top of the Velocity page takes only 5 seconds to load the page. This is the 27th most requested resource , and even it is not completely downloaded until rendering starts after downloading web fonts.

Worse, many browsers assume that video tags contain long format content. Instead of downloading the entire video file immediately, which will waste your Internet traffic, even in cases when you don’t watch all the videos, the browser will first perform a 1-byte request to check if the server supports HTTP Range requests . Then it will follow several range requests of different block sizes to ensure that the video will be adequately (but not too) buffered. The result is multiple TCP transmissions before the browser even decodes the content and a significant delay before the user sees anything. When using high latency cellular data, these round-trips can download videos of hundreds or thousands of milliseconds.



And what's even worse than the native video element? Typical javascript video player. Often the easiest way to embed videos on a site is to use a service such as YouTube or Vimeo, and avoid the difficulties of video encoding, hosting, and UX. This is usually a great idea, but for microformat video or heavy content like video, it just adds a delay due to the javascript player and support for the resources that these hosting services introduce (css / js / jpg / woff). In addition to video markup, you force the browser to download, evaluate, and execute the javascript player, and only after that the video can start downloading.



As many know, I love my Loki jacket because of its built-in sleeves, balaclava and hood, which is designed for helmets. But take a look at the Loki USA home page, which uses a great video posted on Vimeo:





If you look closely, you will see that JavaScript for the player is actually requested shortly after the DOM download is complete. But it does not load completely and is ready to launch the video stream much later.



WPT Results

2. You can not right-click and save video

The longest video content - Vlog, TV, movies - is provided via JavaScript-based players. Usually, these players provide users with a convenient “share now” link or bookmark tool, so you can go back to YouTube (or anywhere) and find the video again. In contrast, microformat content — like memes and blue-graphs — usually does not come through players, and users expect to be able to download GIFs and send them to friends, as they can with any image on the Internet. This dancing cat's mame was funny - I have to share it with all my friends!

If you use video tags to display micro-form videos, users will not be able to right-click, click and drag and save. And their joy from the cat-dancer becomes a disappointing surprise for the UX.

3. Violation of autoplay

Finally, using video and MP4 tags instead of img and GIF tags leads you to the current cat-and-mouse game situation between browsers and unscrupulous ad vendors who abuse the video autoplay attribute to get users attention. Historically, mobile browsers have ignored the attribute of autoplay and / or refused to play videos, requiring them to go into full-screen mode. Over the past couple of years, Apple and Google have relaxed their restrictions on embedded autoplay videos, allowing you to use Gif-like images with the video tag. But again, ad networks misuse this, causing additional restrictions: if you want to auto-reproduce the video tags, you need to turn off the sound ( muted ) from the content or remove the audio track in it.

... but we already have an animated WebP. And animated PNG.

GIF format is not the only format that supports animation. WebP and PNG also have animation support. But, like GIF, they were not designed for animation and weighed significantly more compared to dedicated video codecs, such as H.264, H.265, VP9 and AV1.

Animated PNG is now widely supported in all browsers, and while it also concerns the limitation of the color palette like GIF, it is still an inefficient file format for compressing video.

Animated WebP is better, but compared to classic video formats it is still problematic. In addition to having no accepted standard, the animated WebP lacks subsampling and wide-range support. In addition, the support ecosystem is fragmented. Not even all versions of Android, Chrome and Opera support animated WebP - even if these browsers talk about support using Accept: image / webp . You need Chrome 42, Opera 15+ or Android 5+.



Thus, although animated WebP compression is much better than animated GIF or PNG, although we can do better. (See Comparison of file sizes below.)

And eat fish and ...

By adding support for classic video formats (such as MP4) to be included in the tags img Safari Technology Preview fixed performance problems and UX. Now our videos with microformats can be small and productive (for example, MP4, transferred using the video tag), and they can be easily uploaded, automatically launched and shared by it (for example, as with our old friend, GIF).

img src="ottawa-river.mp4"

So how much faster will it be? Open the developer tools and see the difference between Safari Technology Preview and other browsers:



Unfortunately, Safari does not work well with WebPageTest, and creating reliable test benchmarks is rather difficult. Similarly, the use of Tech Preview is quite small, so comparing performance with RUM tools is not yet practical.

However, we can do two things. First, compare the size of the raw bytes, and second, use the Image.decode () rule to measure the impact of various resources on the device.

Memory saving

The first is saving memory. For comparison, I recoded 100 popular animated gif images from giphy.com and then converted them to vp8 / vp9 / webp / h264 / h265.

NB : These results should be considered only as superficial. Each codec can be configured much better, as you can see that the result with vp9 is worse than with vp8. It is necessary to conduct a more comprehensive study that takes into account SSIM.

Below are the median (p50) conversion results :



Yes, animated WebP is small, but any video format is much smaller. This should not surprise anyone, since modern video codecs are highly optimized for streaming online video. The H.265 algorithms are very good, as I expected, AV1 too.

The benefits here are not only the transfer speed, but also the substantial $$ savings for end users .

Using video in img tags, it will load much faster with cellular communication.

Improved decoding and visual performance

Next, consider the effect of decoding and display effects on viewing. H.264 (and H.265) has a significant advantage in that they use hardware decoding instead of using the main core.

How can we measure it? Since browsers have not yet implemented the proposed Hero image API , we can use Steve Sauder’s User Timing and Custom Metric strategy as a good comparison when the image begins to be displayed to the user. It does not measure the frame rate, but tells you when the first frame is displayed. We can also use the recently adopted Image.decode () convention to measure decoding performance. On the test page below, I add a unique GIF and MP4 in the img tag 100 times and compare the decoding and rendering performance.

let image = new Image;
t_startReq = new Date().getTime();
document.getElementById("testimg").appendChild(image);
image.onload = timeOnLoad;
image.src = src;
return image.decode().then(() => { resolve(image); });

The results are impressive. Even on my powerful MacBook Pro 2017 running locally, without connecting to the network, we see that GIF files take 20 times longer than MP4 to draw the first frame (signaled by the onload ), and 7 times more for decoding .



Surprised? Copy the repository and check it out. I note that the addition of network transmission conditions for GIF and, accordingly, MP4 will disproportionately distort the test results. In particular, since decoding can begin before the end of the last byte, the delta between transmission, mapping and decoding becomes much smaller. In fact, this suggests that only saving memory by itself will significantly improve user experience. However, if you exclude a network connection, as I did when running localhost, you can see that using video has significant performance advantages.

How can you realize this?

So now that Safari Technology Preview supports this design pattern, but how can you use it without displaying corrupted images on unsupported browsers? Good news! It is relatively easy.

Option 1: Use responsive images

Ideally, the easiest way is to use the source type attribute of the HTML5 picture tag.



I would like to say that we can stop at this. However, there is a nasty WebKit error in Safari that causes the preloader to load the first source regardless of the mimetype declaration. The main DOM loader will recognize the error and select the correct source. However, the damage will already be done. Preloader loses the ability to load the image earlier, and, moreover, loads the wrong version, losing bytes. The good news is that I fixed this error, and the fix should appear in Safari TP 45.

In short, using the picture and source type to select the mime type is not recommended unless the next version of Safari is installed on 90% + users.

Option 2. Use MP4, animated WebP and Fallback for GIF

If you do not want to change the HTML markup, you can use HTTP to send MP4 to Safari with content negotiation. To do this, you must create multiple copies of your bluegraphs (as before) and Vary responses in the Accept and User-Agent headers.

This will be a bit cleaner if WebKit BUG 179178 is resolved and you can add a test for the Accept: video / * header (for example, you can check Accept: image / webp ). But the end result is that each browser gets the best format for img video based on microformat, which it supports:



In nginx it would look something like this:

map $http_user_agent $mp4_suffix {
default "";
"~*Safari/605" ".mp4";
}

location ~* .(gif)$ {
add_header Vary Accept;
try_files $uri$mp4_suffix $uri =404;
}

Of course, don't forget to specify Vary: Accept, User-Agent , to tell proxy providers and CDNs to cache each response differently. In fact, you should mark Cache-Control as private and use TLS to ensure that less complex ISP Proxy servers do not cache content.

Option 3: use RESS and “video” tag

If you can manipulate your HTML, you can apply the Responsive-server-side (RESS) technology. This option translates browser detection logic into your HTML output.

For example, you can do this with PHP:



As above, do not forget to fix the Vary: User-Agent header to inform your CDN that there are different versions of your HTML for caching. Some CDNs automatically evaluate Vary headers, while others can support it with a simple CDN configuration update.

Bonus: do not forget to remove the audio track

Now, since you are not converting GIF to MP4, but rather converting MP4 to GIF, we also need to remember to remove the audio track to save memory. (Please tell me that you are not using GIF as a source. Correct.) Audio tracks take extra bytes in the file size, which we can simply free up, since we know that it will play without sound in any case. The easiest way with ffmpeg:

ffmpeg -i cats.mp4 -vcodec copy -an cats.mp4

Is there a size limit

When I write this, Safari will blindly download all the videos you specify on the img tag, no matter how long it takes. On the one hand, this is expected because it helps improve browser performance. However, it can be stupid if you force the user to load a 120-minute video. I tested videos of different sizes, and they were all uploaded while the user was browsing the site. So be kind to the users. If you want to add a longer video, use the video tag to improve performance.

What's next. Adaptive videos and background videos

Now that we can post MP4 via img tags, the doors open for many new uses. Two of which come to mind: adaptive video and background video. , MP4 srcsets , Client Hints Content-DPR, image media , .



CSS background-image: url (.mp4) .

Conclusion

img , Safari Technology Preview GIF- , , GIF. , , . , , , - . , . Google? Microsoft? Mozilla? Samsung? .