New AV1 codec: accelerating video downloads in the browser

In this tutorial, we will learn how to use video on the Web, as is customary in 2019. Chrome and Firefox have begun to support the new AV1 codec - for them the video can be made in half.

Let's talk separately about how to replace GIF with video in AV1 and H.264 - then its size will drop 20-40 times.


')
YouTube is already using it in TestTube . Netflix announced that AV1 will be “ their next generation main codec ”.

We in Evil Martians already use it on our website and on the Ampliffer . In this article, I will share the experience of implementing AV1 and step by step how to insert video so that it works in all browsers.

Codecs and containers

The pictures are simple: either JPEG with PNG for all browsers, or make more compact files in WebP for modern browsers . We can always be sure that the .png files will have a PNG format (with the rare exception of PNG bombs that imgproxy can protect against ).

With video files everything is more complicated. A file extension ( .mp4 , .wmv , .webm or .mov ) refers only to the container. While video files consist of three different components:

1. Video codec determines how much you can compress video, and what you have to sacrifice. The main web video codecs are H.264, HEVC, VP9 and, now, AV1.
2. Audio codec compresses sound. Of course, it is not needed if there is no sound in the video. Popular options: MP3, Opus and AAC.
3. The container stores both video (compressed by some kind of video codec) and audio stream (compressed by some kind of audio codec). As well as additional data such as subtitles and meta-information. Popular containers: MP4, MOV, WebM.

When we see the .mp4 file extension, we can only say that the MP4 container was used. But the codecs in it can be different - the author could take H.264 and AAC, AV1 and Opus, or something else.

Behold AV1

AV1 is a video codec that was released a year ago, in March 2018. It was created to surpass the codecs of the previous generation - HEVC, VP9, ​​H.264 and VP8.


Chart of codec generations from Tzachi Levent-Levi

If you wondered exactly how AV1 managed to beat the rest of the codecs in compression, read the technical details in the Habré translations:
" Next-generation video: introducing AV1 "
" The codec of the new generation AV1: corrective directional filter CDEF "

Due to the new optimizations, AV1 compresses video by 30-50% better than H.264 or VP8, and up to 30% better than HEVC. But the codec was released recently and so far has several childhood diseases:

• The current encoder is not optimized. AV1 compresses video very slowly (the new fast encoder on Rust is already in development). Codec is not suitable for streaming. If we are talking about static videos on landing pages - this problem is not relevant to us.
• So far, the codec is supported only by desktop Chrome and Firefox under Windows. There is no support for Safari and Edge yet (although Microsoft is already testing it). You will need at least 2 files: AV1 for Chrome and Firefox and H.264 for other browsers.

The coolest thing in AV1 is that there is no “ check-in ” squares at low bit rates.


Comparing picture quality for different codecs at different bitrates - AV1 wins

Cooking AV1 correctly

Let's finally move on to practice. First, we define the container. In theory, AV1 can be placed in different containers, but MP4 is more compact and recommended in the specification. For the sound in AV1, we take Opus, because it compresses the sound perfectly .

For the video to work in all browsers, we will generate 3 files:

1. For desktop Chrome and Firefox on Windows ( 31% of the market for March 2019): MP4 container with AV1 for video and Opus for sound.
2. For Safari and Edge ( 16% of the market ) - MP4 with HEVC and AAC.
3. For the rest: a large MP4 file with H.264 and AAC.

You can take only AV1 and H.264 - the video will also work for everyone.

For compression, I recommend taking a console FFmpeg . There are many graphical utilities, but in the console it is easier to save options and then start the conversion automatically. Make sure you are using the latest version of FFmpeg. Versions 4.1 do not support AV1 to MP4.

For Mac OS X:

1. Install Homebrew .
2. brew install ffmpeg

For Linux, it is better to get the latest build from the official site - as long as many distributions do not have a version with support for AV1 to MP4:

1. wget https://johnvansickle.com/ffmpeg/releases/ffmpeg-release-amd64-static.tar.xz
2. tar -xf ffmpeg-release-amd64-static.tar.xz
3. sudo cp ffmpeg-4.1-64bit-static/ff* /usr/local/bin/

For Windows, you can install FFmpeg as directed by William Diaz .

We proceed to the conversion of the H.264 file, which we need for older browsers. Since all of our files use the MP4 container, I will use .av1.mp4 , .hevc.mp4 and .h264.mp4 postfixes. Do not be scared by the long team, we will analyze it all:

 #  SOURCE.mov     - ffmpeg -i SOURCE.mov -map_metadata -1 -c:a libfdk_aac -c:v libx264 -crf 30 -profile:v main -pix_fmt yuv420p -movflags +faststart -vf "scale=trunc(iw/2)*2:trunc(ih/2)*2" video.h264.mp4 

Now open video.h264.mp4 . If the quality is good and the size is large, try increasing -crf ( -crf 35 then -crf 40 ). This option will reduce the file size at the cost of reduced quality. Selecting a balance of quality and size is an art.

If the original video file does not exist, then you can convert the old H.264 file to AV1.

Now it's time to convert AV1 - I remind you that H.264 will be longer. The codec does not yet use the full power of the processor (it makes sense to start converting several files in parallel).

 ffmpeg -i SOURCE.mov -map_metadata -1 -c:a libopus -c:v libaom-av1 -crf 40 -b:v 0 -pix_fmt yuv420p -movflags +faststart -vf "scale=trunc(iw/2)*2:trunc(ih/2)*2" -strict experimental video.av1.mp4 

-crf with -crf for the perfect balance between quality and size.

Now the same for HEVC.

 ffmpeg -i SOURCE.mov -map_metadata -1 -c:a libfdk_aac -c:v libx265 -crf 30 -pix_fmt yuv420p -movflags +faststart -vf "scale=trunc(iw/2)*2:trunc(ih/2)*2" video.hevc.mp4 

Copy video.h264.mp4 , video.hevc.mp4 and video.av1.mp4 to the root of your site.

Understanding FFmpeg Options

Are the commands above look like a demon summoning spell? Do not worry, this is not PostCSS . Let's take a look at the options.

-i SOURCE.mov indicates the incoming file, from where FFmpeg will take video and audio streams, compress them and pack them into a new container.

-map_metadata -1 will remove the meta information from the video (for example, the program in which the video was created). In the Web, such information is rarely useful.

-c:a libopus or -c:a libfdk_aac set audio codecs. If you don't need sound, replace them with -an .

-c:v libaom-av1 selects a video codec - a library that compresses the frames of a video stream.

-crf 40 - Constant Rate Factor, balance of quality and size. It's like a JPEG quality slider, only it goes in a different direction (0 is the best quality and the largest file). The CRF scale is different for H.264 and AV1 - for H.264 it is up to 51, for AV1 it is up to 61. The CRF for AV1 and H.264 will be different.

Facebook picked up an approximate correspondence between CRF values ​​for H.264 and AV1:
19 → 27, 23 → 33, 27 → 39, 31 → 45, 35 → 51, 39 → 57.

-profile:v main used by H.264 to select a codec profile . Only “Main” will work in Safari.

-b:v 0 sets the minimum bit rate for AV1, so that the video has a constant quality.

-pix_fmt yuv420p (pixel format) is a tricky way to reduce file size. It leaves the original resolution for brightness, but reduces the resolution for color. Our eyes see the color worse, so they do not notice this trick. Remove this option if in your case it will interfere.

-movflags +faststart moves everything important to the beginning of the file so that the browser can play the video before the download is complete.

-vf "scale=trunc(iw/2)*2:trunc(ih/2)*2" will resize the sides of the video to the nearest even ones (some codecs can work with a resolution of 300 × 200 and 302 × 200, but will not work with 301 × 200). If you are sure that the resolution is divided by 2 everywhere, you can remove this option.

-strict experimental needed for AV1, its encoder is still experimental.

video.av1.mp4 sets the name of the final file.

We start video in browsers

Now we need each browser to download the video it supports. For this, the <source> has a type attribute. And I advise you to read about the options in <video> .

 <video controls width="600" height="400"> <source src="video.hevc.mp4" type="video/mp4; codecs=hevc,mp4a.40.2" /> <source src="video.av1.mp4" type="video/mp4; codecs=av01.0.05M.08,opus" /> <source src="video.h264.mp4" type="video/mp4; codecs=avc1.4D401E,mp4a.40.2" /> </video> 

<source> look like if…else expressions - the browser reads them from top to bottom until it finds one whose type it supports.

In type you can specify the entire file format: container ( video/mp4 for MP4), video codec ( av01.0.05M.08 for AV1, hevc for HEVC and avc1.4D401E for H.264) and audio codec ( opus for Opus and mp4a.40.2 for AAC).

Bonus: how to convert GIF to AV1 and H.264

In 2019, using GIF for short videos is a big sin. GIF weighs 20-40 times more than H.264 or AV1. GIF beats the CPU harder, makes the battery drain faster. If you need a short looped video, take video codecs. And FFmpeg can convert videos directly from GIF.

Convert GIF to H.264:

 ffmpeg -i IMAGE.gif -map_metadata -1 -an -c:v libx264 -crf 30 -profile:v main -pix_fmt yuv420p -movflags +faststart -vf "scale=trunc(iw/2)*2:trunc(ih/2)*2" video.h264.mp4 

Generating an even smaller AV1:

 ffmpeg -i IMAGE.gif -map_metadata -1 -an opus -c:v libaom-av1 -crf 50 -b:v 0 -pix_fmt yuv420p -movflags +faststart -vf "scale=trunc(iw/2)*2:trunc(ih/2)*2" -strict experimental video.av1.mp4 

Now paste the animation.h264.mp4 and animation.av1.mp4 into the HTML.

 <video autoplay loop muted playsinline width="300" height="200"> <source src="animation.av1.mp4" type="video/mp4; codecs=av01.0.05M.08" /> <source src="animation.h264.mp4" type="video/mp4" /> </video> 

The autoplay and loop options make a gif of video, a looped video that immediately plays after the page loads. playsinline blocks Safari from opening the video to full screen when clicking on the video.

Lead time

AV1 is still experimental. But it can already be used to make a quarter of your users happier. A couple of FFmpeg commands will generate video files. <video> From the very beginning, it was created to give video according to the capabilities of browsers. We are already using AV1 in production and everything works fine (except for the wait time until AV1-coder is finished).