RTCP REMB: we twist the video call settings in the browser

The resourceful guys from Tribe not only found their niche at the junction of conference and gamedev (which is already interesting in itself), but also know well what is happening under the hood of WebRTC. Product Manager Timote Le Bornier periodically describes the technical side of his brainchild; we decided to translate his article about the non-standard, but very interesting functionality of WebRTC. Sure, you will be interested. In the meantime, as you read, we will continue to prepare an insider about our updated video conferencing ... But that's another story.

---

Fine tune and manage stream quality using REMB

So, now you most likely know that we worked on creating a custom MCU for the Tribe - if you suddenly don’t know, then that's why we decided to do it and how we do it .

But more to the point: let's see what happens under the hood.

1. A WebRTC connection begins with an exchange mechanism called “SDP offer / answer” —the exchange of information on the supported codecs and the corresponding parameters between the participants.
2. After that, the participants try to communicate with each other directly: start listening to the UDP ports and use ICE, STUN, and (hereinafter) TURN to establish a UDP connection.
3. When a UDP connection is established, participants use it to exchange encryption parameters using the DTLS-SRTP protocol. After that, participants use:
   
   • SRTP (Secure RTP over UDP) to send and receive media
   • SRTCP (Secure RTCP over UDP) protocol for flow control and out-of-band data exchange.
   
   

Stack of WebRTC protocols (modified schema, original here: webrtc-security.imtqy.com )
')

Understanding RTCP

The above mentioned RTP and RTCP packets are securely stored in SRTP and SRTCP. What are they used for?

RTP provides end-to-end transport for real-time data: audio and video. In some ways, RTCP is the “brother of RTP”: it is used together with RTP and improves the data transfer protocol by monitoring delivery, as well as management and identification.

In the WebRTC session, the MCU acts as an intermediary between the participants: they are connected to the MCU, but not directly with each other.


WebRTC — MCU Base Topology

For example, take a session with two participants. The MCU receives the media stream from the FOO participant, decodes and encodes it, and sends it to the BAR participant. At the same time, the same thing happens with the flow from BAR to FOO.


MTP and RTCP protocols

Symmetry is obvious, so let's focus on one direction and see what is going on there:


Focus on one side

When the BAR browser receives the media stream from the MCU, it also receives statistics from the RTCP Sender Reports (SR). BAR will use RTP's sequential packet numbers, RTP time stamps, own time, and Sender Reports to calculate what we call RTCP Receiver Reports (RR).

The BAR browser uses all of this data to calculate the maximum expected bitrate, and then send this value to the MCU using RTCP REMB (Remote Estimate Maximum Bitrate).

We use REMB (effectively)

Package format RTCP REMB tools.ietf.org/html/draft-alvestrand-rmcat-remb-00#section-2.2

The value of the calculated maximum bitrate is encoded in the fields BR Exp and BR Mantissa.

An example implementation can be seen in the Chromium code: cs.chromium.org/chromium/src/third_party/webrtc/modules/rtp_rtcp/source/rtcp_packet/remb.cc

In the Go language, we converted this message to a PacketALFBRemb structure:


When you need to know the bitrate from this structure, you can do this:


If, on the contrary, you have a bitrate and you want to fill in the Exp and Mantissa fields in the PacketALFBRemb structure, then you will do something like this:

Bandwidth and RTCP REMB

This is one of the advantages of the MCU - the ability to make REMB RTCP package to control the outgoing bitrate in the browser:


MCU sends RTCP REMB to control the outgoing browser bit rate FOO

The MCU can also take a bitrate estimate from the receiver and change its encoding settings in accordance with it so that they match the estimated maximum bitrate; it also optimizes the outgoing bandwidth of the MCU:


MCU changes coding settings using RTCP REMB values ​​from BAR member

We control the bitrate

How can I test the effect of RTCP REMB on the browser bitrate FOO? Pretty easy: just send REMB packets with a variable bit rate (for example, 64Kb and 128Kb every 10 seconds), and then look at the chrome graphs : // webrtc-internals .

In Go, it will be like this:


When you open chrome: // webrtc-internals , you will see something like this:


This is a clear confirmation that you are able to change and track the bitrate on your part in real time. Which parameters are better suited to your needs and whether it is necessary to optimize the bandwidth and the quality of the stream :)

---

You probably noticed that RTCP REMB is now in deprecated status. The idea was that the bandwidth should control the data source (and not the remote receiver / participant) - well, in the article we showed that it is still possible and how to do it.

In our case it was useful to get a point control over the strip and quality :)