Broadcasting online video with nginx

What is online video?

By the term online video, I mean the long-term broadcasting of a live video signal (for example, from a television studio). Traditional means of uploading video (flv- and mp4 streaming) in this case do not work, simply because the file containing the entire video stream does not exist.

In this article we will talk not about the organization of video hosting, but about the organization of video broadcasting on the air. These are two fundamentally different tasks, and usually the ways to solve them differ significantly from each other.

RTMP protocol

RTMP - Real Time Messaging Protocol ( wiki ) is a protocol that is commonly used to distribute live video and audio content to clients. It is convenient because in AS3 there is its support out of the box, it does not require too many resources on the client and supports many buns (for example, broadcasting with variable bitrates and switching clients to a higher quality with a free channel).

NB: In addition to RTMP, there is a set of other media streaming protocols (RTSP, Apple HTTP Live Streaming, etc.), but they are not covered in this article.

')

RTMP is the brainchild of Adobe. The only official server that can stream an RTMP stream is Adobe Flash Media Server. Its price and performance, unfortunately, leave much to be desired, so various developers have attempted to create more or less compatible alternative implementations of the protocol. Unfortunately, the RTMP protocol has known problems with licensing, for example, its official specification, if you follow it strictly, does not allow you to write a working RTMP server. However, there are several implementations:

Red5 is one of the first RTMP servers, written in Java
Wowza is a proprietary RTMP server written in Java. In addition to RTMP, it also supports RTSP, Apple HLS, and Smoothstreaming (Silverlight).
erlyvideo is a free RTMP server with separately distributed proprietary modules, written in Erlang. The project is written by Russian programmers and is actively developing.

Java slows down

In the tests that I conducted, Red5 and Wowza showed indecently poor performance. With a megabit video stream Wowza with 1000 viewers online ate up about 300% of the CPU time (Intel Xeon E5607). If you want to broadcast at least for 20,000 people online, then you need to purchase 20 servers? It's too expensive.

What to do?

Use non-blocking operations. The brakes in Wowza and Red5 follow because of far from the most optimal event handling scheme. To implement a really fast streaming server, you need to use an efficient event handling method (for linux, this is epoll). Roman Harutyunyan ( blog , github-profile , rarutyunyan ) implemented this very scheme of work in his RTMP-server , implemented as a module to nginx.

nginx-rtmp-module

Advantages:

Unreal smart. On the same physical server that I used for the Wowza tests, nginx withstood 2500 megabit connections per core, i.e. 10,000 for the entire server.
Convenient configs :). After kilometer-long XML config Wowza these look like salvation.
Responsiveness of the author to fitchrequest.

Disadvantages:

The project is young, there are bugs, which, however, are promptly corrected.
A little functionality. The server broadcasts only in RTMP, transcoding appeared just a few days ago, it has experimental status and is actually a wrapper over ffmpeg. UNIX way in all its glory.
No multithreading. The module can work only if nginx is started with one worker. To dispose of all processor cores, you need to run several workers on different IP addresses and / or ports.

Multithreading can be implemented using about such a crutch (Ubuntu):

for ip in $(cat /etc/network/interfaces | grep address | awk '{print $2}') ; do touch /etc/nginx/nginx.$ip.conf cp /etc/nginx/nginx.conf.skel /etc/nginx/nginx.$ip.conf sed -i "s/%IPADDR%/${ip}/g" /etc/nginx/nginx.$ip.conf /usr/sbin/nginx -c /etc/nginx/nginx.$ip.conf done

In this case, you should have written the skeleton of the config with the indication% IPADDR% instead of the IP address. For example, such:

 worker_processes 1; error_log logs/error.%IPADDR%.log debug; pid /var/run/nginx.%IPADDR%.pid; worker_rlimit_nofile 65536; events { worker_connections 16384; } rtmp { server { listen %IPADDR%:1935; chunk_size 4000; application live { live on; pull live stream %masterIP%; } } } http { server { listen %IPADDR%:8080; location /stat { rtmp_stat all; rtmp_stat_stylesheet stat.xsl; } location /stat.xsl { root /srv/nginx/html; } } }

However, the author promises to implement normal multithreading in future releases.

With the help of nginx-rtmp-module I managed to completely utilize the 10-gigabit channel. At the same time, the main problem was not the gluttony of nginx itself, but the need to tune the network card and kernel parameters so that the software interrupts did not eat up 100% of the CPU.