Optimizing pseudo-streaming FLV video

One of the projects of our company is an online-video service similar to youtube. For streaming and streaming capabilities, a wonderful nginx web server with the ngx_http_flv_module module is used.

Everything was fine until the number of views reached a level, when not only the network channels of the servers were overloaded, but the disk subsystem of servers ceased to cope with read requests.

Problems with the disk subsystem solved simply installing the 10th raid. But it was necessary to do something with the network and the idea appeared to upload files for viewing not in one stream, but discretely, in chunks of the same size. Fortunately in flash, it was implemented all that is needed. According to the documentation in the NetStream object, there is an appendBytes () method with the necessary functionality. Here is an excerpt from the official documentation:

“Sends a ByteArray object to the NetStream instance for playback. Call this method for a NetStream object in data creation mode. To put a NetStream object into data creation mode, call the NetStream.play (null) method for the NetStream instance created for the NetConnection object that is connected to null. The appendBytes () method cannot be called for a NetStream object that is not in data creation mode, otherwise an exception is thrown. ”
')
It would seem, we take a class Loader, we register the heading Range, but we are waiting for mine here from Adobe developers. The standard Action Script classes that work with the HTTP protocol do not use the Range header. Again, from official sources it becomes known that the Range header is blocked in the code of classes that work in the HTTP protocol, for security reasons, in player version 9. But one thing was pleasing: that the lock is not native, but in the classes themselves, from which it followed that You can write your HTTP client using the Socket base class. Fortunately, in the google bins there was a ready-made class HTTPClientLib . The code below loads the first megabyte of the video file and plays it:

private var ns:NetStream; private var video:Video; private var meta:Object; private var client:HttpClient; private var filesize:Number = 0; private var loadedBytes:Number = 0; private var data:ByteArray = new ByteArray(); private var datadelta:Number = 1024*1024; //    - private var file:String = %   %; private function init():void{ // ,      var nsClient: Object = {}; //       nsClient.onMetaData = metaDataHandler; //   ;          null       - var nc:NetConnection = new NetConnection(); nc.connect(null); // ,          ns = new NetStream(nc); ns.client = nsClient; //     ns.addEventListener(NetStatusEvent.NET_STATUS,netStatusHandler); //   ns.addEventListener(IOErrorEvent.IO_ERROR,nsIOErrorHandler); //      video = new Video(); video.attachNetStream(ns); //  .  ,       video.smoothing = true; uic.addChild(video); // ,      NetStream client = new HttpClient(); //     loadData(); //     NetStream ns.play(null); } private function loadData():void{ //     var uri:URI = new URI(file); //,      var request:HttpRequest = new Get(); //maxdata      //loadedBytes -     var maxdata:Number = loadedBytes+datadelta; //    ,           if (maxdata>=filesize and filesize>0){ request.addHeader('Range','bytes='+loadedBytes+'-'); } else { request.addHeader('Range','bytes='+loadedBytes+'-'+maxdata); } //    ,       data //        //         NetStream  //     client.listener.onData = function(e:HttpDataEvent):void { var bytes:ByteArray = new ByteArray(); bytes = e.bytes; bytes.position = 0; data.writeBytes(bytes); }; //      client.listener.onComplete = function(e:HttpResponseEvent ):void{ //       loadedBytes+=data.length;       filesize = Number(e.response.header.getValue('Content-Length'))/1024; //     ns.appendBytes(data); //  data.clear(); //  .      inLoaded = false; }; //    client.request(uri,request); } 

One important point for the code to work: the NetStram object must be in data creation mode, which is turned on so NetStream.play (null). This must be done before the first video data arrives.

Further, as needed, the remaining parts of the file should be loaded in the same pieces of 1 megabyte. The size of 1 mb (equal to about 15 seconds of video) was obtained experimentally during a large number of tests for our system. The upload is controlled by a timer that executes the following code on the event.

 if ((!inLoaded) && (ns.bufferLength <= Math.ceil(loadtime+timeDelta)) && (loadedBytes < filesize)){ inLoaded = true; loadData(); } 

Download will run, subject to the following conditions:
inLoaded = false - status indicator, stream true - data is loaded, false - not;
ns.bufferLength <= Math.ceil (loadtime + timeDelta) - in the playback buffer, the remaining time is less than or equal to the time of the previous load (loadtime) plus the delta for the stock (timeDelta);
loadedBytes <filesize - the end of the played file is not reached;

An important point in the data creation mode is not generating events NetStream.Play.Start, NetStream.Play.Stop. Therefore, it is necessary to monitor the amount of downloaded data or the video time played.

The variable timeDelta allows you to adjust to the quality of the user's channel: in the event that the playback buffer is empty before the next data fragment is loaded, the delta increases.

A bit about rewinding

In data creation mode, some methods of the NetStream class change their normal functionality. This also applies to the seek () method - it searches for a keyframe (the so-called I-frame) located closest to the specified point. In the data creation mode, after calling the seek method, the class starts to ignore all the data transmitted by the appendBytes () method, before calling the appendBytesAction () method. The method argument can have the following NetStreamAppendBytesAction.RESET_BEGIN or NetStreamAppendBytesAction.RESET_SEEK values, the first implies the presence in the fresh data of the new flv file header and zeroes the playback counters in the class. Here is an example of the video rewind code:

 //    ,   - -,               for (var i:Number=0;i<positions.length;i++){ if ((positions[i]<=seekpositions)&&(positions[i+1]>=seektpositions)){ //                loadedBytes = positions[i]; break; } } // seek       ns.seek(seektime); //C  NetStream         ,      ns.appendBytesAction(NetStreamAppendBytesAction.RESET_SEEK); //    loadData(); 

times is an array of navigation points in time;
positions - an array of byte-shifting navigation points;

A couple of notes

One feature of the HTTPClientLib library is that a file with a socket security policy (crossdomain.xml) for cross-domain work is requested from port 843 of the server, unlike standard objects that can pick it up from the 80th. Therefore, the following entry was added to the nginx server configuration:

 server { listen 843; server_name localhost; location / { rewrite ^(.*)$ /crossdomain.xml; } error_page 400 /crossdomain.xml; location = /crossdomain.xml { root /home/www-root; default_type text/x-cross-domain-policy; } } 

Results

The use of a player that downloads videos using this technique has significantly reduced the amount of “unnecessary - parasitic” traffic, and reduced the load on streaming servers. An average of 200,000 traffic views decreased by 30%. However, the above methodology has a “negative” side, namely: the user has at least 10.1 Flash Player versions.

Full code example