Android streaming video
In this post I want to talk about some of the pitfalls that can be encountered when working with streaming video in Android applications. Specifically, it will be about video conversion and video delivery / playback protocols.
Immediately, I’ll make a reservation that I am not an expert in this field, but just want to share my recently gained experience.
Imagine that you are faced with the task of implementing an Android application capable of playing many files uploaded by users to your server. Write your youtube, with blackjack and codecs. To do this, you will have to solve at least two tasks: video conversion to a format supported on Android , video playback from a remote source. Consider both of these tasks in more detail.
And so, before you play some video on our Android device, you need to transcode this video into a supported format. The Android documentation clearly identifies a list of these formats.
')
In order to transcode files uploaded by users to your service, or to record stream from a TV tuner, you will need the help of a special utility ffmpeg , which is the de facto standard in the industry. Detailed instructions for its installation can be found on the site of the same name project .
The most common (in my opinion) video storage method is the MP4 container using the H.264 AVC codec . We will, in fact, consider them.
First of all, note that Android does not support all the features of the H.264 codec, but only a specific set — a profile called Baseline Profile (BP). For example, BP does not include such useful H.264 features as CABAC or B-Frames .
For us, this means that if we use these features when encoding a video, then Android will not have to lose this video. Although it may, if your phone is powerful enough and the vendor has taken care of installing and supporting additional codecs. For example, the video in the Main Profile is played without any problems on the Samsung Galaxy SII . On the phones of the usual class (for example, Samsung Galaxy Ace ) we will receive a message about the impossibility of playing the video and an error with the codec of the wrong codec in logcat .
But let's move from theory to practice. In order to compress the video, you must run the following command:
Let us consider in more detail each of the parameters:
It is also worth noting that you can do without specifying the profile, and explicitly enable / disable the necessary options for the H.264 codec through the -x264opts parameter, so that they would satisfy the conditions of BP . But this is a lesson for lovers.
The easiest way to play video from a remote server is to download it to temporary storage and play it locally. However, I think everyone understands that in view of the size of modern video recordings - this is not an option.
How to be? The Android platform offers us native support for the following technologies / protocols:
Consider them in order.
The easiest way to distribute video using a conventional web server, which essentially amounts to downloading a previously prepared file using the HTTP (S) protocol. The whole point in this case is that the file does not start playing at the end of the download, but as soon as enough data is downloaded (some buffer is filled).
Here it is worth clarifying that when using the MP4 container, it is necessary to form a file so that metadata about the video stream ( moov atoms ) is located at the beginning of the file (after the ftyp atom), before the video data ( mdat atoms ). This can be done by processing the file with the qt-faststart utility:
The main problem of progressive streaming is the impossibility of rewinding video to a non-downloaded moment, the presence of a sufficient amount of free space on the device and the need to support a large number of "fat" clients downloading videos on a web server.
Reproduction using this technology is supported by the Android platform natively. You can easily play a remote file using the standard MediaPlayer class (apart from the communication channel, power of the device and availability of free space).
This technology is a logical extension of progressive streaming 'a and allows you to solve one of its main problems - rewinding to a fragment that has not yet been downloaded. Applicable for MP4 / FLV containers with H.264 / AAC codec.
The only difference from progressive streaming 'a in this case is the fact that you need a special web server that, given the timestamp in the GET request, will give you the fragment of the video file you need. An example of such a web server can naturally serve as an Orthodox NGINX with its ngx_http_mp4_module .
I could not find any official information regarding the support of this standard in Android . However, empirically, it was found that it is present on at least HTC Desire and Samsung Galaxy SII devices. However, I want to note that even if there is no native support on your device, you can always use third-party players like MX Player , who independently implement the logic of downloading and playing video fragments with the required timestamp, which allows you to organize rewind.
Quite a non-standard data transfer protocol from Apple . Its essence boils down to the fact that the file being distributed is “sawn” into many small parts, joined by a special M3U8 format playlist . Data transfer occurs via the HTTP (S) protocol.
There are no problems with rewind and free space on the device in this case. Moreover, under certain conditions, you have the opportunity to choose the quality of the video being played.
However, there are problems. To “cut” a file and create a playlist will require processor resources, time and space on the server. To broadcast a file to the network, as in the previous examples, you will need an HTTP server (without any additional modules).
You can use VLC to cut a video file:
You can play this file at the URL localhost / pornofilm.m3u8 .
Support for HTTP Live Streaming at the native level in Android has been present since version 3.0. With the help of third-party players ( DicePlayer , MX Player ), judging by the wiki, you can achieve support from version 2.2.
A state-level application protocol designed specifically for video transmission. The command format is very similar to HTTP . The teams themselves resemble buttons on a conventional cassette recorder: PLAY, PAUSE, RECORD, etc.
Unlike HTTP Live Streaming, RTSP does not require splitting files into smaller parts and creating playlists. The necessary parts of the file will be generated and given to the client on the fly. VLC can be used as an RTSP server.
It is worth noting that the RTSP protocol itself does not determine the method of data transmission, but delegates this to other protocols. For example, RTP . To stream a file using the RTP protocol, you will need to run VLC with the following parameters:
However, it would be foolish to raise for each file its own process with a separate port, regardless of the presence of users who want to view it.
So back to the RTSP protocol and video playback on demand (Vidoe On Demand). In order to use VLC as an RTSP server to play VOD, you must first run VLC , specifying the attributes of the RTSP server and the Telnet interface:
After that, as the server is started, you need to configure it. It is most convenient to do this with the help of telnet 'a, since such an approach makes it possible to adjust on the fly:
To play a video (including on the Android platform), you need to request it at the URL rtsp: // localhost: 5554 / pornofilm .
Among the shortcomings, we can note the fact that HTTP is often open on all firewalls and proxies ... with RTSP in the case of Deny, Allow policies, everything is different.
In addition, when using the RTSP server, to add / delete files on the server, you will have to update its configuration (list of vods). Yes, there is telnet for this, but it is still more difficult than just uploading or deleting files from web server directories.
Reproduction using this technology is supported by the Android platform natively. For example, using the same standard class MediaPlayer .
Many people believe that multicast does not work in Android. This is not quite true.
Firstly, in most cases it is simply disabled by default so as not to load the device resources with unnecessary work. You can simply turn it on .
Secondly, yes - on a rather impressive number of devices it is disabled in everything or it does not work correctly. On the Internet, so you can find a lot of tears and even some solutions.
However, as practice shows, everything can be played on multicast video on Android . In my case, the recently released VLC Beta for Android successfully coped with this task.
In addition, using VLC server you can always reduce the playback of multicast 'a to HLS :
or rtsp:
You can try your luck with playing multicast on your device by giving the player a URL like udp: //@192.168.20.1: 1234 .
If everything is clear with the video format ( H.264 BP / MP4 ), then the question is open with the distribution method. Each of them has its advantages and disadvantages.
First of all, I would remove the usual progressive streaming from consideration. Yes, it works always and everywhere, but the lack of rewinding and downloading the entire file is too much.
The next candidate for crash is live streaming . Its main disadvantage is native support in Android since version 3.0. And ignoring more than 80% of users with version 2.x is not an option. Although here you can look at a third-party player, or do your own implementation (I, alas, did not find any free practices to support HLS ).
And the last one I would have struck off the RTSP. Yes, this is a protocol designed specifically for video. Yes, its use is ideologically correct. But there are two points. Firstly - you need to constantly update the server configuration. Secondly, HTTP is open always and everywhere, which is not the case with RTSP / RTP .
Personally, I would stop at pseudo streaming . It allows you to rewind and not download the entire file completely. All we have to do is tweak the web server a bit.
Immediately, I’ll make a reservation that I am not an expert in this field, but just want to share my recently gained experience.
Imagine that you are faced with the task of implementing an Android application capable of playing many files uploaded by users to your server. Write your youtube, with blackjack and codecs. To do this, you will have to solve at least two tasks: video conversion to a format supported on Android , video playback from a remote source. Consider both of these tasks in more detail.
Video conversion
And so, before you play some video on our Android device, you need to transcode this video into a supported format. The Android documentation clearly identifies a list of these formats.
')
In order to transcode files uploaded by users to your service, or to record stream from a TV tuner, you will need the help of a special utility ffmpeg , which is the de facto standard in the industry. Detailed instructions for its installation can be found on the site of the same name project .
The most common (in my opinion) video storage method is the MP4 container using the H.264 AVC codec . We will, in fact, consider them.
First of all, note that Android does not support all the features of the H.264 codec, but only a specific set — a profile called Baseline Profile (BP). For example, BP does not include such useful H.264 features as CABAC or B-Frames .
For us, this means that if we use these features when encoding a video, then Android will not have to lose this video. Although it may, if your phone is powerful enough and the vendor has taken care of installing and supporting additional codecs. For example, the video in the Main Profile is played without any problems on the Samsung Galaxy SII . On the phones of the usual class (for example, Samsung Galaxy Ace ) we will receive a message about the impossibility of playing the video and an error with the codec of the wrong codec in logcat .
But let's move from theory to practice. In order to compress the video, you must run the following command:
ffmpeg -i in.3gp -f mp4
-vcodec libx264 -vprofile baseline -b: v 1500K
-acodec libfaac -b: a 128k -ar 44100 -ac 2
-y out.mp4
Let us consider in more detail each of the parameters:
- -i src input (recoded) file;
- -f mp4 video container used;
- -vcodec libx264 video codec used;
- -vprofile baseline profile used;
- -b: v 1500K bitrate;
- -acodec libfaac used audio codec;
- -b: a 128k audio bitrate;
- -ar 44100 sound frequency;
- -ac 2 number of audio streams;
- -y flag for overwriting the output file;
It is also worth noting that you can do without specifying the profile, and explicitly enable / disable the necessary options for the H.264 codec through the -x264opts parameter, so that they would satisfy the conditions of BP . But this is a lesson for lovers.
Video distribution
The easiest way to play video from a remote server is to download it to temporary storage and play it locally. However, I think everyone understands that in view of the size of modern video recordings - this is not an option.
How to be? The Android platform offers us native support for the following technologies / protocols:
- HTTP / HTTPS progressive streaming;
- HTTP / HTTPS live streaming;
- RTSP (RTP, SDP);
Consider them in order.
Progressive streaming
The easiest way to distribute video using a conventional web server, which essentially amounts to downloading a previously prepared file using the HTTP (S) protocol. The whole point in this case is that the file does not start playing at the end of the download, but as soon as enough data is downloaded (some buffer is filled).
Here it is worth clarifying that when using the MP4 container, it is necessary to form a file so that metadata about the video stream ( moov atoms ) is located at the beginning of the file (after the ftyp atom), before the video data ( mdat atoms ). This can be done by processing the file with the qt-faststart utility:
qt-faststart output.mp4 result.mp4
The main problem of progressive streaming is the impossibility of rewinding video to a non-downloaded moment, the presence of a sufficient amount of free space on the device and the need to support a large number of "fat" clients downloading videos on a web server.
Reproduction using this technology is supported by the Android platform natively. You can easily play a remote file using the standard MediaPlayer class (apart from the communication channel, power of the device and availability of free space).
Pseudo streaming
This technology is a logical extension of progressive streaming 'a and allows you to solve one of its main problems - rewinding to a fragment that has not yet been downloaded. Applicable for MP4 / FLV containers with H.264 / AAC codec.
The only difference from progressive streaming 'a in this case is the fact that you need a special web server that, given the timestamp in the GET request, will give you the fragment of the video file you need. An example of such a web server can naturally serve as an Orthodox NGINX with its ngx_http_mp4_module .
I could not find any official information regarding the support of this standard in Android . However, empirically, it was found that it is present on at least HTC Desire and Samsung Galaxy SII devices. However, I want to note that even if there is no native support on your device, you can always use third-party players like MX Player , who independently implement the logic of downloading and playing video fragments with the required timestamp, which allows you to organize rewind.
Live streaming
Quite a non-standard data transfer protocol from Apple . Its essence boils down to the fact that the file being distributed is “sawn” into many small parts, joined by a special M3U8 format playlist . Data transfer occurs via the HTTP (S) protocol.
There are no problems with rewind and free space on the device in this case. Moreover, under certain conditions, you have the opportunity to choose the quality of the video being played.
However, there are problems. To “cut” a file and create a playlist will require processor resources, time and space on the server. To broadcast a file to the network, as in the previous examples, you will need an HTTP server (without any additional modules).
You can use VLC to cut a video file:
vlc -I dummy /path/to/pornofilm.mpg vlc: // quit --sout '#transcode {width = 320, height = 240, fps = 25, vcodec = h264, vb = 256, venc = x264 {aud, profile = baseline, level = 30, keyint = 30, ref = 1}, acodec = mp3, ab = 96}: std {access = livehttp {seglen = 10, delsegs = false, numsegs = 0, index = / path / to /web/server/root/pornofilm.m3u8,index-url=http://localhost/pornofilm/stream-########.ts }mmux{useuse- /path/to/web/server/root/pornofilm/stream-#######.ts} '
You can play this file at the URL localhost / pornofilm.m3u8 .
Support for HTTP Live Streaming at the native level in Android has been present since version 3.0. With the help of third-party players ( DicePlayer , MX Player ), judging by the wiki, you can achieve support from version 2.2.
Real Time Streaming Protocol (RTSP)
A state-level application protocol designed specifically for video transmission. The command format is very similar to HTTP . The teams themselves resemble buttons on a conventional cassette recorder: PLAY, PAUSE, RECORD, etc.
Unlike HTTP Live Streaming, RTSP does not require splitting files into smaller parts and creating playlists. The necessary parts of the file will be generated and given to the client on the fly. VLC can be used as an RTSP server.
It is worth noting that the RTSP protocol itself does not determine the method of data transmission, but delegates this to other protocols. For example, RTP . To stream a file using the RTP protocol, you will need to run VLC with the following parameters:
vlc -vvv /path/to/pornofilm.mp4 --sout '#rtp {dst = localhost, port = 1234, sdp = rtsp: // localhost: 8080 / pornofilm.sdp}'
However, it would be foolish to raise for each file its own process with a separate port, regardless of the presence of users who want to view it.
So back to the RTSP protocol and video playback on demand (Vidoe On Demand). In order to use VLC as an RTSP server to play VOD, you must first run VLC , specifying the attributes of the RTSP server and the Telnet interface:
vlc -vvv -I telnet --telnet-password 123 --rtsp-host 127.0.0.1 --rtsp-port 5554
After that, as the server is started, you need to configure it. It is most convenient to do this with the help of telnet 'a, since such an approach makes it possible to adjust on the fly:
new porno vod enabled
setup porno input /path/to/pornofilm.mpg
To play a video (including on the Android platform), you need to request it at the URL rtsp: // localhost: 5554 / pornofilm .
Among the shortcomings, we can note the fact that HTTP is often open on all firewalls and proxies ... with RTSP in the case of Deny, Allow policies, everything is different.
In addition, when using the RTSP server, to add / delete files on the server, you will have to update its configuration (list of vods). Yes, there is telnet for this, but it is still more difficult than just uploading or deleting files from web server directories.
Reproduction using this technology is supported by the Android platform natively. For example, using the same standard class MediaPlayer .
Multicast
Many people believe that multicast does not work in Android. This is not quite true.
Firstly, in most cases it is simply disabled by default so as not to load the device resources with unnecessary work. You can simply turn it on .
Secondly, yes - on a rather impressive number of devices it is disabled in everything or it does not work correctly. On the Internet, so you can find a lot of tears and even some solutions.
However, as practice shows, everything can be played on multicast video on Android . In my case, the recently released VLC Beta for Android successfully coped with this task.
In addition, using VLC server you can always reduce the playback of multicast 'a to HLS :
vlc -I dummy udp: //@192.168.20.1: 1234 vlc: // quit --sout '#transcode {width = 320, height = 240, fps = 25, vcodec = h264, vb = 256, venc = x264 { aud, profile = baseline, level = 30, keyint = 30, ref = 1}, acodec = mp3, ab = 96}: std {access = livehttp {seglen = 10, delsegs = false, numsegs = 0, index = / path /to/web/server/root/multicast-porno.m3u8,index-url=http://localhost/multicast-porno/stream-#######.ts (mu) -frames}, dst = / path / to / web / server / root / multicast-porno / stream - ########. ts} '
or rtsp:
new multicast-porno vod enabled
setup multicast-porno input udp: //@192.168.20.1: 1234
You can try your luck with playing multicast on your device by giving the player a URL like udp: //@192.168.20.1: 1234 .
What to choose
If everything is clear with the video format ( H.264 BP / MP4 ), then the question is open with the distribution method. Each of them has its advantages and disadvantages.
First of all, I would remove the usual progressive streaming from consideration. Yes, it works always and everywhere, but the lack of rewinding and downloading the entire file is too much.
The next candidate for crash is live streaming . Its main disadvantage is native support in Android since version 3.0. And ignoring more than 80% of users with version 2.x is not an option. Although here you can look at a third-party player, or do your own implementation (I, alas, did not find any free practices to support HLS ).
And the last one I would have struck off the RTSP. Yes, this is a protocol designed specifically for video. Yes, its use is ideologically correct. But there are two points. Firstly - you need to constantly update the server configuration. Secondly, HTTP is open always and everywhere, which is not the case with RTSP / RTP .
Personally, I would stop at pseudo streaming . It allows you to rewind and not download the entire file completely. All we have to do is tweak the web server a bit.
Source: https://habr.com/ru/post/148754/
All Articles