Introducing GStreamer: Data Sources
Hello, habrayuzer interested in the GStreamer framework! Today we will talk about the sources of media data (sources), and thus continue the course of articles on GStreamer.
Previous article: Introduction to GStreamer: Introduction .
Data Sources is a class of GStreamer plug-ins that allows you to read media from various sources, such as the file system or audio inputs of a sound card. They also allow you to receive a media stream from various streaming servers, such as HTTP (ICECast, ShoutCast), RTSP, RTMP, TCP and UDP. And there is also an opportunity to read data from DVB cards, CDDA disks (popularly known simply as “compact disks”), and many more, with the help of various plug-ins, which are about 30 at the moment.
Note: as mentioned in the last article, data sources have only one pad called src, since it can be connected to another element, but nothing can be connected to it.
In this article we will examine some (perhaps the most demanded) data sources, write some Python code and learn a lot of new things.
The gst-launch-1.0 utility allows you to launch the GStreamer pipeline without writing a single line of code. I will use it to run small examples, and what we need to know today is that the start of the pipeline has something like this:
and the description of the pipeline, in turn, is divided by the description of the elements of the form
It seems that everything in this scheme is clear, there is an element of type element1 with properties property1 and property2 that have value1 and value2, respectively, and there is an element of type element2. The “!” Symbol indicates that the element1 output must be connected to the element2 input.
At this gst-launch capabilities are not limited, just in today's article they will not be useful to us and will be considered in the future.
')
filesrc - in my opinion, is the most frequently used source of multimedia data. As its name implies, it is an element for reading data from files. In the last article, we looked at the scheme of a primitive player, in which filesrc appeared as a source of data.
The filesrc element has several parameters. Consider some of them.
The location property must contain the path to the file from which to read.
Example: /foo/bar.mp3
This parameter sets the size of the read buffer. The value must be described by an unsigned number.
The number of blocks after reading which will be sent the EOS message.
The blocksize and num-buffers properties can be omitted. In this case, the default values ​​will be used. And these properties are available for all data sources, and will not be considered further.
An example of using the source filesrc take from the previous article:
The souphttpsrc element provides the ability to read data over the HTTP (HTTPS) protocol. Reading can be done both from a regular file, accessible via the HTTP protocol, and from the ICECast / ShoutCast server. Also, this element allows reading data on top of the ICY and ICYX protocols. According to the protocols, I could not find any information.
souphttpsrc has a bit more options than filesrc, consider the most important.
This parameter is similar to the same parameter of the filesrc element, except that only the URI http, https, icy or icyx URI schemes are supported.
The name speaks for itself - this parameter sets which one to use the User-Agent when sending HTTP requests.
Sets an automatic transition in case the server responds that the resource has been moved (statuses 301 and 302).
These properties set the username and password in case the resource requires authorization (HTTP Basic Authentication).
These properties set the proxy server address, username and password for authorization (if necessary). HTTP and HTTPS proxies are supported.
Using the source souphttpsrc is the same as using the source filesrc:
The GStreamer plugin suite has an interesting giosrc element. This is almost the same as filesrc, it only works through the GIO library, and allows you to receive data from various sources with which the GIO can work. There are 22 such sources, starting from file, ending with mtp. Also, this element supports the protocols ftp, sftp, smb and webdav.
This element has parameters similar to the filesrc element. An example of using this source is similar to the previous ones:
As the name implies, this element allows you to read data from the RTSP server. We will not consider the properties of this element in detail; I will only say that it has a location property for specifying the address of the data source, proxy, proxy-id and proxy-pw for specifying proxy settings, and a dozen properties specific to the RTSP protocol, the consideration of which is not included in our plans.
But, rtspsrc has one feature, it has several pad-s, and they have names that match the pattern source_% u. This is due to the fact that the server can broadcast several streams simultaneously, for example, audio and video can be scattered over two streams.
Note:
I am not familiar with the RTSP protocol, so I could be mistaken in the assumption above.
The example of using rtspsrc is similar to the previous elements:
The essence of the multifilesrc element is the same as that of the filesrc element - read files from disk, but there is one difference - multifilesrc reads files sequentially (1.mp3, 2.mp3 ...), while filesrc can read only one file . This element has one drawback - file names should differ only by number, for example, file-1.mp3, file-2.mp3, and so on. But such a flaw can be written off as a feature, since with the help of this “flaw” you can easily make a video from photos from images 0000.png-9999.png, or a melody from samples or a mix from tracks.
The multifilesrc element has a few parameters, and oddly enough - they are all useful. Consider them in more detail.
The file path can (and should) include the% d control sequence , which will be automatically replaced by the sprintf () function with a number equal to the current index.
Example: /foo/bar-%04d.mp3
These properties contain the current index, the starting index, and the final index. These properties contain an integer.
This property may contain a boolean value, and if it is true, reading of files will be looped. Roughly speaking, this is an analogue of the “Repeat All” function.
An example of using the multifilesrc element that reads files with names starting from bar-1.mp3 with the loop option enabled:
In my opinion, the most useless element. Reads data from a file descriptor. It can be used when lazy to describe filesrc for gst-launch.
This element has one parameter:
This parameter must contain a number equal to the open file descriptor. The default is zero.
Usage example:
As the name suggests, this element does nothing — it is simply a dummy source of empty buffers.
These elements are designed to receive a stream from the microphone input of a sound card. From the names it follows that they are used to work with various sound subsystems, such as Alsa, PulseAudio, OSS, Jack. The autoaudiosrc element is a “wrapper” over the other elements, and it simplifies the development, taking all the work on choosing a sound source on your shoulders.
Properties of these elements are not very many. The main property is device , it is present in the first three elements and must contain the name of the device (hw: 0, hw: 1 ...). The remaining properties are used specific to each sound subsystem, and we will not consider them.
This item is designed to open a video stream through the Video4Linux2 interface. The V4L2 interface supports most webcams, TV tuners, DVB cards and video capture cards.
Consider the most important parameters:
Specifies the name of the device from which to capture. You can find the list of available devices using
The value of this parameter should be the full path to the device.
Example: device = / dev / video0
These parameters are responsible for the color balance, saturation, contrast and brightness, respectively. Of course, if the video capture device does not support these parameters, then the image will be what the device gives it. The values ​​of these properties should be in the range from -2147483648 to 2147483647.
Example of using the element
This element is similar to fakesrc, but differs in that it generates a real audio signal. It can be used both for testing complex pipelines running via gst-launch, for testing the pipeline's performance in a written application, and also it can be used as a reference generator for measuring the frequency response and distortions introduced by various filters.
Consider the most interesting parameters of this element.
This parameter sets the waveform. There are 13 values ​​available: sine, square, saw, triangle, silence, white-noise, pink-noise, sine-table, ticks, gaussian-noise, red-noise, blue-noise, violet-noise .
It is not difficult to guess that this parameter sets the signal level (volume). Valid values ​​range from 0 to 1.
This property sets the frequency of the signal, and the value can range from 0 to 20000.
An example of using the audiotestsrc element:
This element, like the audiotestsrc, is used to generate test data, namely, the video stream. It can be used both to check the performance of the entire pipeline, or to check for the presence of distortions that are introduced by filters.
This element has a lot of parameters, we will consider only one, the most interesting in my opinion.
This parameter sets the video signal pattern. Valid value can be one of 21. Full list: smpte, snow, black, white, red, green, blue, checkers-1, checkers-2, checkers-4, checkers-8, circular, blink, smpte75, zone-plate , gamut, chroma-zone-plate, solid-color, ball, smpte100, bar .
I will not stop at all values, I will only say that the values ​​of smpte, smpte75 and smpte100 are SMPTE tuning tables that are used in the settings of the TV / video equipment. The checkers- * templates are a grid of NxN pixels, where N equals the number in the template name. The remaining values ​​set the pattern with a monotonous image, such as fill color.
Usage example:
Above, we covered 15 main data sources, which is about half of all available plug-ins from the source category. The remaining plugins were not considered for two reasons:
When considering plug-ins, many parameters were omitted, since basically all of them are used for “tweaking”, and we, as you remember, are getting acquainted with GStreamer. Alternatively, these parameters are read only. To get a list of all available properties, as well as valid values ​​for them, it suffices to run
and in the “Element Properties” section you can see all properties (including inherited).
Today we will look at one simple example of using GStreamer. In the example, I will use Python 2.7, GStreamer 1.0 and GObject Introspection.
Immediately make a reservation, the code may not be perfect, and maybe even the so-called “govnokod”, but its only purpose is to show how to use GStreamer, no more.
This example shows how the elements are linked, how messages are processed and the properties of the elements change.
Examples of running this script:
In case everything goes as it should, you will hear the track you need and see a list of ID3 tags, if any.
An example is available on GitHub .
Today we reviewed about half of the available data sources that are available out of the box, reviewed examples of the launch of these elements, and also reviewed an example of a simple player based on GStreamer. In the following articles we will look at the rest of the plug-in classes, consider even more examples of using GStreamer, and of course, write more code.
See you in a week!
UPD 1: Added elements audiotestsrc, videotestsrc.
Previous article | Next article
Previous article: Introduction to GStreamer: Introduction .
Introduction
Data Sources is a class of GStreamer plug-ins that allows you to read media from various sources, such as the file system or audio inputs of a sound card. They also allow you to receive a media stream from various streaming servers, such as HTTP (ICECast, ShoutCast), RTSP, RTMP, TCP and UDP. And there is also an opportunity to read data from DVB cards, CDDA disks (popularly known simply as “compact disks”), and many more, with the help of various plug-ins, which are about 30 at the moment.
Note: as mentioned in the last article, data sources have only one pad called src, since it can be connected to another element, but nothing can be connected to it.
In this article we will examine some (perhaps the most demanded) data sources, write some Python code and learn a lot of new things.
Go
0. gst-launch-1.0
The gst-launch-1.0 utility allows you to launch the GStreamer pipeline without writing a single line of code. I will use it to run small examples, and what we need to know today is that the start of the pipeline has something like this:
gst-launch-1.0 -pipeline and the description of the pipeline, in turn, is divided by the description of the elements of the form
element1 property1=value1 property2=value2 ! element2 It seems that everything in this scheme is clear, there is an element of type element1 with properties property1 and property2 that have value1 and value2, respectively, and there is an element of type element2. The “!” Symbol indicates that the element1 output must be connected to the element2 input.
At this gst-launch capabilities are not limited, just in today's article they will not be useful to us and will be considered in the future.
')
1. filesrc
filesrc - in my opinion, is the most frequently used source of multimedia data. As its name implies, it is an element for reading data from files. In the last article, we looked at the scheme of a primitive player, in which filesrc appeared as a source of data.
The filesrc element has several parameters. Consider some of them.
location
The location property must contain the path to the file from which to read.
Example: /foo/bar.mp3
blocksize
This parameter sets the size of the read buffer. The value must be described by an unsigned number.
num-buffers
The number of blocks after reading which will be sent the EOS message.
The blocksize and num-buffers properties can be omitted. In this case, the default values ​​will be used. And these properties are available for all data sources, and will not be considered further.
An example of using the source filesrc take from the previous article:
gst-launch-1.0 filesrc location=/foo/bar.mp3 ! decodebin ! autoaudiosink 2. souphttpsrc
The souphttpsrc element provides the ability to read data over the HTTP (HTTPS) protocol. Reading can be done both from a regular file, accessible via the HTTP protocol, and from the ICECast / ShoutCast server. Also, this element allows reading data on top of the ICY and ICYX protocols. According to the protocols, I could not find any information.
souphttpsrc has a bit more options than filesrc, consider the most important.
location
This parameter is similar to the same parameter of the filesrc element, except that only the URI http, https, icy or icyx URI schemes are supported.
user-agent
The name speaks for itself - this parameter sets which one to use the User-Agent when sending HTTP requests.
automatic-redirect
Sets an automatic transition in case the server responds that the resource has been moved (statuses 301 and 302).
user-id and user-pw
These properties set the username and password in case the resource requires authorization (HTTP Basic Authentication).
proxy, proxy-id and proxy-pw
These properties set the proxy server address, username and password for authorization (if necessary). HTTP and HTTPS proxies are supported.
Using the source souphttpsrc is the same as using the source filesrc:
gst-launch-1.0 souphttpsrc location=http://radio.localhost:1234 ! decodebin ! autoaudiosink 3. giosrc
The GStreamer plugin suite has an interesting giosrc element. This is almost the same as filesrc, it only works through the GIO library, and allows you to receive data from various sources with which the GIO can work. There are 22 such sources, starting from file, ending with mtp. Also, this element supports the protocols ftp, sftp, smb and webdav.
This element has parameters similar to the filesrc element. An example of using this source is similar to the previous ones:
gst-launch-1.0 giosrc location=ftp://ftp.localhost/foo/bar.mp3 ! decodebin ! autoaudiosink 4. rtspsrc
As the name implies, this element allows you to read data from the RTSP server. We will not consider the properties of this element in detail; I will only say that it has a location property for specifying the address of the data source, proxy, proxy-id and proxy-pw for specifying proxy settings, and a dozen properties specific to the RTSP protocol, the consideration of which is not included in our plans.
But, rtspsrc has one feature, it has several pad-s, and they have names that match the pattern source_% u. This is due to the fact that the server can broadcast several streams simultaneously, for example, audio and video can be scattered over two streams.
Note:
I am not familiar with the RTSP protocol, so I could be mistaken in the assumption above.
The example of using rtspsrc is similar to the previous elements:
gst-launch-1.0 rtspsrc location=rtsp://rtsp.localhost/movie.mp4 ! decodebin ! autoaudiosink 5. multifilesrc
The essence of the multifilesrc element is the same as that of the filesrc element - read files from disk, but there is one difference - multifilesrc reads files sequentially (1.mp3, 2.mp3 ...), while filesrc can read only one file . This element has one drawback - file names should differ only by number, for example, file-1.mp3, file-2.mp3, and so on. But such a flaw can be written off as a feature, since with the help of this “flaw” you can easily make a video from photos from images 0000.png-9999.png, or a melody from samples or a mix from tracks.
The multifilesrc element has a few parameters, and oddly enough - they are all useful. Consider them in more detail.
location
The file path can (and should) include the% d control sequence , which will be automatically replaced by the sprintf () function with a number equal to the current index.
Example: /foo/bar-%04d.mp3
index, start-index and stop-index
These properties contain the current index, the starting index, and the final index. These properties contain an integer.
loop
This property may contain a boolean value, and if it is true, reading of files will be looped. Roughly speaking, this is an analogue of the “Repeat All” function.
An example of using the multifilesrc element that reads files with names starting from bar-1.mp3 with the loop option enabled:
gst-launch-1.0 multifilesrc location="/foo/bar-%d.mp3" loop=true start-index=1 ! decodebin ! autoaudiosink 6. fdsrc
In my opinion, the most useless element. Reads data from a file descriptor. It can be used when lazy to describe filesrc for gst-launch.
This element has one parameter:
fd
This parameter must contain a number equal to the open file descriptor. The default is zero.
Usage example:
cat /foo/bar.mp3 | gst-launch-1.0 fdsrc ! decodebin ! autoaudiosink 7. fakesrc
As the name suggests, this element does nothing — it is simply a dummy source of empty buffers.
8. alsasrc, pulsesrc, osssrc, jackaudiosrc, autoaudiosrc
These elements are designed to receive a stream from the microphone input of a sound card. From the names it follows that they are used to work with various sound subsystems, such as Alsa, PulseAudio, OSS, Jack. The autoaudiosrc element is a “wrapper” over the other elements, and it simplifies the development, taking all the work on choosing a sound source on your shoulders.
Properties of these elements are not very many. The main property is device , it is present in the first three elements and must contain the name of the device (hw: 0, hw: 1 ...). The remaining properties are used specific to each sound subsystem, and we will not consider them.
9. v4l2src
This item is designed to open a video stream through the Video4Linux2 interface. The V4L2 interface supports most webcams, TV tuners, DVB cards and video capture cards.
Consider the most important parameters:
device
Specifies the name of the device from which to capture. You can find the list of available devices using
ls -l /dev/v4l/by-id The value of this parameter should be the full path to the device.
Example: device = / dev / video0
hue, saturation, contrast. brightness
These parameters are responsible for the color balance, saturation, contrast and brightness, respectively. Of course, if the video capture device does not support these parameters, then the image will be what the device gives it. The values ​​of these properties should be in the range from -2147483648 to 2147483647.
Example of using the element
gst-launch-1.0 v4l2src ! xvimagesink 10. audiotestsrc
This element is similar to fakesrc, but differs in that it generates a real audio signal. It can be used both for testing complex pipelines running via gst-launch, for testing the pipeline's performance in a written application, and also it can be used as a reference generator for measuring the frequency response and distortions introduced by various filters.
Consider the most interesting parameters of this element.
wave
This parameter sets the waveform. There are 13 values ​​available: sine, square, saw, triangle, silence, white-noise, pink-noise, sine-table, ticks, gaussian-noise, red-noise, blue-noise, violet-noise .
volume
It is not difficult to guess that this parameter sets the signal level (volume). Valid values ​​range from 0 to 1.
freq
This property sets the frequency of the signal, and the value can range from 0 to 20000.
An example of using the audiotestsrc element:
gst-launch-1.0 audiotestsrc wave=saw freq=205 ! autoaudiosink 11. videotestsrc
This element, like the audiotestsrc, is used to generate test data, namely, the video stream. It can be used both to check the performance of the entire pipeline, or to check for the presence of distortions that are introduced by filters.
This element has a lot of parameters, we will consider only one, the most interesting in my opinion.
pattern
This parameter sets the video signal pattern. Valid value can be one of 21. Full list: smpte, snow, black, white, red, green, blue, checkers-1, checkers-2, checkers-4, checkers-8, circular, blink, smpte75, zone-plate , gamut, chroma-zone-plate, solid-color, ball, smpte100, bar .
I will not stop at all values, I will only say that the values ​​of smpte, smpte75 and smpte100 are SMPTE tuning tables that are used in the settings of the TV / video equipment. The checkers- * templates are a grid of NxN pixels, where N equals the number in the template name. The remaining values ​​set the pattern with a monotonous image, such as fill color.
Usage example:
gst-launch-1.0 videotestsrc pattern=smpte ! xvimagesink Explanations
Above, we covered 15 main data sources, which is about half of all available plug-ins from the source category. The remaining plugins were not considered for two reasons:
- They are highly specialized;
- I do not have the necessary "glands" to test one or another plug-in.
When considering plug-ins, many parameters were omitted, since basically all of them are used for “tweaking”, and we, as you remember, are getting acquainted with GStreamer. Alternatively, these parameters are read only. To get a list of all available properties, as well as valid values ​​for them, it suffices to run
gst-inspect-1.0 <_> , for example: gst-inspect-1.0 filesrc and in the “Element Properties” section you can see all properties (including inherited).
Examples
Today we will look at one simple example of using GStreamer. In the example, I will use Python 2.7, GStreamer 1.0 and GObject Introspection.
Immediately make a reservation, the code may not be perfect, and maybe even the so-called “govnokod”, but its only purpose is to show how to use GStreamer, no more.
Simple player
This example shows how the elements are linked, how messages are processed and the properties of the elements change.
Player Code
#env python2 # coding=utf-8 import gi gi.require_version("Gst", "1.0") gi.require_version("Gtk", "3.0") from gi.repository import Gst from gi.repository import Gtk from gi.repository import GObject import os import signal import argparse Gst.init("") signal.signal(signal.SIGINT, signal.SIG_DFL) GObject.threads_init() def parse_args(): parser = argparse.ArgumentParser(prog='example1.py') parser.add_argument('--volume', help=' (0-100)', type=int, default=100) parser.add_argument('location') args = parser.parse_args() return args class Player(): def __init__(self, args): self.pipeline = self.create_pipeline(args) ## ## message_bus = self.pipeline.get_bus() message_bus.add_signal_watch() message_bus.connect('message', self.message_handler) ## self.pipeline.get_by_name('volume').set_property('volume', args.volume / 100.) def create_source(self, location): """create_source(str) -> Gst.Element""" if not args.location.startswith('http') and not os.path.exists(args.location): raise IOError("File %s doesn't exists" % args.location) if location.startswith('http'): source = Gst.ElementFactory.make('souphttpsrc', 'source') else: source = Gst.ElementFactory.make('filesrc', 'source') source.set_property('location', location) return source def create_pipeline(self, args): """create_pipeline() -> Gst.Pipeline""" pipeline = Gst.Pipeline() ## source = self.create_source(args.location) decodebin = Gst.ElementFactory.make('decodebin', 'decodebin') audioconvert = Gst.ElementFactory.make('audioconvert', 'audioconvert') volume = Gst.ElementFactory.make('volume', 'volume') audiosink = Gst.ElementFactory.make('autoaudiosink', 'autoaudiosink') ## decodebin pad', ## def on_pad_added(decodebin, pad): pad.link(audioconvert.get_static_pad('sink')) decodebin.connect('pad-added', on_pad_added) ## pipeline [pipeline.add(k) for k in [source, decodebin, audioconvert, volume, audiosink]] ## : ## *src* -> (decodebin + audioconvert) -> volume -> autoaudiosink source.link(decodebin) audioconvert.link(volume) volume.link(audiosink) return pipeline def play(self): self.pipeline.set_state(Gst.State.PLAYING) def message_handler(self, bus, message): """ """ struct = message.get_structure() if message.type == Gst.MessageType.EOS: print(' .') Gtk.main_quit() elif message.type == Gst.MessageType.TAG and message.parse_tag() and struct.has_field('taglist'): print('GStreamer -') taglist = struct.get_value('taglist') for x in range(taglist.n_tags()): name = taglist.nth_tag_name(x) print(' %s: %s' % (name, taglist.get_string(name)[1])) else: pass if __name__ == "__main__": args = parse_args() player = Player(args) player.play() Gtk.main() Examples of running this script:
python2 ./example1.py /foo/bar.mp3 python2 ./example1.py --volume 50 /foo/bar.mp3 python2 ./example1.py http://myradio.localhost:5678/foo.mp3 In case everything goes as it should, you will hear the track you need and see a list of ID3 tags, if any.
An example is available on GitHub .
Well that's all
Today we reviewed about half of the available data sources that are available out of the box, reviewed examples of the launch of these elements, and also reviewed an example of a simple player based on GStreamer. In the following articles we will look at the rest of the plug-in classes, consider even more examples of using GStreamer, and of course, write more code.
See you in a week!
Literature
- GStreamer Application Development Manual
- GStreamer Core Plugins Reference Manual
- GStreamer Base Plugins Reference Manual
- GStreamer Good Plugins Reference Manual
- Video4Linux
- GIO Reference Manual
- The ID3v2 / Shoutcast standart
- SHOUTCast
UPD 1: Added elements audiotestsrc, videotestsrc.
Previous article | Next article
Source: https://habr.com/ru/post/179167/
All Articles