Introducing GStreamer: Introduction

Disclaimer

This article is aimed at novice programmers who are not familiar with GStreamer and want to get to know him. Experienced developers are unlikely to find for themselves something new in this article.

Preamble

Many people have probably heard that there is such a thing as GStreamer, or have seen Ubuntu and similar distros suggest installing various packages, the names of which contain “gstreamer” when you first try to play mp3 or some other file with a “non-free” media format . So, it will be about this library.

Introduction

GStreamer is a powerful framework for building multimedia applications, which adopted the ideas of the “video pipeline” from the Oregon Graduate Institude, and also took something from DirectShow. This framework allows you to create applications of various levels of complexity, ranging from a simple console player (you can play any file directly from the terminal without writing any code), ending with full-fledged audio / video players, multimedia editors and other applications.

GStreamer has a plug-in architecture, and as standard it has a very large set of plug-ins that can solve 99% of the needs of all multimedia software developers.
')

Architecture

GStreamer has several major components:

Items
Pads
Bin and pipeline containers

And now more:

Items

Virtually everything in GStreamer is an element. Everything from normal stream sources (filesrc, alsasrc, etc.) to stream handlers (demultiplexers, decoders, filters, etc.) and ending with final output devices (alsasink, fakesink, filesink, etc.) .).

Pads

A pad is a kind of connection point for an element to another element, if more simply, it is the input and output elements of an element. They are usually referred to as "sink" - input and "src" - output.
Items always have at least one pad. For example, filesrc - an element for reading data from the file system - has only one pad called “src”, since it has no input, but can only turn the stream from the file system into an internal representation that other elements will already work with. Similarly, the alsasink element, it has one pad called "sink", since it can only receive the internal stream and output it to the sound card via alsa. Elements from the category of "filters" (those that somehow transform the stream) have two or more connection points. For example, the volume element has a pad named “sink” to which the stream flows, the volume inside this element is transformed (volume changes), and through the pad called “src” it continues its way. There are also elements where there can be several inputs and outputs.

Containers

Inside containers, the elements carry out their life cycle. The container controls the distribution of messages from item to item, manages the status of items. Containers are divided into two types:

Bin
Pipeline

Pipeline is a top-level container, it manages the synchronization of items, sends statuses. For example, if the pipeline sets the status to PAUSED, this status will be automatically sent to all elements that are inside it. Pipeline is a Bin implementation.
Bin is a simple container that manages the distribution of messages from element to element that are inside it. Bin is usually used to create a group of elements that must perform an action. For example, decodebin is an element for decoding a stream that automatically selects the necessary elements for processing a stream depending on the type of data (vorbisdec, theoradec, etc.) removing additional work from the developer.
There are also complete self-contained containers, such as playbin. Playbin, in fact, is a full-fledged player, which contains in its composition all the necessary elements for playing audio and video, but as you understand, there is no flexibility ...

How does it work

Consider the approximate scheme of a primitive player. The simplest scheme of the player should have something like this:

Consider what happens here. The filesrc element reads a file from the file system and sends the stream to the decodebin container, which in turn decodes the stream to the internal representation, and subsequently sends the stream to the alsasink element, which also sends the audio stream to the sound card. To test this scheme, just run the command.

gst-launch-1.0 filesrc location=/path/to/file.ogg ! decodebin ! alsasink

And if you heard your melody, then everything is fine.

Notes

The gst-launch-1.0 utility comes in the gstreamer1.0-tools package.
In the case of using OSS / Jack / etc., this scheme may not work. Therefore, alsasink can be replaced with an autoaudiosink element, which will select the desired element for audio output.

What's next?

In the future, I plan to write a series of articles in which various elements and their capabilities will be considered, as well as examples of the code of actual use of the GStreamer capabilities will be given.