Interview with Edward Hervey about the video editor PiTiVi
This is the fourth interview about opensource multimedia. Previous conversations related to projects Jokosher , Totem and Empathy ( translation ). We will talk to Edward Hervey (Edward Hervey), accompanying the project PiTiVI . Edward will talk about the current state of the video editor and plans for the future.
So, tell us a little about yourself first and open projects in which you participate?
My name is Edward Hervey, I am half French, half Scots living in Barcelona, and I am one of the founders of Collabora Multimedia . I've been using Linux and free software since about 1995. But I started making my own contribution in 2003, when I finally had enough programming knowledge to start working on PiTiVi. For several years now I have been working on the GStreamer multimedia framework to raise video editing to a decent level. I also accompany over the years python bindings for GStreamer, GNonLin (the main PiTiVi dependencies are responsible for non-linear editing), as well as FFMpeg plug-ins for GStreamer.
Describe the goals of PiTiVI against the background of dominant video editing applications such as iMovie, Avid or Final Cut Pro?
')
PiTiVi has different goals, but I would point out the fundamental one - the desire to create a video editor without any restrictions (unlike analogs that have very specific limitations in use or support). Get rid of the limitations of formats, devices, filters, ... we can support them through the GStreamer framework. All other editors are tightly linked to their code, but we proudly carry the name of a free editor without binding to any patented library! This means that if a company makes its own GStreamer plugin for a specific format / codec / device ... you can freely use it in PiTiVi. This barrier has been overcome by choosing our design model.
We do not want to get a monolithic video editor, so we decided to rewrite PiTiVi in Python, and make the project modular. The user interface is not tied to the engine, in fact, you can use the engine separately. It is designed so that you can realize the chronological tape in different ways, it is more dependent on your knowledge and how you place the accents, which in turn depend on the use case in the production process.
The ultimate goal of PiTiVi is to provide basic editing tools without any restrictions. This, unfortunately, is worth a lot of effort: you need to make sure that all errors, problems or limitations are fixed at the proper level, instead of adding hack after hack. And it takes time ...
PiTiVI is written in Python. The first question that many people ask when they hear this is: “How does the use of Python affect the performance of multimedia applications, especially such resource-intensive ones as a video editor?”
It's simple :) What has the most CPU load when editing video? That's right, multimedia processing (be it viewing, applying effects, rendering, ...). All this is done exclusively with the help of GStreamer and its plug-ins (written in C, work in separate streams and are used in PiTiVi through Python binding). And what loads the rest of the processing power of the processor? User interface. This is done by GTK + / X / Cairo (written in C and used in PiTiVi through Python bindings).
So, the only thing left is the logic of connecting everything together and bringing in the concepts characteristic of non-linear video editing (NLE) systems. We want to create code as quickly and flexibly as possible, so a language like Python is very convenient. Besides, writing in Python is a pleasure :)
For the curious, I will cite an interesting fact: when playing in PiTiVi, the only part implemented in Python is the code responsible for ... displaying the current position. The same when rendering. Everything else is fully handled by GStreamer, its plugins and GTK +.
A lot of work has been done in GStreamer to support the MXF container format. Most people have probably never heard of such a format, what value does it have?
Since the advent of computer editing (also called non-linear editing), each program has used different formats / codecs for work. What was used by professionals and amateurs was also different, including for cameras, video recorders, capture devices. Add to this the fact that most of the formats have not been documented. We get different formats in the software, in the chamber, at the exit, ... a complete mess.
Some of the key players in the professional market, representatives of SMPTE (Society of Film and Television Engineers) decided to act together and wrote the specifications of the new container format, which will be fully documented, will allow you to store metadata, will be recorded directly to the camera, include streaming broadcasting and a quick search ... He uses the same structural elements as AAF (Extended Development Format), which is open and used by Avid, Final Cut Pro, and many other non-linear editing systems.
With the addition of MXF support in GStreamer, we can use the same formats in PiTiVi as professional NLE! The ability to work directly with common formats is the main factor that allows using PiTiVi in already existing production processes.
What multimedia formats does PiTiVI support?
All that is available through GStreamer plugins installed on your system, as well as some formats for quick video editing, accurate search, I-frame codecs, etc.
At what stage is PiTiVI now and what are your plans for the future?
Over the past 6 months, we have done a deep refactoring of the PiTiVi code, based on the analysis that I performed last summer, summarizing the responses for the last 5 years. In addition to adding the most easily implemented functions, we have already coped with multi-layer editing, creating thumbnails, cropping, ripple / roll modes, support for a common file format, and much more.
We are going to make the first preliminary release of the new pitivi in May. Gradually, new functions will be added to the interface, for example, transitions, effects, title overlays, slip / slide editing and support for a larger number of formats, which will allow, for example, importing projects from other editors.
For those who are not familiar with the terminology, what do you mean by multi-layer editing, sketching, cropping and ripple / roll modes?
Multi-layer editing: the ability to sort video not only by time, but also into several parallel layers.
Creating thumbnails: the resulting image is displayed for each source used in the Timeline scale, which will allow you to more easily navigate the chronological tape without having to search.
Crop: change the position of the beginning or end of the source.
Ripple editing: reduce or increase one of the parts, while the rest of the data adapts to the new size of the changed clip.
Roll editing: the same as ripple, except that the total time in the group of clips remains the same.
Who besides you participates in the project PiTiVI?
The two main developers apart from me are Brandon Lewis (Brandon Lewis), who started working with GSoC 2007 and is busy with a user interface. Alessandro Decina joined us in October, he helps me in the low-level part of PiTiVi and improves GStreamer. Both have been working in Collabora Multimedia since last autumn and are working on PiTiVi 100% of the time. Explanation for those who are thinking about business reasons for the involvement of so many resources in PiTiVi from Collabora Multimedia: firstly, we carry out consulting work for customers in the content creation industry, they want to use PiTiVI to perform specific tasks in their production processes, secondly , we have clients who want to use the PiTiVi infrastructure we created to configure automatic and video manipulations. For example, automatic framing, scaling and transcoding.
How will future changes to GStreamer affect PiTiVI features?
Although there are quite working functions in the current GStreamer core and plugins, we are working on improving a number of features to improve the editing interface.
One of the much needed functions is the implementation of GstIndex support over a wider range of plugins. This will allow you to scan files that are inconvenient for editing during import and create an index that allows you to quickly navigate inside these files. This will allow applications like PiTiVi to get more information about the file format, the location of key frames, thanks to which you will find out which section is more advantageous to skip and where to cut without loss.
Endless work to be done in GStreamer to transfer more and more libraries as plugins. People often mistakenly assume that GStreamer is a codec library, which is completely wrong, GStreamer is a framework that allows application developers not to worry about processing, and library developers not to worry about applications that will use them. We plan to transfer plugins like gavl and the vast majority of excellent AviSynth filters to GStreamer (which will be completed this summer as part of Google Summer of Code). All this, of course, improves the user experience of PiTiVi.
In addition, we are going to reduce the memory and CPU consumption for the GStreamer core and various plugins. We are looking at a more efficient use of hardware devices, such as video cards with VDPAU technology, VA-API and gst-plugins-gl plug-ins. This will allow GStreamer applications to direct processing to hardware devices, leaving more processor time for universal processing.
Jokosher and PiTiVI are multimedia applications that use GStreamer and are written in Python. The natural question is, do these projects collaborate?
Unfortunately, not as active as we would like. We help each other on all fronts (GStreamer, Python bindings, GNonLin plugins), but when it comes to the user interface and the main logic, we have different code usage options and different code bases. It would be great to have closer integration, it would allow you to use Jokosher to edit the audio part of a video clip, and leave video for PiTiVi.
What are the main technical problems you encountered while creating a video editor for Linux / Unix systems? Is Linux infrastructure ready for organizing such applications? And what do you think about the state of multimedia in Linux / Unix in general?
Open-source multimedia has gone * LONG * the way over the last 10 years. For those who do not remember ... we were forced to use different players depending on the format, the only way to capture video from digital cameras. But the problem is that everyone makes their own software in their own corner. The total lack of consistency between various multimedia applications, libraries and device support on Linux is, in my opinion, the reason why we did not become a suitable multimedia platform. Unlike other platforms (Win / Mac), we don’t have to rewrite our own libraries and we can reuse existing ones ... Why is this not happening? GStreamer is trying to get everyone together and solve most of these problems, but still gets kicks from some developers for unknown reasons. On the day when we solve this issue ... we will become dominant on desktop platforms, as we did for embedded devices and servers.
What other multimedia applications do you use?
Totem , Rhythmbox , Audacity (Oh, he doesn't use GStreamer). Sometimes AviSynth / VirtualDub (Win32 applications) via WINE, but this can change, thanks to the GSoC project, designed to transfer AviSynth filters to GStreamer.
Well, thank you, Edward, for taking the time for this interview, good luck in the future!
To learn more about the PiTiVi project, visit pitivi.org or #pitivi on irc.freenode.net.
Interview prepared by Christian FK Schaller.
_____
Thanks to ShutteR77 and ilembitov for their help in preparing the translation.
So, tell us a little about yourself first and open projects in which you participate?
My name is Edward Hervey, I am half French, half Scots living in Barcelona, and I am one of the founders of Collabora Multimedia . I've been using Linux and free software since about 1995. But I started making my own contribution in 2003, when I finally had enough programming knowledge to start working on PiTiVi. For several years now I have been working on the GStreamer multimedia framework to raise video editing to a decent level. I also accompany over the years python bindings for GStreamer, GNonLin (the main PiTiVi dependencies are responsible for non-linear editing), as well as FFMpeg plug-ins for GStreamer.
Describe the goals of PiTiVI against the background of dominant video editing applications such as iMovie, Avid or Final Cut Pro?
')
PiTiVi has different goals, but I would point out the fundamental one - the desire to create a video editor without any restrictions (unlike analogs that have very specific limitations in use or support). Get rid of the limitations of formats, devices, filters, ... we can support them through the GStreamer framework. All other editors are tightly linked to their code, but we proudly carry the name of a free editor without binding to any patented library! This means that if a company makes its own GStreamer plugin for a specific format / codec / device ... you can freely use it in PiTiVi. This barrier has been overcome by choosing our design model.
We do not want to get a monolithic video editor, so we decided to rewrite PiTiVi in Python, and make the project modular. The user interface is not tied to the engine, in fact, you can use the engine separately. It is designed so that you can realize the chronological tape in different ways, it is more dependent on your knowledge and how you place the accents, which in turn depend on the use case in the production process.
The ultimate goal of PiTiVi is to provide basic editing tools without any restrictions. This, unfortunately, is worth a lot of effort: you need to make sure that all errors, problems or limitations are fixed at the proper level, instead of adding hack after hack. And it takes time ...
PiTiVI is written in Python. The first question that many people ask when they hear this is: “How does the use of Python affect the performance of multimedia applications, especially such resource-intensive ones as a video editor?”
It's simple :) What has the most CPU load when editing video? That's right, multimedia processing (be it viewing, applying effects, rendering, ...). All this is done exclusively with the help of GStreamer and its plug-ins (written in C, work in separate streams and are used in PiTiVi through Python binding). And what loads the rest of the processing power of the processor? User interface. This is done by GTK + / X / Cairo (written in C and used in PiTiVi through Python bindings).
So, the only thing left is the logic of connecting everything together and bringing in the concepts characteristic of non-linear video editing (NLE) systems. We want to create code as quickly and flexibly as possible, so a language like Python is very convenient. Besides, writing in Python is a pleasure :)
For the curious, I will cite an interesting fact: when playing in PiTiVi, the only part implemented in Python is the code responsible for ... displaying the current position. The same when rendering. Everything else is fully handled by GStreamer, its plugins and GTK +.
A lot of work has been done in GStreamer to support the MXF container format. Most people have probably never heard of such a format, what value does it have?
Since the advent of computer editing (also called non-linear editing), each program has used different formats / codecs for work. What was used by professionals and amateurs was also different, including for cameras, video recorders, capture devices. Add to this the fact that most of the formats have not been documented. We get different formats in the software, in the chamber, at the exit, ... a complete mess.
Some of the key players in the professional market, representatives of SMPTE (Society of Film and Television Engineers) decided to act together and wrote the specifications of the new container format, which will be fully documented, will allow you to store metadata, will be recorded directly to the camera, include streaming broadcasting and a quick search ... He uses the same structural elements as AAF (Extended Development Format), which is open and used by Avid, Final Cut Pro, and many other non-linear editing systems.
With the addition of MXF support in GStreamer, we can use the same formats in PiTiVi as professional NLE! The ability to work directly with common formats is the main factor that allows using PiTiVi in already existing production processes.
What multimedia formats does PiTiVI support?
All that is available through GStreamer plugins installed on your system, as well as some formats for quick video editing, accurate search, I-frame codecs, etc.
At what stage is PiTiVI now and what are your plans for the future?
Over the past 6 months, we have done a deep refactoring of the PiTiVi code, based on the analysis that I performed last summer, summarizing the responses for the last 5 years. In addition to adding the most easily implemented functions, we have already coped with multi-layer editing, creating thumbnails, cropping, ripple / roll modes, support for a common file format, and much more.
We are going to make the first preliminary release of the new pitivi in May. Gradually, new functions will be added to the interface, for example, transitions, effects, title overlays, slip / slide editing and support for a larger number of formats, which will allow, for example, importing projects from other editors.
For those who are not familiar with the terminology, what do you mean by multi-layer editing, sketching, cropping and ripple / roll modes?
Multi-layer editing: the ability to sort video not only by time, but also into several parallel layers.
Creating thumbnails: the resulting image is displayed for each source used in the Timeline scale, which will allow you to more easily navigate the chronological tape without having to search.
Crop: change the position of the beginning or end of the source.
Ripple editing: reduce or increase one of the parts, while the rest of the data adapts to the new size of the changed clip.
Roll editing: the same as ripple, except that the total time in the group of clips remains the same.
Who besides you participates in the project PiTiVI?
The two main developers apart from me are Brandon Lewis (Brandon Lewis), who started working with GSoC 2007 and is busy with a user interface. Alessandro Decina joined us in October, he helps me in the low-level part of PiTiVi and improves GStreamer. Both have been working in Collabora Multimedia since last autumn and are working on PiTiVi 100% of the time. Explanation for those who are thinking about business reasons for the involvement of so many resources in PiTiVi from Collabora Multimedia: firstly, we carry out consulting work for customers in the content creation industry, they want to use PiTiVI to perform specific tasks in their production processes, secondly , we have clients who want to use the PiTiVi infrastructure we created to configure automatic and video manipulations. For example, automatic framing, scaling and transcoding.
How will future changes to GStreamer affect PiTiVI features?
Although there are quite working functions in the current GStreamer core and plugins, we are working on improving a number of features to improve the editing interface.
One of the much needed functions is the implementation of GstIndex support over a wider range of plugins. This will allow you to scan files that are inconvenient for editing during import and create an index that allows you to quickly navigate inside these files. This will allow applications like PiTiVi to get more information about the file format, the location of key frames, thanks to which you will find out which section is more advantageous to skip and where to cut without loss.
Endless work to be done in GStreamer to transfer more and more libraries as plugins. People often mistakenly assume that GStreamer is a codec library, which is completely wrong, GStreamer is a framework that allows application developers not to worry about processing, and library developers not to worry about applications that will use them. We plan to transfer plugins like gavl and the vast majority of excellent AviSynth filters to GStreamer (which will be completed this summer as part of Google Summer of Code). All this, of course, improves the user experience of PiTiVi.
In addition, we are going to reduce the memory and CPU consumption for the GStreamer core and various plugins. We are looking at a more efficient use of hardware devices, such as video cards with VDPAU technology, VA-API and gst-plugins-gl plug-ins. This will allow GStreamer applications to direct processing to hardware devices, leaving more processor time for universal processing.
Jokosher and PiTiVI are multimedia applications that use GStreamer and are written in Python. The natural question is, do these projects collaborate?
Unfortunately, not as active as we would like. We help each other on all fronts (GStreamer, Python bindings, GNonLin plugins), but when it comes to the user interface and the main logic, we have different code usage options and different code bases. It would be great to have closer integration, it would allow you to use Jokosher to edit the audio part of a video clip, and leave video for PiTiVi.
What are the main technical problems you encountered while creating a video editor for Linux / Unix systems? Is Linux infrastructure ready for organizing such applications? And what do you think about the state of multimedia in Linux / Unix in general?
Open-source multimedia has gone * LONG * the way over the last 10 years. For those who do not remember ... we were forced to use different players depending on the format, the only way to capture video from digital cameras. But the problem is that everyone makes their own software in their own corner. The total lack of consistency between various multimedia applications, libraries and device support on Linux is, in my opinion, the reason why we did not become a suitable multimedia platform. Unlike other platforms (Win / Mac), we don’t have to rewrite our own libraries and we can reuse existing ones ... Why is this not happening? GStreamer is trying to get everyone together and solve most of these problems, but still gets kicks from some developers for unknown reasons. On the day when we solve this issue ... we will become dominant on desktop platforms, as we did for embedded devices and servers.
What other multimedia applications do you use?
Totem , Rhythmbox , Audacity (Oh, he doesn't use GStreamer). Sometimes AviSynth / VirtualDub (Win32 applications) via WINE, but this can change, thanks to the GSoC project, designed to transfer AviSynth filters to GStreamer.
Well, thank you, Edward, for taking the time for this interview, good luck in the future!
To learn more about the PiTiVi project, visit pitivi.org or #pitivi on irc.freenode.net.
Interview prepared by Christian FK Schaller.
_____
Thanks to ShutteR77 and ilembitov for their help in preparing the translation.
Source: https://habr.com/ru/post/59732/
All Articles