📜 ⬆️ ⬇️

Creating a simple audio editor

We offer readers the continuation of the article from our partners, Music Paradise. Last time, the team presented a tutorial on extracting audio data from wav files; Today we will discuss how to use this functionality in a wider context - when developing a full-fledged audio editor with a standard set of functions.


“In the previous article we looked at the process of extracting audio data and even managed to build a graph based on them. However, we did not make any changes to the audio data and, accordingly, there was no need to save the audio file. We only noted that the process of preservation is inverse to reading. Therefore, in order not to be unfounded, we decided to back up the words with a deed and consider the full cycle of working with an audio file. The expediency of this undertaking is confirmed by numerous questions on this subject on the Internet, and most of them remain open.

Having set a goal to shed light on this problem until the end and give a ready solution for working with audio data, we will describe the process of creating a simple application in the arsenal of which will be all the basic operations for editing an audio file: copy, cut, paste, delete and, of course, save the result to device. For the sake of consistency of presentation and simplicity of perception, we will take the implementation of part of the functional from the previous article, having previously subjected the code to some changes.

Without losing time, we immediately transfer to our new project the already familiar GraphicalWavePlot structure and the PlottingGraphImg class, but add the method to the latter:
')
public async Task<SoftwareBitmapSource> GetImage() { if (softwareBitmap.BitmapPixelFormat != BitmapPixelFormat.Bgra8 || softwareBitmap.BitmapAlphaMode == BitmapAlphaMode.Straight) { softwareBitmap = SoftwareBitmap.Convert(softwareBitmap, BitmapPixelFormat.Bgra8, BitmapAlphaMode.Premultiplied); } var source = new SoftwareBitmapSource(); await source.SetBitmapAsync(softwareBitmap); return source; } 

We need it to control the visual state of the audio track.

In addition to the added method, do not forget to change the color of the wave drawing. In the previous article we used white, but here it is not the best idea - on a white background, the plot will be indistinguishable.

Another class known to us, WavFile, will undergo multiple changes. We must bring it to the following form:

Expand
 public class WavFile { public string PathAudioFile { get; } public string FileName { get; } private TimeSpan duration; public TimeSpan Duration { get { return duration; } } private const int ticksInSecond = 10000000; #region HeadData int headSize; int chunkID; int fileSize; int riffType; int fmtID; int fmtSize; int fmtCode; int channels; int sampleRate; int byteRate; int fmtBlockAlign; int bitDepth; int fmtExtraSize; int dataID; int dataSize; public int Channels { get { return channels; } } public int SampleRate { get { return sampleRate; } } #endregion #region AudioData private List<float> floatAudioBuffer = new List<float>(); #endregion public WavFile(string _path) { PathAudioFile = _path; FileName = Path.GetFileName(PathAudioFile); ReadWavFile(_path); } public float[] GetFloatBuffer() { return floatAudioBuffer.ToArray(); } public void SetFloatBuffer(float[] _buffer) { floatAudioBuffer.Clear(); floatAudioBuffer.AddRange(_buffer); CalculateDurationTrack(); CalculateDataSize(); CalculateFileSize(); } public void CalculateDurationTrack() => duration = TimeSpan.FromTicks((long)(((double)floatAudioBuffer.Count / SampleRate / Channels) * ticksInSecond)); public void CalculateDataSize() => dataSize = floatAudioBuffer.Count * sizeof(Int16); public void CalculateFileSize() => fileSize = headSize + dataSize; void ReadWavFile(string filename) { try { using (FileStream fileStream = File.Open(filename, FileMode.Open)) { BinaryReader reader = new BinaryReader(fileStream); chunkID = reader.ReadInt32(); fileSize = reader.ReadInt32(); riffType = reader.ReadInt32(); long _position = reader.BaseStream.Position; int zeroChunkSize = 0; while (_position != reader.BaseStream.Length - 1) { reader.BaseStream.Position = _position; int _fmtId = reader.ReadInt32(); if (_fmtId == 544501094) { fmtID = _fmtId; break; } else { _position++; zeroChunkSize++; } } fmtSize = reader.ReadInt32(); fmtCode = reader.ReadInt16(); channels = reader.ReadInt16(); sampleRate = reader.ReadInt32(); byteRate = reader.ReadInt32(); fmtBlockAlign = reader.ReadInt16(); bitDepth = reader.ReadInt16(); if (fmtSize == 18) { fmtExtraSize = reader.ReadInt16(); reader.ReadBytes(fmtExtraSize); } dataID = reader.ReadInt32(); dataSize = reader.ReadInt32(); headSize = fileSize - dataSize - zeroChunkSize; byte[] byteArray = reader.ReadBytes(dataSize); int bytesInSample = bitDepth / 8; int sampleAmount = dataSize / bytesInSample; float[] tempArray = null; switch (bitDepth) { case 16: Int16[] int16Array = new Int16[sampleAmount]; System.Buffer.BlockCopy(byteArray, 0, int16Array, 0, dataSize); IEnumerable<float> tempInt16 = from i in int16Array select i / (float)Int16.MaxValue; tempArray = tempInt16.ToArray(); break; default: return; } floatAudioBuffer.AddRange(tempArray); CalculateDurationTrack(); } } catch { Debug.WriteLine("File error"); } } public void WriteWavFile() { WriteWavFile(PathAudioFile); } public void WriteWavFile(string filename) { using (FileStream fs = File.Create(filename)) { fs.Write(BitConverter.GetBytes(chunkID), 0, sizeof(Int32)); fs.Write(BitConverter.GetBytes(fileSize), 0, sizeof(Int32)); fs.Write(BitConverter.GetBytes(riffType), 0, sizeof(Int32)); fs.Write(BitConverter.GetBytes(fmtID), 0, sizeof(Int32)); fs.Write(BitConverter.GetBytes(fmtSize), 0, sizeof(Int32)); fs.Write(BitConverter.GetBytes(fmtCode), 0, sizeof(Int16)); fs.Write(BitConverter.GetBytes(channels), 0, sizeof(Int16)); fs.Write(BitConverter.GetBytes(sampleRate), 0, sizeof(Int32)); fs.Write(BitConverter.GetBytes(byteRate), 0, sizeof(Int32)); fs.Write(BitConverter.GetBytes(fmtBlockAlign), 0, sizeof(Int16)); fs.Write(BitConverter.GetBytes(bitDepth), 0, sizeof(Int16)); if (fmtSize == 18) fs.Write(BitConverter.GetBytes(fmtExtraSize), 0, sizeof(Int16)); fs.Write(BitConverter.GetBytes(dataID), 0, sizeof(Int32)); float[] audioBuffer; audioBuffer = floatAudioBuffer.ToArray(); fs.Write(BitConverter.GetBytes(dataSize), 0, sizeof(Int32)); // Add Audio Data to wav file byte[] byteBuffer = new byte[dataSize]; Int16[] asInt16 = new Int16[audioBuffer.Length]; IEnumerable<Int16> temp = from g in audioBuffer select (Int16)(g * (float)Int16.MaxValue); asInt16 = temp.ToArray(); Buffer.BlockCopy(asInt16, 0, byteBuffer, 0, dataSize); fs.Write(byteBuffer, 0, dataSize); } } } 


It is impossible not to notice the overloaded WriteWavFile method. Thanks to him, our class can now not only receive audio data, but also save it into an accessible file. Now you can see for yourself: we were not deceitful when we said that the writing process is inverse to reading. It uses the same FileStream, only this time for recording. It also became clear why the fields needed to store the data of the file structure became publicly available within the current class. We draw your attention to the fact that some of them - duration, dataSize, fileSize - need to be controlled. To find the fileSize, we did not complicate the algorithm and used the simplest version of the calculation.

These are all fragments that we can transfer from the previous article, the rest will be written from scratch. Let's go in order.

Add the AudioDataEditor class to our project. Note that it is derived from WavFile:

Expand
  class AudioDataEditor:WavFile { private List<float> audioBufferInMemory = new List<float>(); public List<float> AudioBufferInMemory { get { return audioBufferInMemory; } set { audioBufferInMemory = value; } } public AudioDataEditor(string path) : base(path){ } private void SetExactPosition(ref double startPosition, ref double endPosition, int length) { SetExactPosition(ref startPosition, length); SetExactPosition(ref endPosition, length); } private void SetExactPosition(ref double position, int length) { position = (int)(length * position); if (Channels == 2) { if (position % 2 == 0) { if (position + 1 >= length) position--; else position++; } } } public void Copy(double relativeStartPos, double relativeEndPos) { float[] audioData = GetFloatBuffer(); AudioBufferInMemory.Clear(); double startPosition = relativeStartPos; double endPosition = relativeEndPos; SetExactPosition(ref startPosition, ref endPosition, audioData.Length); float[] temp = new float[(int)(endPosition - startPosition)]; Array.Copy(audioData.ToArray(), (int)startPosition, temp, 0, temp.Length); AudioBufferInMemory.AddRange(temp); } public void Cut(double relativeStartPos, double relativeEndPos) { Copy(relativeStartPos, relativeEndPos); Delete(relativeStartPos, relativeEndPos); } public void Paste(double relativeStartPos) { if (AudioBufferInMemory.Count > 0) { List<float> temp = new List<float>(); temp.AddRange(GetFloatBuffer()); double startPosition = relativeStartPos; SetExactPosition(ref startPosition, temp.Count); temp.InsertRange((int)startPosition, AudioBufferInMemory); SetFloatBuffer(temp.ToArray()); WriteWavFile(); } } public void Delete(double relativeStartPos, double relativeEndPos) { List<float> _temp = new List<float>(); _temp.AddRange(GetFloatBuffer()); double startPosition = relativeStartPos; double endPosition = relativeEndPos; SetExactPosition(ref startPosition, ref endPosition, _temp.Count); _temp.RemoveRange((int)startPosition, (int)(endPosition - startPosition)); SetFloatBuffer(_temp.ToArray()); WriteWavFile(); } } 


The AudioDataEditor class implements the operations we talked about at the beginning; their implementation will not seem difficult for you to study in detail.

And each process of changing audio data is accompanied by their saving in the current file. We only note that when working with two-channel audio data, it is worth being more careful with alternating samples.

So, in part, we implemented the main mechanism of our application. But for the full functioning of this is not enough, you need an interface that can correctly control the application. Moreover, editing and performing basic operations with an audio track requires a perfect mechanism, like modern editors, and not just entering the beginning and end of the space to be copied or deleted. Here we also need the display of the audio wave obtained earlier. By highlighting its fragment, we will mark the desired segments.

We will implement the mechanism for selecting a fragment of an audio track by creating our own interface element. Perform the following steps: select Add → NewItem ... from the context menu, select UserControl in the window that appears and name it GraphicsEditor, then click the Add button.



Created by GraphicsEditor.xaml as a result of the changes, it looked as follows:

 <UserControl x:Class="AudioEditorTest.GraphicsEditor" xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation" xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml" xmlns:local="using:AudioEditorTest" xmlns:d="http://schemas.microsoft.com/expression/blend/2008" xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006" mc:Ignorable="d" Height="Auto" Width="Auto"> <Grid> <Image Name="BackgroundImage" Stretch="Fill"/> <Canvas Name="MainContainer" Background="#0001FFFF"> <Rectangle Name="SelectedChunk" Visibility="Collapsed" Height="150" Width="50" Fill="#19000000"/> </Canvas> </Grid> </UserControl> 


GraphicsEditor.xaml.cs:

Expand
 public sealed partial class GraphicsEditor : UserControl { public GraphicsEditor() { this.InitializeComponent(); DataContext = this; MainContainer.PointerPressed += new PointerEventHandler(PointerPressed); MainContainer.PointerMoved += new PointerEventHandler(PointerMoved); MainContainer.PointerReleased += new PointerEventHandler(PointerReleased); MainContainer.PointerExited += new PointerEventHandler(PointerReleased); } private double leftPos; private double rightPos; private double minSelectionWidth = 10; private bool isPressed = false; private double startSelectionPosition = 0; private double currentSelectionWidth = 0; public event EventHandler SelectionChanged = delegate { }; private bool enableSelection = false; public bool EnableSelection { get { return enableSelection; } set { enableSelection = value; } } private SoftwareBitmapSource imageSource; public SoftwareBitmapSource ImageSource { get { return imageSource; } set { imageSource = value; BackgroundImage.Source = value; } } public double LeftPos { get { return leftPos; } set { leftPos = value; SetRightSelectionMargin(); } } public double RelativeLeftPos { get { return leftPos / ActualWidth; } } public double RightPos { get { return rightPos; } set { rightPos = value; SetRightSelectionMargin(); } } public double RelativeRightPos { get { return rightPos / ActualWidth; } } private void SetRightSelectionMargin() { currentSelectionWidth = Math.Abs(rightPos - leftPos); ManageGraphics(); } private void ManageGraphics() { SelectedChunk.SetValue(Canvas.LeftProperty, LeftPos); SelectedChunk.Width = currentSelectionWidth; } private void PointerReleased(object sender, PointerRoutedEventArgs e) { isPressed = false; if (Math.Abs(startSelectionPosition - GetPointerPositionX(e)) < minSelectionWidth) { EnableSelection = false; } } private void PointerMoved(object sender, PointerRoutedEventArgs e) { if (isPressed) { EnableSelection = true; double xPosition = GetPointerPositionX(e); currentSelectionWidth = Math.Abs(startSelectionPosition - xPosition); if (currentSelectionWidth > minSelectionWidth) { if (xPosition < startSelectionPosition) { if (xPosition < 0) { LeftPos = 0; } else if (xPosition < RightPos - minSelectionWidth) { LeftPos = xPosition; } RightPos = startSelectionPosition; } else if (xPosition > startSelectionPosition) { if (xPosition > this.ActualWidth) { RightPos = this.ActualWidth; } else if (xPosition > LeftPos + minSelectionWidth) { RightPos = xPosition; } LeftPos = startSelectionPosition; } } } } private void PointerPressed(object sender, PointerRoutedEventArgs e) { SelectedChunk.Visibility = Visibility.Visible; isPressed = true; startSelectionPosition = GetPointerPositionX(e); LeftPos = startSelectionPosition; RightPos = LeftPos + 2; } private double GetPointerPositionX(PointerRoutedEventArgs e) { PointerPoint pt = e.GetCurrentPoint(MainContainer); Point position = pt.Position; return position.X; } } 


Briefly about the operation of this element: GraphicsEditor allows you to track the position of the mouse cursor on the screen and respond to a click, changing the Rectangle parameters as a result, which, in turn, when applied to the drawn wave, allows you to visually select an arbitrary fragment of the audio track.

Everything is ready to integrate our practices in the files of the main page of the application. In MainPage.xaml.cs, we declare variables of the reference type:

  private StorageFile currentFile, sourceFile; private PlottingGraphImg imgFile; private AudioDataEditor editor; 

Repeat the methods already implemented in the previous article:

 private async Task OpenFileDialog() { var picker = new Windows.Storage.Pickers.FileOpenPicker(); picker.SuggestedStartLocation = Windows.Storage.Pickers.PickerLocationId.MusicLibrary; picker.FileTypeFilter.Add(".mp4"); picker.FileTypeFilter.Add(".mp3"); picker.FileTypeFilter.Add(".wav"); sourceFile = await picker.PickSingleFileAsync(); if (sourceFile == null) await OpenFileDialog(); } public async Task ConvertToWaveFile() { OpenLoadWindow(true); MediaTranscoder transcoder = new MediaTranscoder(); MediaEncodingProfile profile = MediaEncodingProfile.CreateWav(AudioEncodingQuality.Medium); CancellationTokenSource cts = new CancellationTokenSource(); string fileName = String.Format("{0}_{1}.wav", sourceFile.DisplayName, Guid.NewGuid()); currentFile = await ApplicationData.Current.TemporaryFolder.CreateFileAsync(fileName); Debug.WriteLine(currentFile.Path.ToString()); try { var preparedTranscodeResult = await transcoder.PrepareFileTranscodeAsync(sourceFile, currentFile, profile); if (preparedTranscodeResult.CanTranscode) { var progress = new Progress<double>((percent) => { Debug.WriteLine("Converting file: " + percent + "%"); }); await preparedTranscodeResult.TranscodeAsync().AsTask(cts.Token, progress); } else { Debug.WriteLine("Error: Convert fail"); } } catch { Debug.WriteLine("Error: Exception in ConvertToWaveFile"); } OpenLoadWindow(false); } 

Hiding the user interface during the audio file conversion will be the line:

 private void OpenLoadWindow(bool enable) => LoadDummy.Visibility = enable ? Visibility.Visible : Visibility.Collapsed; 

To play an audio track, use the logic:

 private async Task SetAudioClip(StorageFile file) { var stream = await file.OpenReadAsync(); Player.SetSource(stream, ""); } private void Playback_Click(object sender, RoutedEventArgs e) { if (Player.CurrentState == MediaElementState.Playing) { Player.Stop(); } else { Player.Play(); } } private void Player_CurrentStateChanged(object sender, RoutedEventArgs e) => PlayBtn.IsChecked = Player.CurrentState == MediaElementState.Playing; 

We update content using methods:

 private async Task BuildImageFile() { editor = new AudioDataEditor(currentFile.Path.ToString()); await Update(); } private async Task Update() { imgFile = new PlottingGraphImg(editor, (int)AudioEditorControl.ActualWidth, (int)AudioEditorControl.ActualHeight); AudioEditorControl.ImageSource = await imgFile.GetImage(); await SetAudioClip(currentFile); } 

When you start the application or when you try to load a new track, the method will work:

 private async void LoadAudioFile(object sender, RoutedEventArgs e) { await OpenFileDialog(); await ConvertToWaveFile(); await BuildImageFile(); } 

Now there are calls of operations: copy, cut, paste, delete.

  private void Copy_Click(object sender, RoutedEventArgs e) { if (AudioEditorControl.EnableSelection) editor.Copy(AudioEditorControl.RelativeLeftPos, AudioEditorControl.RelativeRightPos); } private void Cut_Click(object sender, RoutedEventArgs e) { if (AudioEditorControl.EnableSelection) editor.Cut(AudioEditorControl.RelativeLeftPos, AudioEditorControl.RelativeRightPos); Update(); } private void Paste_Click(object sender, RoutedEventArgs e) { editor.Paste(AudioEditorControl.RelativeLeftPos); Update(); } private void Delete_Click(object sender, RoutedEventArgs e) { if (AudioEditorControl.EnableSelection) editor.Delete(AudioEditorControl.RelativeLeftPos, AudioEditorControl.RelativeRightPos); Update(); } 

Go to the file MainPage.xaml and implement the following interface:

 <Grid Background="{ThemeResource ApplicationPageBackgroundThemeBrush}"> <Grid> <Grid.RowDefinitions> <RowDefinition Height="Auto"/> <RowDefinition Height="*"/> <RowDefinition Height="Auto"/> </Grid.RowDefinitions> <StackPanel HorizontalAlignment="Left" VerticalAlignment="Center" Orientation="Horizontal"> <Button x:Name="OpenFileBtn" FontFamily="Segoe MDL2 Assets" FontSize="36" Content=" xE8E5;" Background="White" Margin="5" Click="LoadAudioFile"/> <Rectangle Height="50" Width="2" Fill="Black"/> <ToggleButton x:Name="PlayBtn" FontFamily="Segoe MDL2 Assets" FontSize="36" Content=" xE768;" Background="White" Margin="5" Click="Playback_Click"/> </StackPanel> <StackPanel Grid.Row="1" Orientation="Vertical" VerticalAlignment="Center"> <local:GraphicsEditor x:Name="AudioEditorControl" Height="150"/> </StackPanel> <StackPanel Grid.Row="2" HorizontalAlignment="Center" VerticalAlignment="Center" Orientation="Horizontal"> <Button x:Name="CopyBtn" FontFamily="Segoe MDL2 Assets" FontSize="36" Content=" xE8C8;" Background="White" Margin="5" Click="Copy_Click"/> <Button x:Name="CutBtn" FontFamily="Segoe MDL2 Assets" FontSize="36" Content=" xE8C6;" Background="White" Margin="5" Click="Cut_Click"/> <Button x:Name="PasteBtn" FontFamily="Segoe MDL2 Assets" FontSize="36" Content=" xE77F;" Background="White" Margin="5" Click="Paste_Click"/> <Button x:Name="DeleteBtn" FontFamily="Segoe MDL2 Assets" FontSize="36" Content=" xE74D;" Background="White" Margin="5" Click="Delete_Click"/> <MediaElement Name="Player" AutoPlay="False" CurrentStateChanged="Player_CurrentStateChanged" /> </StackPanel> </Grid> <Grid x:Name="LoadDummy" Visibility="Collapsed" Background="#7F000000"> <ProgressRing IsActive="True" Width="100" Height="100" Foreground="White"/> </Grid> </Grid> 


Fine! Our application is ready, it remains to add the missing libraries and check the code for errors. We have created a simple audio editor, but even in it there are possible experiments with personal audio files. Similar logic is implemented in our various projects. Say, Audio Editor has similar functionality, but its advantage is to work with each channel of the audio file separately. Another application, Audio Genesis , has a difference in the number of audio tracks processed at the same time and the ability to mix them into one track. Each of the developed applications has its own field of application, perhaps, you will come up with the idea of ​​a unique tool. ”

Source: https://habr.com/ru/post/330418/


All Articles