Development for Sailfish OS: working with sound on the example of the DayTimer application

Hello! This article is a continuation of a series of articles devoted to the development of applications for the Sailfish OS mobile platform. This time it will be about developing an application to count the days to an event (for example, before graduation, New Year or Birthday) selected by the user. For each event, the user can add a description and record audio notes. We start the article with a description of the user interface, and then analyze the operation of the application.

Application user interface

The main screen of the application is a list of user events. Each element of the list contains the name and date of the event, by clicking on it, a screen with information about the event opens. List items also have a context menu that allows you to delete an event.

The main screen of the application also contains a pull-out menu with one item that opens a dialog for adding a new note.

The dialog for adding / editing an event contains text fields for entering names and descriptions, buttons for setting the date and time of the event, and components for recording and playing audio notes, which we will discuss in detail in this article.
')
The number of remaining days is displayed on the “cover” of the application, and when the date and time of the event arrives, the user is notified.

Working with audio

Note in advance that in order for the emulator to record and play audio files, including through QML, it is necessary that the qt5-qtmultimedia-plugin-mediaservice-gstmediacapture and qt5-qtmultimedia-plugin-mediaservice- packages be installed on it gstmediaplayer .

The easiest way to do this is to add them to the Requires section of the .yaml file so that they are installed along with the application:

Requires: # Following packages are needed to playback and record audio on the Sailfish OS Emulator. - qt5-qtmultimedia-plugin-mediaservice-gstmediaplayer - qt5-qtmultimedia-plugin-mediaservice-gstmediacapture 

In addition, in the latest versions of the emulator, the volume level is set to the minimum by default, therefore, in order to hear sound files being played, the following commands must be executed on the emulator:

 pactl set-sink-mute sink.emulator 0 pactl set-sink-volume sink.emulator 40000 

You can run them by connecting to the emulator via ssh:

 ssh nemo@localhost -p 2223 -i <  SailfishOS SDK>/vmshare/ssh/private_keys/SailfishOS_Emulator/nemo 

The functionality of recording and listening to a voice note was taken out by the AudioPlayer component embedded in the QML component.

Description of the component AudioPlayer

Work with audio comes from the pages of viewing and editing the event. On the edit page, the component is displayed with a record button, which has two states: start and stop recording (standard and red record button, respectively), a play button ( play ), which changes to a stop button during the listening process, and a download bar.



To start recording, the icon-m-dot icon was selected, and for the user's convenience, a private button was created to stop the recording ( red-dot ). To start and stop listening, the icon-m-media and icon-m-tabs buttons were selected, respectively. Icons available for use can be viewed on the official Jolla website or in the documentation provided with the SDK.

Recording starts when the button is pressed and ends when the button is pressed again or after the set time has elapsed (120 seconds by default), the button for listening becomes available after recording, you can also overwrite the note an unlimited number of times.

Recording is not available on the preview page in the AudioPlayer component, the operation of all other elements does not change.

To store our voice memos, the " DayTimer " folder is located in the " / home / nemo / " directory, which is the system folder in Sailfish OS. However, users can listen and delete their audio notes outside the application. The application checks the availability of the files created earlier and in the case of their absence blocks the function of listening to the voice note for this event.

Additional project settings

To record a voice note, use the QAudioRecorder class, and to listen to the QML type MediaPlayer from the Qt Multimedia library. To work with the library you need to register in the pro-file: QT += multimedia . It is also useful to specify the dependency in the yaml file (the Requires section): qt5-qtmultimedia .

Advanced options when working with audio

It is worth noting that when developing under Sailfish OS, the entire functionality of the audio settings provided by the QAudioEncoderSettings class is also available , which contains many useful methods, for example, the bitRate () method, which returns the bitrate value of the sound file. If desired, we can fix the existing bit rate to the one we need, using the bit rate setting method setBitRate () .

You can learn all available codecs using the supportedAudioCodecs () method of the QAudioEncoderSettingsControl class, which also has a codecDescription () method that describes each codec separately. You can change the audio codec using the setCodec () method of the same class.

In addition, in the classroom, you can find methods for setting the quality, encoding, and more.

Record audio to file

Let us analyze how the work with the database of sound files. When you open the add event page, a temporary file is created in which the created entries are saved. Further actions depend on the user's decision, since the page is a dialog box. The file is deleted if the user cancels the changes, or his name is written (or overwritten, when the voice memo is overwritten) into the application database, if the user has confirmed the changes.

Since there is no standard QML component that records audio to a file, we created and registered our own QML component VoiceRecorder , which uses methods of the QAudioRecorder class, as described in a previous article . In addition to the methods detailed below, the class contains the audioInput property, which stores the actual name of the audio file (you can change it using the setAudioInput () method). The properties and some methods of the QMediaRecorder class are also used.

Next, we turn to the analysis of the methods of the QAudioRecorder class that we use . This class contains the audioRecorder object, with which the record () and stop () methods work, which are responsible for recording a voice note. Also, the isRecording property is defined in VoiceRecorder , which determines whether the recording is currently being recorded or not. The recordingChanged signal signals when the value of this property changes.

 class VoiceRecorder : public QObject { Q_OBJECT Q_PROPERTY(bool isRecording READ isRecording NOTIFY recordingChanged) public: explicit VoiceRecorder(QObject *parent = nullptr); bool isRecording() const; // ... signals: void recordingChanged(); private: QAudioRecorder *m_audioRecorder; bool m_recording; }; 

The class has methods for recording ( record () ) and stopping ( stop () ), which use states from the QMediaRecorder class.

 void VoiceRecorder::record(const QString &name) { audioRecorder->setOutputLocation(QUrl(name)); if (audioRecorder->state() == QMediaRecorder::StoppedState) { audioRecorder->record(); b_recording = true; emit recordingChanged(); } } void VoiceRecorder::stop () { if (audioRecorder->state() == QMediaRecorder::RecordingState) { audioRecorder->stop(); b_recording = false; emit recordingChanged(); } } 

To overwrite a voice note or delete an event in a class, use methods to check for audio ( isExistVoice () ) and remove it ( removeVoice () ), using standard methods of the QFile class ( exists () and remove () , respectively).

In the createVoiceName () method, using the QDir class mkpath () method, a folder is created in which voice notes will be stored. To create audio file names, the QUuid class is used , so the names will be unique.

 QString VoiceRecorder::createVoiceName() { QDir path = QDir::home(); const QString storagePath = QStringLiteral("%1/DayTimer").arg(QDir::homePath()); if (!path.exists(storagePath)) { path.mkpath(storagePath); } QUuid u = QUuid::createUuid(); QString str = QStringLiteral("%1/DTVoice%2.wav").arg(storagePath).arg(u.toString()); return str; } 

Work with audio from a QML page

Let's connect the AudioPlayer component on the viewing and editing pages and proceed to analyze its QML components, for which the VoiceRecorder class was created.

 AudioPlayer { id: voice property bool flagRecordButtonVisible: true } 

All the work on recording a voice note is tied directly to the recordButton button, the code for which is shown below. It is worth noting the visible property: it is determined based on the page on which we connect the AudioPlayer , and sets the accessibility of the button for recording using flagRecordButtonVisible .

 IconButton { id: recordButton icon.source: "image://theme/icon-m-dot" visible: flagRecordButtonVisible onClicked: { progressBar.value = 0; isRecord = true; } // ... } 

In the AudioPlayer component , we created an object of type MediaPlayer , and also set the isRecord property, which is used as a flag to set the maximum value of the ProgressBar . If the flag is true , then the maximum value of the ProgressBar is equal to 120 seconds, otherwise - the duration of the audio file. A detailed description of the ProgressBar operation will be given below.

 Item { id: audioPlayer property bool isRecord: false MediaPlayer { id: player } } 

The following is the button code for listening to the voice memo. The player.source () method in the setSource () function sets the path to the file you want to play.

 IconButton { id: playButton // ... onClicked: { module.setSource(); isRecord = false; timer.stop(); progressBar.value = 0; player.seek(player.duration); recordButton.enabled = true; playButton.icon.source = "image://theme/icon-m-media"; // ... } } // ... function setSource() { if (voiceRecorder.isExistVoice(tempName)) { player.source = tempName; } else { player.source = voiceName; } } 

During the implementation, we were faced with the fact that when we first listened to the overwritten voice note, the old record was played, since the standard player.stop () method did not unload the old file from the buffer. To solve this problem, we used the seek () method, which sets the position from which the audition will begin.

To visualize the recording and playback of a voice note, use the ProgressBar , in which the time is counted using the Timer component. The maximum value of ProgressBar sets depending on whether a voice note is recorded or heard (the property of the isRecord component is used ). In the first case, these are the previously mentioned 120 seconds, and in the second, the length of the voice memo being listened. It is worth noting that the value itself is specified in milliseconds divided by the timer interval. When the ProgressBar reaches its maximum value, recording or listening ends.

 ProgressBar { id: progressBar // ... maximumValue: { if (player.duration > 0 && !isRecord) { return player.duration / 100; } else { return 1200 } } Timer { id: timer interval: 100 repeat: true onTriggered: { if (progressBar.value >= progressBar.maximumValue) { // ... timer.stop(); progressBar.value = 0; // ... } else { progressBar.value += 1 } } } 

Conclusion

As a result, an application was created that allows you to create events with voice notes and track the remaining number of days before them. DayTimer application sources are published on BitBucket, and the application itself is available for download to anyone interested in OpenRepos.

Technical issues can also be discussed on the Sailfish OS Russian-speaking community channel in a Telegram or VKontakte group .

Authors: Svetlana Yurkina, Marina Kuchma, Irina Moskovkina