How we did the guitar tuner
In this post we will talk about creating our free Guitar Tuner app for Android.
At the very beginning, at the stage of the emergence of an idea, I wanted to make a universal tuner: the phone hears a sound, determines the note, shows it. In theory, this is realized quite simply: sound is an oscillation, oscillation has a frequency, oscillation frequency is a note.
There is such a wonderful thing in mathematics as the Fourier transform. This conversion gives the frequency spectrum of the analog signal. That is, any signal can be represented as the sum of sinusoidal curves of different frequencies and different amplitudes.
')
Thus, if the signal itself is a sinusoid, then in the frequency decomposition it will have only one member, that is, there will be one frequency in the signal. If the signal is complex, then the spectrum is complex.
Musical instruments play a certain note and at the same time a multitude of overtones (overtones), which means that the spectrum has a complex structure, but the played note (frequency) in the spectrum has the greatest amplitude. As a rule, there is also a certain number of harmonics in the signal, that is, frequencies that are multiples of the fundamental (doubled, tripled, etc.).
Thus, to determine the note sounding on the guitar, it is necessary to remove the audio stream from the microphone of the phone, take the Fourier transform from it, find the frequency with the highest amplitude, and match the frequency and the note.
At each stage we were in trouble. In addition to the latter, of course, where the correspondence tables are known, and easily computable from the known frequency of 440 Hz for the note A of the 1st octave, as well as the known relation between the frequencies of one note of the neighboring octaves (2 times) and the known number of notes in the octave.
First, in those old times of the first version of Android OS , there was no open Java API for working with sound stream from a microphone. There was an API for playing the sound, but not for reading the stream from the microphone. However, there was a Native-code Speech Recongnition system, which worked with a removable audio stream. As a result, as a hack, a wrapper was written to work with these Native streams. As it turned out, this responded to very sad results. But more about that later.
The next step was to obtain the Fourier transform of the signal. Naturally, the Fast Fourier Transform (FFT) algorithm was used. We immediately thought that this, as the most computationally difficult task, would be the bottleneck in the performance of the tuner. The natural solution would be to write this part in C, not in Java, but we decided to compare both solutions. Having implemented the Cooley – Tukey version of the FFT algorithm in Java and C, we held a small benchmark and saw that the time difference was almost two orders of magnitude! Who won, obviously. However, we managed to improve the results of the Java version by almost an order of magnitude, replacing working with complex numbers as with Complex objects with simple pairs of numbers. As a result, the result of the Java-version was quite fast, and we decided to use it in order to avoid the use of excess native-code (in general, less portable).
Then the most interesting problems began. Looking at the spectral images, we did not see what we expected. Instead of a large amplitude of the fundamental frequency and much smaller amplitudes of multiple harmonics, we saw a barely noticeable fundamental frequency and amplitudes of the harmonics obeying incomprehensible laws. After long attempts to understand the dependencies that came before us, in the end we blamed everything on Speech Recongnition . Like, it is he who identifies those frequencies that prevail in human speech.
Since we didn’t have an alternative source for reading the stream at that time, it was necessary to get out somehow. We had to abandon the initial goal of making a universal tuner, and instead make a tuner for pre-beat instruments: six- and seven-string guitars, four-, five-, six-string bass guitars and others. In other words, the user of the tuner, before making a sound on the instrument, chooses the string that he wants to tune himself, and then only plays. Thus, we already know the reference frequency that should ideally be obtained. And knowing from the spectral picture of the signal one of the harmonics of the desired note, we can (by division) know the deviation of the actual fundamental frequency from the reference frequency. In other words, the frequency with the maximum amplitude in a “corrupted” speech is, with a recognizer, an integer divided by the desired frequency, the harmonic number is obtained. Further, by division, the main frequency of the sounding note is actually calculated, the difference between the desired and sounding frequencies is calculated, normalized to the length of the displayed scale, displayed to the user.
Later, an API appeared to work with audio stream from a microphone, and the tuner was redone using it, but there is no support for it at the moment.
resources, so it is not in perfect condition. Despite this, it is actively used by more than 50,000 Android users.
Such is the instructive story. I hope the information received will be useful to you in the future. I am pleased to answer all questions.
At the very beginning, at the stage of the emergence of an idea, I wanted to make a universal tuner: the phone hears a sound, determines the note, shows it. In theory, this is realized quite simply: sound is an oscillation, oscillation has a frequency, oscillation frequency is a note.
There is such a wonderful thing in mathematics as the Fourier transform. This conversion gives the frequency spectrum of the analog signal. That is, any signal can be represented as the sum of sinusoidal curves of different frequencies and different amplitudes.
')
Thus, if the signal itself is a sinusoid, then in the frequency decomposition it will have only one member, that is, there will be one frequency in the signal. If the signal is complex, then the spectrum is complex.
Musical instruments play a certain note and at the same time a multitude of overtones (overtones), which means that the spectrum has a complex structure, but the played note (frequency) in the spectrum has the greatest amplitude. As a rule, there is also a certain number of harmonics in the signal, that is, frequencies that are multiples of the fundamental (doubled, tripled, etc.).
Thus, to determine the note sounding on the guitar, it is necessary to remove the audio stream from the microphone of the phone, take the Fourier transform from it, find the frequency with the highest amplitude, and match the frequency and the note.
Problems
At each stage we were in trouble. In addition to the latter, of course, where the correspondence tables are known, and easily computable from the known frequency of 440 Hz for the note A of the 1st octave, as well as the known relation between the frequencies of one note of the neighboring octaves (2 times) and the known number of notes in the octave.
First, in those old times of the first version of Android OS , there was no open Java API for working with sound stream from a microphone. There was an API for playing the sound, but not for reading the stream from the microphone. However, there was a Native-code Speech Recongnition system, which worked with a removable audio stream. As a result, as a hack, a wrapper was written to work with these Native streams. As it turned out, this responded to very sad results. But more about that later.
The next step was to obtain the Fourier transform of the signal. Naturally, the Fast Fourier Transform (FFT) algorithm was used. We immediately thought that this, as the most computationally difficult task, would be the bottleneck in the performance of the tuner. The natural solution would be to write this part in C, not in Java, but we decided to compare both solutions. Having implemented the Cooley – Tukey version of the FFT algorithm in Java and C, we held a small benchmark and saw that the time difference was almost two orders of magnitude! Who won, obviously. However, we managed to improve the results of the Java version by almost an order of magnitude, replacing working with complex numbers as with Complex objects with simple pairs of numbers. As a result, the result of the Java-version was quite fast, and we decided to use it in order to avoid the use of excess native-code (in general, less portable).
Then the most interesting problems began. Looking at the spectral images, we did not see what we expected. Instead of a large amplitude of the fundamental frequency and much smaller amplitudes of multiple harmonics, we saw a barely noticeable fundamental frequency and amplitudes of the harmonics obeying incomprehensible laws. After long attempts to understand the dependencies that came before us, in the end we blamed everything on Speech Recongnition . Like, it is he who identifies those frequencies that prevail in human speech.
Solutions
Since we didn’t have an alternative source for reading the stream at that time, it was necessary to get out somehow. We had to abandon the initial goal of making a universal tuner, and instead make a tuner for pre-beat instruments: six- and seven-string guitars, four-, five-, six-string bass guitars and others. In other words, the user of the tuner, before making a sound on the instrument, chooses the string that he wants to tune himself, and then only plays. Thus, we already know the reference frequency that should ideally be obtained. And knowing from the spectral picture of the signal one of the harmonics of the desired note, we can (by division) know the deviation of the actual fundamental frequency from the reference frequency. In other words, the frequency with the maximum amplitude in a “corrupted” speech is, with a recognizer, an integer divided by the desired frequency, the harmonic number is obtained. Further, by division, the main frequency of the sounding note is actually calculated, the difference between the desired and sounding frequencies is calculated, normalized to the length of the displayed scale, displayed to the user.
Later, an API appeared to work with audio stream from a microphone, and the tuner was redone using it, but there is no support for it at the moment.
resources, so it is not in perfect condition. Despite this, it is actively used by more than 50,000 Android users.
Such is the instructive story. I hope the information received will be useful to you in the future. I am pleased to answer all questions.
Source: https://habr.com/ru/post/111782/
All Articles