Python, tone shift and Pianoputer

From the translator:

The article, which I suggest you read, is not new - it was published as early as March 29. But on Reddit, she posted it just a few days ago, and she definitely didn’t lose her actuality. Its interestingness is that the author demonstrates the practical use of three large and popular libraries in a simple and short example: numpy, scipy and pygame. Many have heard about the first two, but more and more in the context of scientific works, so it is interesting to look at their use in “ordinary” life. At the end of the article there is an excellent video demonstration of the result, at least it is definitely worth seeing.

The author's code was kept unchanged, despite the fact that it was not designed according to PEP-8 and I cannot vouch for its validity. One way or another, this working code is on the GitHub; you will find the link at the end of the article.
')
Record the sound, change the tone 50 times and match each new sound with a key on the computer keyboard. It turns out Pianoputer!



The sound can be encoded as an array (or list, list ) of values, like this:



To play this sound twice as fast, delete every second value in the array:



Having done this, we not only halved the duration of the sound, but doubled its frequency, therefore its tone became higher.

On the contrary, if you repeat each value, you get a slower sound, with a longer period, and therefore lower tone:



Here is a simple Python function that changes the speed of sound according to the transmitted coefficient:

import numpy as np def speedx(sound_array, factor): """ Multiplies the sound's speed by some `factor` """ indices = np.round( np.arange(0, len(snd_array), factor) ) indices = indices[indices < len(snd_array)].astype(int) return sound_array[ indices.astype(int) ] 

It is more difficult to change the duration, while maintaining the tone (stretching the sound), or to change the tone, saving the duration (tone shift).

Stretching sound

You can stretch the sound using the classic phase vocoder method. We first break the sound into intersecting pieces, and then move them so that they intersect more (to reduce the sound) or less (to stretch it), as in the picture:



The difficulty here is that the moved pieces can interact poorly, and a certain phase transformation is needed to prevent this from happening. Here is the code on Python, freely rewritten from here :

 def stretch(sound_array, f, window_size, h): """ Stretches the sound by a factor `f` """ phase = np.zeros(window_size) hanning_window = np.hanning(window_size) result = np.zeros( len(sound_array) /f + window_size) for i in np.arange(0, len(sound_array)-(window_size+h), h*f): # two potentially overlapping subarrays a1 = sound_array[i: i + window_size] a2 = sound_array[i + h: i + window_size + h] # resynchronize the second array on the first s1 = np.fft.fft(hanning_window * a1) s2 = np.fft.fft(hanning_window * a2) phase = (phase + np.angle(s2/s1)) % 2*np.pi a2_rephased = np.fft.ifft(np.abs(s2)*np.exp(1j*phase)) # add to result i2 = int(i/f) result[i2 : i2 + window_size] += hanning_window*a2_rephased result = ((2**(16-4)) * result/result.max()) # normalize (16bit) return result.astype('int16') 

Tone offset

After stretching the sound, the tone shift is made simple. To get a higher tone, we stretch the sound, preserving the tone, and then speeding up the result so that the final sound has the same length as the original one, but a higher tone due to the change in speed.

Doubling the frequency of the sound will increase the tone by one octave, or 12 semitones. Thus, in order to raise the tones by n semitones, it is necessary to multiply the height by 2 ^ (n / 12):

 def pitchshift(snd_array, n, window_size=2**13, h=2**11): """ Changes the pitch of a sound by ``n`` semitones. """ factor = 2**(1.0 * n / 12.0) stretched = stretch(snd_array, 1.0/factor, window_size, h) return speedx(stretched[window_size:], factor) 

Application: Pianoputer

Let's try our new toner blender. To start knocking on the bowl:



Then create 50 derived sounds from very low to very high:

 from scipy.io import wavfile fps, bowl_sound = wavfile.read("bowl.wav") tones = range(-25,25) transposed = [pitchshift(bowl_sound, n) for n in tones] 

Assign a sound to each key on the keyboard, following the order specified in this file , like this:



And here is the Python code that turns your computer into a piano (piano computer ):

 import pygame pygame.mixer.init(fps, -16, 1, 512) # so flexible ;) screen = pygame.display.set_mode((640,480)) # for the focus # Get a list of the order of the keys of the keyboard in right order. # ``keys`` is like ['Q','W','E','R' ...] keys = open('typewriter.kb').read().split('\n') sounds = map(pygame.sndarray.make_sound, transposed) key_sound = dict( zip(keys, sounds) ) is_playing = {k: False for k in keys} while True: event = pygame.event.wait() if event.type in (pygame.KEYDOWN, pygame.KEYUP): key = pygame.key.name(event.key) if event.type == pygame.KEYDOWN: if (key in key_sound.keys()) and (not is_playing[key]): key_sound[key].play(fade_ms=50) is_playing[key] = True elif event.key == pygame.K_ESCAPE: pygame.quit() raise KeyboardInterrupt elif event.type == pygame.KEYUP and key in key_sound.keys(): key_sound[key].fadeout(50) # stops with 50ms fadeout is_playing[key] = False 

That's all! And now I’ll play you a traditional Turkish song ( actually, no. Appro. Lane )!



If you want to try the same thing at home, here are all the files you need. I think it would be great if someone among the readers from HTML5 / JS / elm-developers created a browser version of Pianoputer, so it would be available to a wider audience.

Speaking at all, it seems to me that computers are not sufficiently used for playing music. I understand that it is easier to take a real piano keyboard or write a real instrument, but you only see what you can achieve with a regular bowl and 60 lines on Python!

Even a cheap computer has enough controls to become a full-fledged music station: you can sing into a microphone, show gestures through a webcam, modulate all sorts of things with a mouse and control the rest from the keyboard. So many means of expression, and for everyone there is a library on Python ... Artistic hackers, no one wants to step in this direction?