Record and modify sound in the browser

Recently decided for fun to make a site on which there will be a recording and modification of the sound. And I also wanted some appropriate animation. I know how to work with sound in C ++ or C #, but I have never done it in a browser.
A little googling, it turned out that there are not too many opportunities to record sound. The most common is using Flash. I have no experience in Flash, besides, I wanted to make the whole UI and functionality in JavaScript + HTML, so I had to somehow do without Flash or with minimal participation. As a result, I found a jQuery jRecorder plugin for recording sound, which eventually uses Flash, or rather ActionScript code, inside itself. But since the work with sound was wrapped in JavaScript, this option suited me.


My idea was to make the person speak something into the microphone, that sound was recorded, and then reproduced a little distorted. For fun, I wanted to add there some more simple animation. But, I'm a programmer, not a designer, so drawing a Flash or HTML5 movie is not at all mine. I decided to unscrew more simple sposb - I drew the page of the site myself, but decided to use gif as animation. Googled a funny Hamster who was chewing on something, and the thought came to mind - let him be silent (listening to a person say something into the microphone), and then “say” it. That is, there was such a problem:
- Sound recording
- Sound distortion
- Play the sound and turn on the animation

Well, work has begun to boil. First, I wrote a simple JS-code for the tests, which switches the gif image to the static picture of the Hamster:

function setPictureHamsterStop() { document.getElementById("switch").src = "2.png"; } function setPictureHamsterSpeech() { document.getElementById("switch").src = "3.gif"; } 

Next, it was necessary to embed the jRecorder code into my page, namely, that Gif was shown during audio playback and Png during recording. jRecorder embeds the Flash window into the page and makes it invisible.
In your page you need to insert a small block of CSS on top, and basically the body to place the initialization script with the settings:

 $.jRecorder( { host : '______wav' , callback_started_recording: function(){callback_started(); }, callback_stopped_recording: function(){callback_stopped(); }, callback_activityLevel: function(level){callback_activityLevel(level); }, callback_activityTime: function(time){callback_activityTime(time); }, callback_finished_sending: function(time){ callback_finished_sending() }, swf_path : 'jRecorder.swf', } ); 

I decided to put the site on a free hosting, for which I used my Google Drive account. How to use it under a hosting on Habré already wrote . There are a lot of restrictions, one of them does not allow
I write a php-script file to Google Drive from the outside. Therefore, the site can only be static. But it did not bother me, since all the work is happening “on the client”.
Next, I copied all the JS code from the jReader and first of all removed from it callback handlers that I don’t need. The main events for me were callback_started, callback_stopped, callback_finished_sending. Callbacks speak for themselves. The algorithm is simple:
- After the recording starts, callback_started comes, and we set the picture as static (Hamster is silent and listens)
- after stopping the recording, we get into callback_stopped and do SendFile
- OnSendFinished shows the gif-animation, as the sound starts to play (this is according to the logic of jRecorder itself

But here is the problem: when to start or stop recording? I didn’t want to do this with a simple button, let the hamster say the words only when something was really said into the microphone, and there was no simple noise or silence.
For this, I decided to analyze the sound level from the microphone, for good luck, jRecorder throws a callback_activityLevel, in which the sound level is transmitted - level. I just had to come up with an algorithm. And I decided to do this:
- The method of selection set the optimal sound level, which can be considered noise (by the way, later, after digging into the ActionScript source code jRecorder, it turned out that it has a similar value and it is equal to mine).
- Again, the method of selection set the threshold length of the recording noise. That is, I started a simple counter, which each time increases by 1 if noise has come. If this counter is greater than the threshold value, then we stop recording (there’s no need for us to record and reproduce noise).
- Each time when entering the callback_activityLevel handler, we check if this level is noise: if yes, then increase the noise counter by 1, and if not, reset this counter (let's start counting again).
- Additionally, set the Boolean flag, which is set to true if the noise threshold has been exceeded at least once during the entire record. This is in order not to drive "empty" records over the network - we save traffic.
')
As a result, if a person doesn’t say anything for a long time and doesn’t get any additional noise into the microphone, we don’t play anything. In the case of a conspiracy (well, or noise, which also happens) we write 30 seconds of speech,
or if a person stops talking earlier, our noise threshold meter will stop recording itself. After stopping the sound is played:

 var SILENCE_LEVEL = 5; var PEAK_LEVEL = 10; var MAX_SILENCE_TICKS = 50; var MICROPHONE_AMPLIFY_LEVEL = 10; var silenceCounter = 0; var wasLevelPeak = 0; var isRecording = 0; function callback_started(){ //     -    . setPictureHamsterStop(); silenceCounter = 0; totalTime = 0; wasLevelPeak = 0; isRecording = 1; } function callback_stopped(){ silenceCounter = 0; isRecording = 0; if (wasLevelPeak) { //   -  ,      . //           . wasLevelPeak = 0; $.jRecorder.sendData(); } else { $.jRecorder.record(30); } } function callback_finished_sending(){ //  GIF ,     . var timer = setTimeout('setPictureHamsterSpeech();', 2000); var timer = setTimeout('$.jRecorder.record(5);', totalTime * 1000); } function callback_activityLevel(level){ //   . if (level > PEAK_LEVEL && isRecording) { wasLevelPeak = 1; // ,  -... silenceCounter = 0; } //  ""    . if(level < SILENCE_LEVEL && isRecording) { silenceCounter = silenceCounter + 1; } //       -    // (   ,    ). if (silenceCounter == MAX_SILENCE_TICKS && isRecording) { silenceCounter = 0; $.jRecorder.stop(); } } 

With the Java-Script part of the record-play sorted. Now the next task is to modify the sound. jRecorder comes with source codes on Action Script, but I don’t know it, and never really worked with Flash.
But the ActionScript code was very natively comprehensible, and I quickly figured out the logic of sound recording and playback. I needed to add a sound modification code, compile it into a * .swf file, and put in place of the existing jRecorder.swf. I put the Trial version of Flash, opened the AudioRecorderCS4.fla project, google the sound modification code, and to my luck, on the official website of Adobe I found examples of working with sound.

During recording from the microphone are packs of raw byte - samples. In jRecorder, a sound processor is written, which, triggered by SampleDataEvent, added a new packet of bytes to the common heap so
the result was a large array of bytes of recorded sound:

 private function onSampleData(event:SampleDataEvent):void { _recordingEvent.time = getTimer() - _difference; dispatchEvent( _recordingEvent ); //       while(event.data.bytesAvailable > 0) _buffer.writeFloat(event.data.readFloat()); } 

To make the sound funnier, you just need to skip a few bytes, that is, during playback, the sound will play just faster:

 private function onSampleData(event:SampleDataEvent):void { _recordingEvent.time = getTimer() - _difference; dispatchEvent( _recordingEvent ); /*   */ event.data.position = 0; while(event.data.bytesAvailable > 0) { _buffer.writeFloat(event.data.readFloat()); if (event.data.bytesAvailable > 0) //  , ,     -  { _buffer.writeFloat(event.data.readFloat()); } if (event.data.bytesAvailable > 0) { event.data.position += 2; //  ,  - } } } 

Is done. Ctrl + Enter, compile, replace jRecorder.swf, and get a working prototype. A little Krivoruk of graphics: he drew a rocket in space, “fitted” gif pictures in size so that the hamster “sat” in a rocket
(using the Online Image Editor ) and laid out CIE at Google Drive hosting. We open the site, Flash asks permission to access the microphone:

If the user agrees, the write-play cycles begin. As a result, it turned out to be a somewhat amusing hand-made article, plus an experience in working with sound. Here is the result: Space Hamster .
It may well happen that in some browser it will not work if there are any reviews, I will try to collect statistics on this issue.