Animated GIF with sound

“ What allowed the GIF to remain is the looping animation that Netscape added. If Netscape had not added GIF support to its browser, GIF would have died in 1998 "

- Alexander Trevor,
CompuServe GIF Team Leader

The GIF format in June of this year celebrated its 25th anniversary, and is today the oldest graphic format that is distributed on the Internet. Devoting the weekend to viewing funny animated gifs you realize that some of them would be much better with sound. All current solutions for looping animation with sound (for example: coub.com , gifsound.com ) offer to abandon the GIF, but this is not an option. And I decided to donate by viewing the gifs on the weekend to solve this extremely important problem.

The first gif on the Internet with the sound of the link . It is necessary to press the blue button, and then on the gif. The player should work in all modern browsers (tested in the latest Firefox and Chrome).
')
There will not be a gif under the cut, but there will be a process of creating an extension for the standard, writing a converter and a player.

Since 1987, the GIF format has experienced only two significant changes:

1. In 89, the second version of the format (named GIF 89a) was released. It became possible to specify the delay between pictures (several pictures in one file were in the first GIF format 87a). Third-party developers are now able to add their own blocks to a file (Application Extension Block).
2. In 90, Netscape added its own block, which allowed you to specify how many times the animation would repeat.

Format expansion development

As mentioned above, the GIF 89a standard allows applications to place their data in a GIF file. The format of the extension unit for applications:

Size, byte	Content
2	Application extension block header (always 0x21, 0xFF)
one	Block size (always 11)
eight	Application Id
3	Application Authentication Code (can be used to verify that a block is created by a specific application)
*	Nested data blocks
one	Block end pointer (0x00)

Let's try to fit the WAVE file header here:

Size, byte	Content
four	File Header (always “RIFF”)
four	Data size
four	Data format (for WAVE files - “WAVE”)

Since the block size is controlled by the GIF format, we remove the data size field from the header, and write “RIFFWAVE” to the application ID. The remainder of the WAVE file is written as nested GIF blocks.

We will insert the block with sound right in front of the first block with the image (in fact, you can insert it anywhere).

Converter Development

The input converter accepts GIF and WAVE files and outputs GIF with a RIFFWAVE block at the output. The source code can be viewed on google code .

The code is quite simple, read the WAVE file, create a GIF block from it. Then we read the gif file and write down all the blocks, as soon as we find the first block with the picture, we insert a block with a sound in front of it. The most important part of the code is the conversion of the WAVE file into a GIF block:

def get_wav_block(file): #     (signature, size, format) = unpack('4sI4s', file) if signature != 'RIFF': raise Exception('Not a RIFF file') if format != 'WAVE': raise Exception('Not a WAVE file') data = file.read(size - 4) #    wave_block_header = struct.pack('BBB8sBBB', 0x21, 0xff, 11, 'RIFFWAVE', 0, 0, 0) data_subblocks = [wave_block_header]; #      255  for i in range(0, len(data) // 255): data_subblocks.append(chr(0xff)) data_subblocks.append(data[i*255:(i + 1)*255]) if (len(data) % 255) > 0: rest = len(data) % 255 data_subblocks.append(chr(rest)) data_subblocks.append(data[-rest:]) #     data_subblocks.append(chr(0)) return ''.join(data_subblocks) 

Making a gif with sound

Making an animated gif:

 # ffmpeg    ffmpeg -ss 0:00:9.73 -t 2.86 -i warlus.webm -s 500x280 -r 10 frames/image%03d.png #   GIF convert -delay 10 -loop 0 frames/*.png source.gif 

We get a sound and check that everything looks (and sounds) as we planned:

 mplayer -ss 0:00:9.73 -endpos 2.86 warlus.webm -ao pcm:file="source.wav" -vo null mplayer -loop 0 -audiofile source.wav source.gif 

Convert to GIF with sound

 python wave2gif.py example/source.gif example/source.wav result.gif 

Making the player

The basis of the player will take jsgif - a player for animated GIF in JavaScript. jsgif parses the gif and loses by drawing each frame on the Canvas. Add to it a function that, when a block is detected by the “RIFFWAVE”:

• convert the data from the block back to the WAVE file format;
• convert the resulting file to the data: URL and pass to the Audio element.

 var doSound = function(sound) { // Header var size = sound.data.length + 4 var size_text = String.fromCharCode(size & 255, (size >> 8) & 255, (size >> 16) & 255, (size >> 24) & 255); var header = [ "RIFF", size_text, // length "WAVE" ].join(''); var out = [header, sound.data].join(''); var dataURI = "data:audio/wav;base64," + escape(window.btoa(out)); sound_element = new Audio(); sound_element.src = dataURI; }; 

jsgif does not work fast, but with the addition of sound, it became even slower. Also, to get the file data, the player calls XMLHttpRequest, so the player works only with pictures from one domain. But is it an interference to art.

What's next?

Enjoy gifs with sound. You can implement a plugin for browsers that would allow to play such gifs without additional XMLHttpRequest, and which will probably work faster. If someone came across a similar task, I would be grateful for indicating which way to look in order to write a plugin that handles certain types of files .

Sources

GIF89a Specification
WAVE PCM soundfile format
jsgif: A GIF player in JavaScript