Visualization for music player

The information in the article will cover the topic of creating a visualization for a music player. It so happened that the program was written in as3, because This is the language I'm currently programming.
It all started from what I saw on Phthalo's Corona on the AIMP player. I thought for a long time how it works and finally came up with something.

The first prototype was written in C ++.


As a result, I only got a prototype of the motion of particles as in this visualization. I stopped at this and after half a year I again wanted to work with the sound. After 2 days, this flash drive was ready.
LINK
Mouse click - change blur mode.

In order for it to work correctly, you need to open some kind of music player (for example, VKontakte) in the browser, and close all unnecessary tabs with audio data, because when you receive spectral data via SoundMixer.computeSpectrum, the flash drive may crash because the fact that other applications do not allow them to take audio data. If it does not fall, then nothing can be closed. As noted, in my chrome, the sound from other tabs is not caught.
')
We now turn to a review of how this all works.

Physics

Particles are located in the area (650 in this case). 600 active particles and 50 particles that appear at a random point in the area and fall down, when such a particle touches the floor, it respawns again in the area. By itself, all particles are affected by gravity.
In addition to gravity, particles can be influenced by an attractor and a repulsor. Now I will explain what these things are magic. In fact, they are not magic at all, just the names they have are just as abstruse as any "professors" like to say.

Attractor - this is such a thing that attracts everything that is possible. In this case, the particles, and the farther away the particle is from the attractor, the weaker it is attracted.

A repulsor is the same as an attractor, only it on the contrary pushes everything away from itself.

So, on particles under certain conditions (based on the sound spectrum), an attractor and a repulsor act. Because of this, they so jump in the area.

Drawing

A bitmapdata for particles is created, one for all (this is logical). This is a small white circle, why white will explain below.

var radius: Number = PARTICLE_SIZE * 0.5; var particleShape: Shape = new Shape(); particleShape.graphics.beginFill(0xFFFFFF); particleShape.graphics.drawCircle(PARTICLE_SIZE * 0.5, PARTICLE_SIZE * 0.5, radius); m_ParticleBmp = new BitmapData(PARTICLE_SIZE, PARTICLE_SIZE, true, 0x0); m_ParticleBmp.draw(particleShape); 

Particles are drawn like this:

 var velDir: Number = Math.atan2(part.vel.y, part.vel.x); var speed: Number = Math.max(Math.abs(part.vel.y), Math.abs(part.vel.x)); if (speed > 20) speed = 20; matr.identity(); matr.scale(1.3 + speed / 20.0 * 4.0, 0.5); matr.rotate(velDir); matr.translate(part.pos.x, part.pos.y); m_BackBuffer.draw(m_ParticleBmp, matr); 

The matrix transforms the particles so that they stretch as they move. Nothing complicated.

Effects

Palette

When you start the visualization, palettes are created from which color is sampled when drawing the final image (standard fit for the demo-decoders :)

They are randomly selected in the process.

To work with the palette, you need to prepare 3 arrays of 256 values. These will be arrays for red, green and blue.

In order to enable them, you must use the BitmapData class paletteMap function. It receives as parameters these arrays and the original image.

It works like this:

1. takes the color of the pixel from the original image, integer;
2. this color is broken into rgb (byte) components;
3. for these components, the value is taken from the corresponding array, these will be the new rgb values;
4. new rgb values ​​are going back to integer.

Now I will explain why the particles need to be painted in white, and the background in black.

When applying a blur filter, all pixels will be blurred and as a result, if nothing is drawn, they will turn black. When applying the palette, black is replaced with any color, so when blurring, the color will smoothly turn into the color we need (in the buffer into which the particles are drawn, the background will remain black).

The palette is constructed in such a way that rgb arrays are in the superscripts (255 is white), the color of the particles is recorded, and the background is written in the lower indices (0 is black). In all other indices [1..254], the gradient colors from the background color to the color of the particle are recorded. Of course, no one bothers to add more than 2 colors to the palette and make a gradient between them.

Distortion map

In the original visualization, a very interesting twist of a trace of particles was made. I peered into it for a long time to understand how it works.

In the end, I came to the conclusion that this is a hyperbolic spiral .

The equation of a hyperbolic spiral (in polar coordinates):
r * phi = a, where a is the speed of divergence of the helix

Cartesian coordinates:
x = a * cos (phi) / phi,
y = a * sin (phi) / phi, where a / phi = r

There was a question how to do it now O_O. First, you need to somehow set the formula for such a distortion, and second, how to do it on a flash.

Of course, at first I tried to do everything in the old manner, walking through all the pixels of the image and moving the pixels in certain directions, besides, it would allow me to immediately put in and blur. But in practice everything turned out to be much different. The cycle in the image killed fps. I had to look for another solution.

I got the idea to make a twirl effect, immediately found the code through DisplacementMapFilter. But it was not what was in the original.

I had to return to the review of the hyperbolic spiral. The whole difficulty was that the helix was clearly defined through the angle and radius in polar coordinates, and I needed to add another parameter — thickness. As a result, after the derivation of some formulas, I wrote an algorithm for generating the displacement map.

The displacement map is based on the following algorithm:
Running through all the pixels of the image, each pixel is checked for belonging to the spiral (taking into account the thickness). If the pixel belongs to a spiral, then at this point it is necessary to calculate the offset.

To find the direction of the displacement, you can take a tangent at a given point. To simplify the calculations, I decided to find the tangent to the circle at this point. With this approach, the calculation error will be noticeable only in the tail of the helix. Since as the radius (or angle) increases, the y coordinate tends to a, i.e. to the line.

f '(x, y) = ArcTan (y / x) - Pi / 2

Now we just find the vector by the resulting angle and multiply it by the offset coefficient. As a result, such an offset map was obtained (green and blue channels are used)


In the second mode, another displacement map is superimposed, which is generated by Perlin noise.

 m_NoiseBuffer.perlinNoise(SCREEN_WIDTH, SCREEN_HEIGHT - FLOOR_POS, 2, 1, false, true, BitmapDataChannel.RED | BitmapDataChannel.BLUE, false, [m_OctaveOffset, m_OctaveOffset]); 

Here is a nice image is obtained at the output:


It would be possible to add more octaves during generation, but the noise is generated every update (for this, the octave offset is used, for the effect of motion).

As a result, the drawing steps are:

1. A displacementMapFilter filter is applied to the backBuffer with offsets.
2. A blur filter is applied to the backBuffer (x: 8, y: 8, quality: 1).
3. ColorTransform is applied to the backBuffer, so we adjust the image fading speed.
4. Draw the particles in the backBuffer.
5. We apply paletteMap to screeBuffer, we use backBuffer as the source image.
6. Well, the last draw an inverted piece of floor

SOURCES .

That's all, thank you for your attention!