Real time music generation

“How to automate the composition of music?” - this question has been disturbing the minds of musicians since the Middle Ages. Kepler turned the trajectories of the planets into music; Mozart and his contemporaries invented the game of "musical bones" - they chose bars from a large table and made of them minuets by throwing dice. But only with the advent of computers, the algorithmic generation of music received a real development. Probability theory, Markov chains, artificial neural networks - all this has become tools for creating music.

Of course, purely mathematical music is rarely harmonious, despite the beauty of mathematics as a science. Therefore, people began to try to automate the writing of familiar, "classical" music. Identify patterns in the works, decompose music into "components" and understand why a particular combination of notes evokes different emotions in the listener. Why is a simple gamut movement in different contexts perceived differently? Is there a perfect piece? There are still no hard answers to these questions - there are only philosophical reflections of musicologists and theorists. But musicians and mathematicians around the world do not give up and continue to dig this question.

We will consider that with the introduction it is finished and we will go straight to my work. I started to get involved in the generation of music a couple of years ago and chose this topic as my master's work at the SFMU faculty.
The topic of my thesis sounds scary: "Incremental generation of music based on dynamic patterns." Translated into Russian: the generation of music in real time using variable patterns.
')
By writing this post, I was inspired by a similar work , published yesterday and a personal request from Vokabre to highlight this issue. I will try to use the terms from music theory to a minimum and tell everything in simple language, but sometimes I can be brought in, so I apologize in advance.

Formulation of the problem

I set myself the task to write a music generator, which could do the following:

• Generate melodious melodies;
• Harmonize the melody, according to the rules of harmony;
• Generate the accompaniment of the available chords;
• All of the above can be done in real time (i.e., on the fly, beat by beat);
• And at the same time allow to interfere in the generation process (change instruments, tone, height of notes, etc.);
• Well, and optionally: display all of this in the form of notes, save in midi and mp3.

That is, I wanted to get a tool that would turn a user into a composer and a conductor of an improvising ensemble at the same time: everyone plays smoothly, in harmony, someone solo, and the rest are accompanied and the leader can order someone to play at any time. Wow!

Sample music generated

Tools used

I developed everything in the Java language, for which there is a wonderful music library jMusic , which already contains musical primitives, such as notes, phrases or parts, as well as MIDI tools. To play the sound, a Gervill synthesizer was used , which is part of JDK 1.7 and allows you to play music using external sound cans in the .sf2 format. JAVE (Java Audio Video Encoder) library was also used - it pinches captured wav-audio into compact mp3.

Generator structure

It's all obvious. The model is simplified and does not claim to be universal, but the basic structural units of a musical work are clearly demonstrated.



Conventionally, the whole generation process can be divided into three stages: the generation of the main melody, its harmonization and, finally, the generation of minor voices on this harmony. The resulting musical beat is played and generation continues. Consider the modules separately:

Module main melody

This is the most complex of modules. He actually generates melodies. Partly randomly, partly deliberately - patterns are used for this. The pattern in this context is a sequence of two or more notes (or smaller patterns) having a relative duration, not tied to any key or chord. That is, the pattern is how notes can be connected to each other. The simplest patterns are as follows:



In Bach, some of the works are very easily described by the same patterns, for example, this prelude in the C minor has the same pattern (only the chord that is moving changes)



I repeat once again: the pattern is not absolute notes, it is only the relationship between the notes. “The second note is higher than the previous one by half a tone”, or “the second note is separated from the first one by three steps of the current key”, or “the second note is the next note on the current chord” (i.e., for the C major chord, “C” will sound "Mi").

In this way, all possible relationships between two neighboring notes belonging to the same motive are described. A base of patterns often used in music is created and the main melody is built with their help.



The construction is carried out in three stages.

1. Building a frame of the simplest patterns.
2. Replacing some of the notes with shorter duration patterns (while maintaining movement towards the next note)
3. Making final adjustments and adding decorations (forchlags and trills)

As a result, we get some kind of melody. She will be our motive for the next four bars. In the future, this motive will either be repeated, but from a different note, either it will be slightly modified, or it will be replaced with another motive altogether - depending on the structure of the current proposal.

Harmonization module

Harmonization of the melody is a task worthy of a separate study. For the sake of a program capable of solving problems in harmony, students of muses. Colleges are willing to go to great lengths. For them, harmony is something like a synonym for the word "matan". And for good reason - there are not so many rules of harmony, but there are a lot of exceptions from them. Therefore, the development of an algorithm that would truly harmonize one cycle, taking into account previous cycles, but without information on subsequent cycles, can take a very long time. In truth, I even had a discussion about the existence of such an algorithm with my friend theorist of music.

Thus, it is necessary to simplify the task and make it workable within a reasonable time. So let's introduce a number of restrictions: first, we will only harmonize with triads (no sept and non-chords, so far), second, we will disregard a few rules regarding reduced chords that can make life difficult, and third, we will make the movement to the dominant in the end of the 4th bar and the tonic at the end of the 8th. That the listener had a feeling of completeness.



Here is the scheme of the admissibility of the use of some chords after others. There are also a few more nuances regarding the transfer of chords across the bar and so on, but I’ll omit them in order not to go into details.

The input harmonizer receives a beat with the melody, and according to the existing rules it harmonizes it, creating a grid of chords. Here comes the last generating module.

Secondary chord generation module

The operation of this module is quite simple: it takes a chord grid and, substituting the simplest patterns, generates simple minor voices. The basis of the notes that make up the chord, and thus guaranteed harmony. Accompaniment is what is usually played in the left hand of the piano, or standard brute force on the guitar. In fact - also a set of patterns, only a shorter duration.

Reproduction

When everything is generated, the resulting score is available to all players: one outputs it as sound, the other draws notes, and the third writes to the file. To a brilliant work is not lost!

User interface

The user interface allows you to set parameters both before and during the generation itself (the changes will take effect in the next cycle). You can change instruments, their roles (melody / secondary voice / accompaniment), volume, octave, and so on. You can also adjust the overall tempo and volume of the piece, its tone and frequency of the appearance of jewelry. This is such a conductor's console for a virtual ensemble.

The program is still in development and much more will be added. The modular system allows you to improve individual components without breaking others.

UPDATE 02.07.2013

I received numerous requests to share the program or source code. So, the situation here is this: for the next couple of months, I plan to introduce new features into the program, significantly improve the generation algorithm, and make a few more buns. After that, I will most likely create a project on github, where you can get the source code and the program itself. On this occasion, apparently, there will be another separate post on Habré.

I was also often asked to give an entire master's dissertation to read - which I gladly do. The full text of my work is available here: http://armaxis.ru/files/MusicGenerator.pdf . I also laid out a presentation that demonstrated on the defense, which highlights the main points of the program.

Thank you all for so much active discussion of this topic, I really did not expect this!

UPDATE 04.10.2013
Laid out the source of the project on Github.
Join the development! =) https://github.com/Armaxis/jmg