MuseNet Deep Neural Network Writes Music
On May 29th, we start the course “Neural networks in Python” , which tells about deep machine learning and teaches how to solve the most complex tasks that are under the supervision of an engineer for deep learning. To show how interesting and versatile work in the field of machine learning, we suggest to get acquainted with a series of translations of articles on the topic of neural networks in Python.
We created and present to you MuseNet - a deep neural network that can create four-minute musical compositions with the help of ten different instruments. MuseNet is experimenting by combining different musical styles: from country music to Mozart’s masterpieces to the immortal hits of The Beatles.
')
The basis of the network is not easy our perception and understanding of music, but the skill to find patterns of harmony, rhythm and style in hundreds of thousands of MIDI files, and then create our own works. The MuseNet neural network uses the same technology as GPT-2 - the scale model is trained to predict the best sequences in both text and audio. Knowledge of a variety of styles allows the network to create completely unique combinations of music from different generations.
We created tokens for composers and various musical instruments to give MuseNet maximum control over the samples that the network creates. As students learn, composers and tools tokens are added to each sample, which helps the network learn to use this information in forecasting.
MuseNet uses the recalculated and optimized Sparse Transformer cores to train a 72-level network. In our opinion, the generation of music is a very good niche for testing and developing the capabilities of the Sparse Transformer.
We collected data for learning MuseNet in a variety of sources. We were greatly helped by the projects ClassicalArchives and BitMidi , which provided their colossal collections at our disposal. We also used collections of the most diverse music found on the Internet: jazz, pop, African, Indian and Arabic music. In addition, we used the MAESTRO collection.
Training takes place on serial data, taking into account the set of tokens, we ask the network to predict the next token. We tried several different ways to encode MIDI files. At first we thought about the chord approach, which looked at each combination of notes that sounded at the same time as a separate chord and assigned the corresponding token. We also tried to condense musical patterns, concentrating on the beginning of each note.
During the training we:
In addition, we use something like a built-in music critic: we ask him to evaluate the composition and decide whether this sample is created from the data presented or is old. This assessment allows you to select samples in the course of generation.
To make the model have a more structural context, we added integration. In addition to standard positional integrations, we added integrations that track the passage of time in samples. This allows you to simultaneously and simultaneously build sounding notes. We also added two structural integrations that allow MuseNet to understand in which segment of the overall composition a particular sample should be located.
Those interested can get acquainted with the work of MuseNet, neural network already even gives its own online concerts on Twitch . For those who are interested in the creation of Python neural networks and deep machine learning, we conduct an open webinar where we will talk about the course program and the skills that students will gain.
What is MuseNet
We created and present to you MuseNet - a deep neural network that can create four-minute musical compositions with the help of ten different instruments. MuseNet is experimenting by combining different musical styles: from country music to Mozart’s masterpieces to the immortal hits of The Beatles.
')
The basis of the network is not easy our perception and understanding of music, but the skill to find patterns of harmony, rhythm and style in hundreds of thousands of MIDI files, and then create our own works. The MuseNet neural network uses the same technology as GPT-2 - the scale model is trained to predict the best sequences in both text and audio. Knowledge of a variety of styles allows the network to create completely unique combinations of music from different generations.
Composer and tool tokens
We created tokens for composers and various musical instruments to give MuseNet maximum control over the samples that the network creates. As students learn, composers and tools tokens are added to each sample, which helps the network learn to use this information in forecasting.
Long term structure
MuseNet uses the recalculated and optimized Sparse Transformer cores to train a 72-level network. In our opinion, the generation of music is a very good niche for testing and developing the capabilities of the Sparse Transformer.
Data collection and processing
We collected data for learning MuseNet in a variety of sources. We were greatly helped by the projects ClassicalArchives and BitMidi , which provided their colossal collections at our disposal. We also used collections of the most diverse music found on the Internet: jazz, pop, African, Indian and Arabic music. In addition, we used the MAESTRO collection.
Training takes place on serial data, taking into account the set of tokens, we ask the network to predict the next token. We tried several different ways to encode MIDI files. At first we thought about the chord approach, which looked at each combination of notes that sounded at the same time as a separate chord and assigned the corresponding token. We also tried to condense musical patterns, concentrating on the beginning of each note.
bach piano_strings start tempo90 piano:v72:G1 piano:v72:G2 piano:v72:B4 piano:v72:D4 violin:v80:G4 piano:v72:G4 piano:v72:B5 piano:v72:D5 wait:12 piano:v0:B5 wait:5 piano:v72:D5 wait:12 piano:v0:D5 wait:4 piano:v0:G1 piano:v0:G2 piano:v0:B4 piano:v0:D4 violin:v0:G4 piano:v0:G4 wait:1 piano:v72:G5 wait:12 piano:v0:G5 wait:5 piano:v72:D5 wait:12 piano:v0:D5 wait:5 piano:v72:B5 wait:12 During the training we:
- Moving the notes along the pitch scale (as the learning progresses, the number of transpositions decreases);
- We add and reduce both the overall sound and the sound of individual samples;
- We work with timing: we accelerate and slow down individual fragments of the composition;
- Use for work with tokens mixup .
In addition, we use something like a built-in music critic: we ask him to evaluate the composition and decide whether this sample is created from the data presented or is old. This assessment allows you to select samples in the course of generation.
Integration
To make the model have a more structural context, we added integration. In addition to standard positional integrations, we added integrations that track the passage of time in samples. This allows you to simultaneously and simultaneously build sounding notes. We also added two structural integrations that allow MuseNet to understand in which segment of the overall composition a particular sample should be located.
MuseNet is already giving concerts
Those interested can get acquainted with the work of MuseNet, neural network already even gives its own online concerts on Twitch . For those who are interested in the creation of Python neural networks and deep machine learning, we conduct an open webinar where we will talk about the course program and the skills that students will gain.
Source: https://habr.com/ru/post/451912/
All Articles