Search for talent on Youtube using acoustic analysis
The Google Music cloud music service was presented to the public during the May 2011 Google I / O Conference. Unfortunately, it is only available from the US, although our people successfully go to the site through a proxy and even found a way to install the client for Android.
From the moment of the announcement, the service has become considerably more functional, and some features can be called unique. In June, the Instant Mix playlist generator (developed by Google Research) was added, which complements your playlist with new songs of the same style by analyzing the acoustic properties of the song: rhythm, tonality and instrumental coloring of the tracks, as well as the acoustic characteristics of the vocal. Instant Mix is ​​a key element of Google music service.
To evaluate the quality of Instant Mix work (and check the honest work of acoustic analysis), you can look at which tracks he selects for two songs with completely different moods, the same U2 group.
Here is the choice of the algorithm for the fun song Mysterious Ways .
')
1. U2 "Mysterious Ways"
2. David Bowie "Fame"
3. Oingo Boingo "Gratitude"
4. Infectious Grooves “Spreck”
5. Red Hot Chili Peppers “Special Secret Song Inside”
(listen)
But the choice for the song MLK c gentle vocals and a completely different set of tools.
1. U2 "MLK"
2. Jewel “Don't”
3. Antony and the Johnsons “What Can I Do?”
4. The Beatles “And I Love Her”
5. Van Morrison “Crazy Love”
(listen)
Representatives of Google Research initially stated that their goal was not to promote fashion groups, but to select songs from the “long tail”, that is, little-known teams. The main thing is the quality of music, because acoustic analysis works with objective physical characteristics and does not pay attention to brands. Instant Mix playlist generator can learn and explore the user's musical preferences.
And so the developers took the next logical step and applied the technology of acoustic analysis and machine learning to the entire sample of music videos on Youtube in order to try to identify potential “hits” among new, unknown artists.
First of all, they using selected image recognition technologies selected only those videos that were filmed at home, that is, home video. It was easy to do: the temperature of the color, the average or close-up of the person throughout the entire video clip (head and shoulders in the frame), the fixed position of the camera.
And then the most interesting part began - an acoustic analysis was used, similar to that used in the Instant Mix. To it was added the engine of recognition of the minimum number of user ratings (for self-study) - and after some time the technology was able to independently assess the quality of the video, anticipating the average user rating. In fact, the developers got a real technology of early recognition of hits that are popular with the mass audience. Perhaps, with its help, it will be possible to search in a huge number of amateur videotapes of performances of potential stars - and this will result in a “Star Factory” performed by a computer algorithm.
In order to get some benefit from the technology created, the developers launched the Youtube video game Slam , which includes the “best” video clips selected by the algorithm, and all users can vote for them. For several weeks of the game several quite interesting musicians have already been found. Some of these videos had less than 100 views when the algorithm selected them.
via Research Blog
From the moment of the announcement, the service has become considerably more functional, and some features can be called unique. In June, the Instant Mix playlist generator (developed by Google Research) was added, which complements your playlist with new songs of the same style by analyzing the acoustic properties of the song: rhythm, tonality and instrumental coloring of the tracks, as well as the acoustic characteristics of the vocal. Instant Mix is ​​a key element of Google music service.
To evaluate the quality of Instant Mix work (and check the honest work of acoustic analysis), you can look at which tracks he selects for two songs with completely different moods, the same U2 group.
Here is the choice of the algorithm for the fun song Mysterious Ways .
')
1. U2 "Mysterious Ways"
2. David Bowie "Fame"
3. Oingo Boingo "Gratitude"
4. Infectious Grooves “Spreck”
5. Red Hot Chili Peppers “Special Secret Song Inside”
(listen)
But the choice for the song MLK c gentle vocals and a completely different set of tools.
1. U2 "MLK"
2. Jewel “Don't”
3. Antony and the Johnsons “What Can I Do?”
4. The Beatles “And I Love Her”
5. Van Morrison “Crazy Love”
(listen)
Representatives of Google Research initially stated that their goal was not to promote fashion groups, but to select songs from the “long tail”, that is, little-known teams. The main thing is the quality of music, because acoustic analysis works with objective physical characteristics and does not pay attention to brands. Instant Mix playlist generator can learn and explore the user's musical preferences.
And so the developers took the next logical step and applied the technology of acoustic analysis and machine learning to the entire sample of music videos on Youtube in order to try to identify potential “hits” among new, unknown artists.
First of all, they using selected image recognition technologies selected only those videos that were filmed at home, that is, home video. It was easy to do: the temperature of the color, the average or close-up of the person throughout the entire video clip (head and shoulders in the frame), the fixed position of the camera.
And then the most interesting part began - an acoustic analysis was used, similar to that used in the Instant Mix. To it was added the engine of recognition of the minimum number of user ratings (for self-study) - and after some time the technology was able to independently assess the quality of the video, anticipating the average user rating. In fact, the developers got a real technology of early recognition of hits that are popular with the mass audience. Perhaps, with its help, it will be possible to search in a huge number of amateur videotapes of performances of potential stars - and this will result in a “Star Factory” performed by a computer algorithm.
In order to get some benefit from the technology created, the developers launched the Youtube video game Slam , which includes the “best” video clips selected by the algorithm, and all users can vote for them. For several weeks of the game several quite interesting musicians have already been found. Some of these videos had less than 100 views when the algorithm selected them.
via Research Blog
Source: https://habr.com/ru/post/132092/
All Articles