Synchronization of rhythm in music games

Recently, I began work in Unity on the beat-box music game Boots-Cut . In the process of prototyping the basic mechanics of the game, I discovered that it is quite difficult to correctly synchronize notes with music. There are quite a few articles on this topic on the Internet. Therefore, in my article I will try to give the most important tips for developing a music game (especially in Unity).

It turned out that the most important are the following three aspects:

• Use AudioSettings.dspTime instead of Time.timeSinceLevelLoad to track the position in a song.
• You must always use the position in the song to update the movements.
• Do not update the notes in each frame for the time difference, interpolate them.

We take this into account and get down to work!
')

Main class

You must create a SongManager class to track the position in a song, create notes, and other song management functions.

Position Tracking

In all music games, you need to track the position in the song in order to know which note to create. Below are the fields required to track the position in the song:

 //    ( ) float songPosition; //    ( ) float songPosInBeats; //  float secPerBeat; //  ( )     float dsptimesong; 

We initialize these fields in the Start() function:

 void Start() { //      // bpm   secPerBeat = 60f / bpm; //    dsptimesong = (float) AudioSettings.dspTime; //  GetComponent<AudioSource>().Play(); } 

For convenience, we convert bpm to secPerBeat . Later secPerBeat will be used to calculate the position in the song in the beats, which is very important for creating notes.

In addition, we record the start time of the song in dsptimesong . We use AudioSettings.dspTime instead of Time.timeSinceLevelLoad , because Time.timeSinceLevelLoad updated only in each frame, and AudioSettings.dspTime updated more often, as this is an audio system timer. To keep track of the song, you need to use the audio timer. In this way, we can avoid the delay caused by the time difference between frame updates and audio updates.

The Update() function calculates the position in a song using AudioSettings.dspTime :

 void Update() { //    songPosition = (float) (AudioSettings.dspTime - dsptimesong); //    songPosInBeats = songPosition / secPerBeat; } 

We calculate the position in seconds by subtracting the song start time ( dsptimesong ) from the current AudioSettings.dspTime . We got the position in seconds, but in the music world the notes are recorded in beats. Therefore, it is better to convert the position in seconds to position in beats. Dividing the songPosition by secPerBeat (second / (second / hit)), we get the position in beats.

Look at the picture:



The position of the notes in the beats is 1, 2, 2.5, 3, 3.5, 4.5, and the duration of the beat is 0.5 s. Therefore, if after the beginning of the song 1.75 seconds passed ( songPosition == 1.75 ), then we know that we are in the position 1.75 ( songPosition ) / 0.5 ( secPerBeat ) = 3.5 beats, and it is necessary to create a beat note 3.5.

Song info

Let's go to the fields in which we recorded information about the song:

 //    float bpm; //      float[] notes; // ,     int nextIndex = 0; 

For simplicity, I show a song with only one track of notes (three tracks are made in Guitar Hero Mobile , and only one track in Taikono Tatsujin ).

bpm is the number of beats per minute. As we have seen, for convenience they are converted to secPerBeat .

notes is an array in which all positions of notes in beats are stored. For example, for the notes presented in the figure, the notes array will contain {1f, 2f, 2.5f, 3f, 3.5f, 4.5f} :



And finally, nextIndex is an integer needed to traverse the array. It is initialized to 0, because the next note to be created will be the first note of the song. When creating a note, the counter nextIndex incremented by one.

Making notes

We determine whether a note should be created in the Update() function. However, you first need to determine how many strokes will be shown in advance.

For example, for the following track:



The current position in beats is 1, but hit 3 has already been created. This means that 3 hits are shown in advance.

Add under songPosInBeats = songPosition / secPerBeat; , the following lines:

 if (nextIndex < notes.Length && notes[nextIndex] < songPosInBeats + beatsShownInAdvance) { Instantiate( /*   */ ); //   nextIndex++; } 

First you need to check if there are any notes left in the song ( nextIndex < notes.Length ). If the notes still remain, then we check if the song reached the beat at which the next note should be created ( notes[nextIndex] < songPosInBeats + beatsShownInAdvance ). If reached, create a note and increment nextIndex to keep track of the next note you want to create.

Movement notes

Finally, let's talk about how to move the created notes in accordance with the tempo of the song. It's quite simple if you remember the item “Do not update the notes in each frame for the time difference, interpolate them”.

Always update the movement of the position in the song, because:

1. Audio system timer has a time difference with frame timer
2. Strikes can be exactly in the middle of two frames (which leads to the difference in time)

So how do you move notes? Interpolation!

For simplicity, I will cut out all the code in the MusicNote class and leave only the Update() function, in which we move every note:

 //   void Update() { transform.position = Vector2.Lerp( SpawnPos, RemovePos, (BeatsShownInAdvance - (beatOfThisNote - songPosInBeats)) / BeatsShownInAdvance ); } 

The diagram below shows this clearly:

Conclusion

I talked about the basics of programming a music game. Following these principles, you can create games with synchronization. In games with multiple tracks, you can create nested notes arrays, deleting notes is performed by checking the position relative to the deletion line, long-duration notes are implemented by tracking the initial and final stroke, etc.

Thank you for reading the article, I hope it will be useful. My own music game Boots-Cuts will be ready next year, stay tuned.