Flappy Bird Machine Learning Algorithm

I will introduce you to the full HTML5 tutorial on the Flappy Bird video game machine learning demo. The purpose of this experiment is to write an artificial intelligence game controller based on neural networks and a genetic algorithm .

That is, we want to create an AI robot that can learn the best game in Flappy Bird. As a result, our little bird will be able to safely fly over obstacles. In the best scenario, she will never die.

After reading the theory behind this project, you can download the source code at the end of this tutorial. All code is written in HTML5 using the Phaser framework . In addition, we used the Synaptic Neural Network library to implement a neural network so as not to create it from scratch.
')

Demo

To get started, look at the demo to evaluate the algorithm in action:



Run in full screen

Video presentation

In addition to the demo, you can watch a short video with a simple explanation of the algorithm. Will appeal to those who like to receive information quickly!

What is a machine learning algorithm?

According to the wording of Arthur Samuel in 1959, machine learning is a way to make computers work without programming explicitly. In general, this is a process of fine tuning training, gradually improving the original random system.

That is, the goal here is to create artificial intelligence that can find the right solution from a bad system by fine tuning the model parameters. For this, the machine learning algorithm uses many different approaches.

Specifically, in this project, the basic approach to machine learning (machine learning algorithm, ML) is based on neuroevolution. In this form of machine learning, evolutionary algorithms, such as the genetic algorithm (GA), are used to train artificial neural networks (ANN).

That is, in our case, we can say that ML = GA + ANN.

Artificial Neural Network

An artificial neural network is a subset of the machine learning algorithm. It is based on the structure and functions of biological neural networks. These networks are created from a variety of neurons that transmit signals to each other.

That is, to create an artificial brain, we need to simulate neurons and connect them so that they form a neural network.

A standard artificial neural network consists of an input data layer, one or more hidden layers and an output data layer. Each layer has several neurons. The input and output neurons are connected directly to the external environment. Hidden neurons connect between them.

In this project, each object (bird) has its own neural network, used as an AI brain for passing the game. It consists of the following three layers:

1. The two-neuronal input data layer represents what the bird sees:
   
   • horizontal distance to the nearest gap
   • height difference with the closest gap
2. hidden layer with six neurons
3. output layer with one neuron, creating the action:
   • if the output is> 0.5, then make a dash, otherwise do nothing

The figure below shows the neural network architecture for this example:

Genetic algorithm

When we discussed the machine learning algorithm, we said that a genetic algorithm is used to train and improve neural networks.

A genetic algorithm is a search-based optimization technique that replicates natural selection and genetics. It uses the same combination of selection, crossover, and mutations to change the initial random population.

Here are the main steps in the implementation of our genetic algorithm:

1. create an initial population of 10 objects (birds) with random neural networks
2. letting all objects play simultaneously using their own neural networks
3. for each object, we calculate its fitness function to assess its quality (for more details, see the Fitness Function section)
4. after the death of all objects, we estimate the current generation to create a new one with the help of genetic operators (for more details, see the substitution strategy
5. back to stage 2

Fitness function

In addition to the genetic algorithm (step 3), we will take a closer look at the fitness function - what it is and how to define it.

Since we want the population to evolve from the best objects, we need to determine the fitness function.

In general, the fitness function is a metric that measures the quality of an object. If we have the metrics of the quality of each bird, we will be able to choose the most adapted objects and use them to recreate the next generation.

In this project we reward the bird in direct proportion to the distance done. In addition, we punish it by the current distance to the nearest gap. In this way we will be able to distinguish between birds flying the same distance.

To summarize: our function of fitness is the difference between the total distance made by the bird and the current distance to the nearest gap.

Replacement strategy

In addition to the genetic algorithm (step 4), these are the stages of applying natural evolution to the dying generation. In essence, the best objects survive, and their descendants replace the worst objects as follows:

1. sort the objects of the current generation according to their fitness level
2. choose the four best objects (winners) and pass them directly to the next generation
3. create one descendant as a result of crossing over between the two best winners
4. create three descendants as the crossover results of two random winners
5. create two descendants as direct copies of two random winners
6. we apply random mutations to each descendant to add variations

Source

And finally, the link for downloading the source code:

https://github.com/ssusnic/Machine-Learning-Flappy-Bird

Flappy Bird Machine Learning Algorithm - Conclusion

In this tutorial, we successfully implemented an AI robot to learn how to play Flappy Bird. As a result of several iterations, we can get an almost invulnerable player. To achieve this goal, we used two approaches to machine learning algorithms: artificial neural networks and a genetic algorithm.

For the experiment, you can try changing some of the parameters in the code and see what happens. For example, you can change the number of neurons in a hidden layer or the number of objects in a population. In addition, you can try to change the function of fitness. Moreover, try changing some physical parameters - the distance between obstacles, gravity, and so on.

Try also to apply the same approach to evolution in other games.