Flappy Bird is a game that many have played, yet very few consider themselves great at it. From my experience of playing, achieving a score of even passing five pipes is an accomplishment.
For those who are unfamiliar, Flappy Bird is a popular mobile game released in 2013. The premise of the game is a bundle of simple, frustrating, and addictive. Tapping on the screen helps your bird flap its wings and gain altitude. The objective is to navigate the bird through an endless series of pipes that appear from the right side of the screen. Hitting a pipe sends your bird plunging to the ground resulting in a game over.
 |
|---|
| I swear it's much harder than it appears. |
Whenever I think I'm close to breaking out a high score, I usually end up clipping a corner of one of those pesky pipes. I can always chalk up my shortcomings to not having a fast enough reaction time or not having the focus to play long enough. I'm only human after all, I'm not perfect.
That's where it all began...
Overview
This project uses the NeuroEvolution of Augmenting Topologies (NEAT) algorithm to train Intelligence Agents (IA) to play the game of Flappy Bird. IAs go through numerous rounds of training to determine how they will respond in the future (this is covered in further detail later). This project relies on the Python NEAT library, neat-python.
This project is heavily influenced by Danny Zhu's Medium Post and Tech with Tim's Flappy Bird Playlist. I utilized many of their ideas and implemented a greater object-oriented approach in the overall design of the project.
What's NEAT?
 |
|---|
| Credit: https://github.com/suhdonghwi/neat |
NEAT is an algorithm built on the fundamental concepts of the Theory of Evolution discovered by Charles Darwin. Survival of the Fittest! The fittest agent is the one that passes its genes to the next generation. Each round of training starts with a set number of individuals competing against each other. Their performances are then calculated using a fitness function (essentially a final score), in the case of Flappy Bird this will be either the number of pipes the bird flew through or how long it survived.
In NEAT, each agent's genes are represented by a network. These networks consist of interconnected nodes and edges. Input nodes receive information from the environment. For example, in the case of Flappy Bird, it could be several inputs: distance from the ground, the top or bottom pipe, velocity, acceleration, etc. Output nodes, on the other hand, determine the agent's actions, such as flapping up or down. Edges connect nodes, transmitting signals that are modified by weights. These weights decide which output nodes activate and which do not, essentially deciding the agent's behavior based on the received information.
Each agent's performance after each round of training determines whether its genes are passed to the next round. Agents with superior performance reproduce AKA have their network structures and weights (their genes) copied with some variations. These variations introduce evolution into the system and allow NEAT to gradually refine its agents' behavior over multiple generations.
For a more in-depth dive into NEAT, than I could ever do, I recommend watching this video by Connor Shorten:
Creating Flappy Bird
To start creating my artificial intelligence I needed the game Flappy Bird. While there are approaches to implementing a Flappy Bird AI using computer vision with a version of Flappy Bird you or I would play, that seemed out of the scope of the project. Instead, I created a version of Flappy Bird using the pygame Python package. However, in this version of Flappy Bird, hundreds of birds compete at once for later training purposes.
 |
|---|
| This is my testing it prior to AI implementation. Even with 100 birds I am still not any good. |
Design
I designed my Flappy Bird clone using an object-oriented approach. This meant all game objects you can see on the screen are objects in the code, i.e. the pipes, ground, and birds. All these objects provide APIs to receive and send game data, i.e. get the position, determine if objects are colliding, and message to flap (in the case of the bird).
This object-orient approach allowed me to create two kinds of Bird subclasses: user-controlled and NEAT-controlled. The user-controlled Bird was the object I showed earlier with one hundred birds. This object was created purely for testing purposes on my part to ensure the game worked correctly for the later NEAT training. Every time frame the game checks if the space bar is pressed, if so the bird flaps.
The NEAT-controlled Bird object is controlled by the AI network. Instead of flapping on user input, the bird asks the network based on its current distance horizontally from the pipes and vertically from the top and bottom pipe, whether to flap or not. As one would imagine, this approach does not work instantly.
Training the AI
One hundred independent NEAT networks that each control a bird compete on who will receive the longest time in flight. After each round, the best of the population will reproduce with some minor alterations as the worst of the group dies out. This is what the first generation of training amounted to.
 |
|---|
| They struggle a lot a first. |
The second generation is much the same.
 |
|---|
| They seem to show a little bit of improvement. |
By the third generation, the best network finds the secret to this simple game.
 |
|---|
| Already magnitudes better than I am! |
Challenging the AI further
The AI proved this game was far too simple for it. Let's speed the game up for every pipe the birds pass. Changing the speed on the birds, however, won't work off the bat. The birds currently take 3 inputs: the relative horizontal distance and vertical distances from the top and bottom pipes. These inputs do not provide the birds enough data to account for speed changes.
The new bird NEAT network takes in 6 inputs: the previous 3, the distance the bird is vertically above the ground, the velocity of the pipes, and the vertical velocity of the bird. After performing a much longer 50 generations of training, this is how the best bird performed.
