Using machine learning to play Flappy Bird

This is a project that automatically learns to play Flappy Bird using machine learning, with the learning method being Q Learning.

Creating the Flappy Bird game using Phaser.js, as shown below

The key lies in this formula

Encountered some difficulties when initially training using this formula

When using only these two state spaces, where QState is a two-dimensional space

It caused the obstacle at the lower point to not know the distance from the ground or the sky and often exceeded the boundaries

So I added another state space, the distance to the sky

But this led to other problems, when passing through bricks at a normal speed, theoretically it should act like this

The position of the red dot will gradually train to have a higher Q Value when not pressing compared to pressing

But in this scenario

Due to the rapid descent speed, it resulted in the Q Value training to require pressing to avoid hitting the bricks

Because these two states cannot converge to the correct position, but converge to other positions

So we must add another state space for the speed space

Basically, this should complete the training.