Exploring Reinforcement Learning: A Case Study Applied to the Popular Snake Game

Abstract

Abstract Reinforcement Learning is a machine learning approach in which an agent interacts with their environment to gather information, and make an informed decision based on the accumulated information. In this research, we investigate the applicability of various reinforcement learning techniques for Snake, a video game popular on the Nokia 3310 mobile phone. Q-Learning (Quality-Learning), SARSA (State-Action Reward State-Action) and PPO (Proximal Policy Optimization), were implemented and evaluated for Snake. Q-Learning and SARSA did not generate optimal results due to the large environment of the game. Meanwhile, PPO was implemented with three varying approaches for input; a vector, CNN and raycasting based approach. PPO, in conjunction with raycasting, resulted in the best performance, with the snake agent learning for both collecting food and avoiding obstacles. Furthermore, A* Pathfinding was tested and it achieved a performance better than Q-Learning and SARSA but not better than PPO as it was less adaptable to large environments. In the future, agents in large dynamic game environments, may benefit further from utilizing PPO. KeywordsReinforcement learningQ-learningSARSAPPO