Curriculum based Reinforcement Learning for traffic simulations

Abstract

Non-Playable Characters (NPCs) are essential for creating a fully immersive video game experience. However, their behaviours are typically programmed using rigid methods such as Finite State Machines and Behaviour Trees. These methods are hard to maintain, struggle with implementing intricate cooperative behaviours, and patterns in NPC behaviour can easily be exploited by human players. Machine Learning and Reinforcement Learning (RL) methods in particular, offer a solution to these challenges by allowing dynamic and real-time NPC responses to human player actions and other characters in the game. This work describes and extends upon a Curriculum-based RL approach to train a single model for multiple agents representing vehicles to respect traffic regulations. These vehicles learn to navigate a road network which includes a traffic lights crossroad intersection and a roundabout junction. A comparison of our simulated data against datasets obtained from real world data is conducted; results demonstrate that the learned models have similar trends to the real-world data.