A Deep Reinforcement Learning Approach to Traffic Signal Control With Temporal Traffic Pattern Mining

Abstract

Traffic signal control is a critical method that ensures the efficiency of traffic flow in cities across the world. There are massive studies that focus on generating optimal signal timing plans. Most of the these studies are model-based, where the signal plan is determined by optimization models with fixed parameters. Reinforcement learning (RL) is a model-free method that learns the optimal control policy over time which avoids limitations of traditional methods. In the past a few years, deep RL methods become particularly attractive since deep neural networks can provide scalable learning capability given the complex traffic condition in real life. In this manuscript, we develop a deep actor-critic method that can provide efficient traffic signal plans. First, a novel deep neural network model is applied to mine the recent traffic condition information as a sequence of temporally sequential image representations of the intersection, which improves from relying solely on limited traffic information such as queue length; then the deep neural network model is integrated with an actor-critic model which avoids drawbacks from the value-based or the policy-based methods. Simulation experiments illustrate that the deep actor-critic method outperforms the classic model-based method and several existing deep reinforcement learning methods according to different measures such as queue length, average delay time, and throughput.