Cooperative Control for Multi-Intersection Traffic Signal Based on Deep Reinforcement Learning and Imitation Learning

Abstract

Traffic signal control has long been considered as a critical topic in intelligent transportation systems. Most existing related methods either suffer from inefficient training or mainly focus on isolated intersections. This article aims at the cooperative control for multi-intersection traffic signal, in which a novel end-to-end learning model is established and an efficient training method is proposed analogously, which is capable of adapting to large-scale scenarios. In the proposed method, the input traffic status in multi-intersection are expressed by a tensor without information loss, which significantly reduces model complexity than using a huge matrix, since additional convolutional layers can be required to extract features from a huge matrix. For the output, a multidimensional boolean vector is employed to simplify the control policy with abiding the practical phase changing rules, and then a multi-task learning structure is used to get the cooperative policy. Instead of only using the reinforcement learning to train the model, we employ imitation learning to integrate a rule based model to do the pre-training, which greatly accelerates the convergence. Afterwards, the reinforcement learning method is adopted to continue the fine training, where proximal policy optimization algorithm is incorporated to solve the policy collapse problem in multi-dimensional output situation. Numerical experiments demonstrate the distinctive advantages of the proposed method with comparison to the efficiency and accuracy of the related state-of-the-art methods.