A Novel Reinforcement Learning-Based Cooperative Traffic Signal System Through Max-Pressure Control

Abstract

Improving the efficiency of traffic signal control is an effective way to alleviate traffic congestion at signalized intersections. To achieve effective management of the system-wide traffic flows, current research tends to focus on applying reinforcement learning (RL) techniques for collaborative traffic signal control in a traffic road network. However, the existing collaboration-based methods often ignore the impact of transmission delay for exchanging traffic flow information on the system. Most of the studies assume that the signal controllers can collect all instantaneous vehicular features without delay. To fill the gap, we propose an RL-based cooperative traffic signal control scheme considering the data transmission delay issue in a traffic road network. In this paper, we (1) design our new RL agents to cooperatively control the traffic signals by improving the reward and state representation based on the state-of-the-art max-pressure control theory; (2) propose a traffic state prediction method to address the data transmission delay issue by decreasing the discrepancy between the real-time and delayed traffic conditions; (3) evaluated the performance of our proposed work on both synthetic and real-world scenarios with a different range of data transmission delays. The results demonstrate that our method surpassed the performance of the previous max-pressure-based traffic signal control methods and addressed the data transmission delay issue.