Distributed Signal Control of Arterial Corridors Using Multi-Agent Deep Reinforcement Learning

Abstract

Traffic congestion at signalized intersections often leads to serious impacts on adjacent intersections on a corridor. To enhance intersections’ throughput efficiency, traffic signals are commonly coordinated across intersections. Traditional signal coordination methods control the adjacent intersections by setting a fixed phase offset. However, these traditional coordination methods may have poor adaptability to dynamic traffic conditions, which can cause additional congestion. To reduce arterial traffic delays, this paper develops an adaptive coordination control method based on multi-agent reinforcement learning (MARL). Most existing MARL-based methods rely on impractical assumptions to improve their performance in complex and dynamic traffic scenarios. To overcome these assumptions, this paper proposes a fully scalable MARL algorithm for arterial traffic signal coordination based on the proximal policy optimization algorithm. We apply a parameter-sharing training protocol to mitigate the slow convergence due to nonstationarity and to reduce computational requirements. In addition, a new action setting is designed by using the lead-lag phase sequence to simultaneously improve the implementation and coordination flexibility of the method. Extensive simulation experiments and comparisons with existing methods demonstrate that the proposed method performed stably in both simulated and real-world arterial corridors. Hence, the proposed signal coordination method can alleviate traffic congestion more effectively than existing traditional and MARL-based methods.