Adaptive Traffic Signal Control for large-scale scenario with Cooperative Group-based Multi-agent reinforcement learning

Abstract

Recent research reveals that reinforcement learning can potentially perform optimal decision-making compared to traditional methods like Adaptive Traffic Signal Control (ATSC). With the development of knowledge through trial and error, the Deep Reinforcement Learning (DRL) technique shows its feasibility for the intelligent traffic lights control. However, the general DRL algorithms cannot meet the demands of agents for coordination within large complex road networks. In this article, we introduce a new Cooperative Group-Based Multi-Agent reinforcement learning-ATSC (CGB-MATSC) framework. It is based on Cooperative Vehicle Infrastructure System (CVIS) to realize effective control in the large-scale road network. We propose a CGB-MAQL algorithm that applies k-nearest-neighbor-based state representation, pheromone-based regional green-wave control mode, and spatial discounted reward to stabilize the learning convergence. Extensive experiments and ablation studies of the CGB-MAQL algorithm show its effectiveness and scalability in the synthetic road network, Monaco city and Harbin city scenarios. Results demonstrate that compared with a set of general control methods, our algorithm can better control multiple intersection cases on congestion alleviation and environmental protection.