Perimeter control with heterogeneous metering rates for cordon signals: A physics-regularized multi-agent reinforcement learning approach

Abstract

Listen

Translate article

Perimeter Control (PC) strategies have been proposed to address urban road network control in oversaturated situations by regulating the transfer flow of the Protected Network (PN) based on the Macroscopic Fundamental Diagram (MFD). The uniform metering rate for cordon signals in most existing studies overlooks the variance of local traffic states at the intersection level, which may cause severe local traffic congestion and degradation of the network stability. PC strategies with heterogeneous metering rates for cordon signals allow precise control for the perimeter but the complexity of the problem increases exponentially with the scale of the PN. This paper leverages a Multi-Agent Reinforcement Learning (MARL)-based traffic signal control framework to decompose this PC problem, which considers heterogeneous metering rates for cordon signals, into multi-agent cooperation tasks. Each agent controls an individual signal located in the cordon, decreasing the dimension of action space for the controller compared to centralized methods. A physics regularization approach for the MARL framework is proposed to ensure the distributed cordon signal controllers are aware of the global network state by encoding MFD-based knowledge into the action-value functions of the local agents. The proposed PC strategy detects the overall traffic state within the PN and distributes local instructions to cordon signal controllers in the MARL framework via the physics regularization. Through numerical tests in a microscopic traffic environment, the proposed PC strategy shows promising robustness on different demand patterns and transferability by inheriting the trained models to several distinct signals without retraining. It outperforms state-of-the-art feedback PC strategies and centrliazed RL methods in increasing network throughput, decreasing distributed delay for gate links, and reducing carbon emissions.