A Deep Reinforcement Learning Based Ramp Metering Control Method Considering Ramp Outflow

Abstract

Ramp metering is an effective measure to address highway traffic congestion, but traditional methods often struggle with peak periods and extreme scenarios like traffic accidents. This paper introduces deep reinforcement learning for ramp metering to tackle congestion in high-traffic scenarios. For single-entrance ramp scenarios, this paper proposes the DQNOP algorithm which combines three weighted reward functions to achieve multiple objectives. Additionally, an Overflow Protection (OP) module is designed to adaptively address ramp overflow issues. Then, the DQN-OP algorithm is extended to multi-entrance ramp scenarios, and the Shared State Independent Reward (SSIR) mechanism is introduced, leading to the IQL-SSIR algorithm. Experimental results show that the proposed DQN-OP and IQL-SSIR algorithms both outperform traditional algorithms. Specifically, the DQN-OP algorithm achieves approximately a 12% improvement over traditional algorithms, while the IQL-SSIR algorithm achieves approximately a 5% improvement.