Reinforcement Learning-Based Dynamic Zone Placement Variable Speed Limit Control for Mixed Traffic Flows Using Speed Transition Matrices for State Estimation

Abstract

Current transport infrastructure and traffic management systems are overburdened due to the increasing demand for road capacity, which often leads to congestion. Building more infrastructure is not always a practical strategy to increase road capacity. Therefore, services from Intelligent Transportation Systems (ITSs) are commonly applied to increase the level of service. The growth of connected and autonomous vehicles (CAVs) brings new opportunities to the traffic management system. One of those approaches is Variable Speed Limit (VSL) control, and in this paper a VSL based on Q-Learning (QL) using CAVs as mobile sensors and actuators in combination with Speed Transition Matrices (STMs) for state estimation is developed and examined. The proposed Dynamic STM-QL-VSL (STM-QL-DVSL) algorithm was evaluated in seven traffic scenarios with CAV penetration rates ranging from 10% to 100%. The proposed STM-QL-DVSL algorithm utilizes two sets of actions that include dynamic speed limit zone positions and computed speed limits. The proposed algorithm was compared to no control, rule-based VSL, and two STM-QL-VSL configurations with fixed VSL zones. The developed STM-QL-DVSL outperformed all other control strategies and improved measured macroscopic traffic parameters like Total Time Spent (TTS) and Mean Travel Time (MTT) by learning the control policy for each simulated scenario.