Heterogeneous traffic intersections control design based on reinforcement learning

Abstract

AbstractTraffic light control is a cost‐effective method to alleviate traffic congestion and deep reinforcement learning (DRL) that is increasingly favored as a method for real‐time traffic light control. However, the complexities of modern urban intersections, including crossroads and T‐junctions, pose challenges for DRL‐based traffic light control systems that do not work well for such heterogeneous intersections. To address this problem, a Heterogeneous Advantage Actor‐Critic (HA2C) model is proposed to control traffic lights for heterogeneous intersections. First, HA2C employs an intersection structure transformation scheme to mask intersection heterogeneity. Second, it develops a two‐stage approach on top of an Advantage Actor‐Critic (A2C) reinforcement learning model to learn both general and structure‐specific policies, leading to more accurate decisions. The extensive simulations on both synthetic and real‐world maps demonstrate that HA2C outperforms the state‐of‐the‐art models in terms of higher throughput and faster travel time, while using a smaller model size in most scenarios.