CycLight: Learning traffic signal cooperation with a cycle-level strategy

Abstract

This study introduces CycLight, a novel cycle-level deep reinforcement learning (RL) approach for network-level adaptive traffic signal control (NATSC) systems. Traditional RL-based step-by-step traffic signal controllers, while precise and responsive, demand high computational resources for frequent data exchanges and can increase traffic accident risks due to disordered phase switching. Different from the step-by-step control strategy aforementioned, the proposed CycLight adopts a cycle-level strategy, optimizing cycle length and splits simultaneously using Parameterized Deep Q-Networks (PDQN) algorithm. This cycle-level approach effectively reduces the computational burden associated with frequent data communication, meanwhile enhancing the practicality and safety of real-world applications. Moreover, a decentralized framework is formulated for multi-agent cooperation, while attention mechanism is integrated to accurately assess the impact of the surroundings on the current intersection. CycLight is tested in a large synthetic traffic grid using the microscopic traffic simulation tool, Simulation of Urban MObility (SUMO). Experimental results not only demonstrate the superiority of CycLight over other state-of-the-art approaches but also showcase its robustness against information transmission delays.