Adaptive Optimization of Traffic Signal Timing via Deep Reinforcement Learning

Jing Zhao,Zibo Ma,Liguo Zhang,Fengyao Jiang,Tongchao Cui,Wenxing Deng

doi:10.1155/2021/6616702

Jing Zhao, Zibo Ma + Show 4 more

Open Access

https://doi.org/10.1155/2021/6616702

Copy DOI

Abstract

With rapid development of the urbanization, how to improve the traffic lights efficiency has become an urgent issue. The traditional traffic light control is a method that calculates a series of corresponding timing parameters by optimizing the cycle length. However, fixing sequence and duration of traffic lights is inefficient for dynamic traffic flow regulation. In order to solve the above problem, this study proposes a traffic light timing optimization scheme based on deep reinforcement learning (DRL). In this scheme, the traffic lights can output an appropriate phase according to the traffic flow state of each direction at the intersection and dynamically adjust the phase length. Specifically, we first adopt Proximal Policy Optimization (PPO) to improve the convergence speed of the model. Then, we elaborate the design of state, action, and reward, with the vehicle state defined by Discrete Traffic State Encoding (DTSE) method. Finally, we conduct experiments on real traffic data via the traffic simulation platform SUMO. The results show that, compared to the traditional timing control, the proposed scheme can effectively reduce the waiting time of vehicles and queue length in various traffic flow modes.

Highlights

In recent years, the data processing capabilities of computers have been significantly improved, and more and more new reinforcement learning methods have been proposed by scholars
The experiment introduces the construction of the simulation environment of SUMO [22] traffic simulation software for traffic light control, including road model, traffic light configuration, and vehicle attributes in simulation. en, the experiment proves the effectiveness of the proposed scheme in this study and compares it with traditional timing control, which reveals the advantages of this method
System Framework. e ideal traffic light control should response dynamically to traffic flow and can adjust the output signal phase in real time [23]. is study proposes using the reinforcement learning (RL) method to learn from the traffic flow in all directions of the intersection and optimizes the phase time and sequence. e RL framework of this study is shown in Figure 1, which is mainly composed of two parts, namely, the agent and the environment. e environment part is simulated by traffic simulation software SUMO. e agent is built by a neural network and has the ability of perceiving the environment and output actions

Summary

Introduction

The data processing capabilities of computers have been significantly improved, and more and more new reinforcement learning methods have been proposed by scholars. Is study proposes using the RL method to learn from the traffic flow in all directions of the intersection and optimizes the phase time and sequence.

Results

Conclusion