Improving traffic signal control operations using proximal policy optimization

Abstract

AbstractExisting traffic signal control systems present many limitations including fixed signal timing schemes, insufficient efficiency and flexibility, and difficulty in adapting to the changing traffic flows. In recent years, the development of deep reinforcement learning (RL) has shown great research potential and application prospects. This paper proposes an intersection signal control method based on the proximal policy optimization (PPO) method. Specifically, this paper uses the value vector representation method of traffic characteristics to encode the traffic state and then feeds the state encoding into the long short‐term memory (LSTM) network to obtain the signal phase output. Finally, to obtain the optimal signal control strategy, the PPO algorithm is utilized to train the neural network and adjust the signal phase. The proposed algorithm is benchmarked against other classic RL and adaptive signal control schemes in an experimental environment based on the traffic simulation software simulation of urban mobility (SUMO). Experimental results showed that the proposed algorithm greatly improves traffic efficiency. Specifically, the mean velocity of the vehicles increases by 45.09%, and the mean occupancy rate of each lane, the length of the longest jams during each step, and the mean halting duration dropped by 21.38%, 25.86%, and 12.94%, respectively.