Road Model Design Based on Reward Function in Traffic Light Control

Abstract

Traffic congestion has become serious across the world since the recent two centuries. Among existing control methods, Deep Q-learning Network (DQN) based reinforcement learning achieved superior performance over state-of-art methods. Reward function is a crucial part of reinforcement learning. Different terms in reward function guide the agent attaining different goals. In traffic light control problem, the reward function could be either too simple to perform well or too complicated to find the best parameters. It influents the performance of the algorithm and bring the challenge in the standardized design of the control algorithm. Amid at this problem, we proposed a road model based reward function and applied it in DQN algorithm. The road structure was introduced and modelled, then from this model we then took factors which can represent different car flow and different road situation. Also, they were easily obtained from the real world. Those factors were applied to design the reward function. Experiments with simulation dataset showed the efficiency of the road model based reward function in the DQN algorithm.