Robustness Analysis of Discrete State-Based Reinforcement Learning Models in Traffic Signal Control

Abstract

With the growing traffic congestion problem, more and more deep reinforcement learning (DRL) methods have been applied in traffic signals control(TSC). But researches show that DRL is very fragile with abnormal data. In this paper, special traffic state abnormal data (TSAD) are simulated, based on which the robustness of DRL is analyzed and improved for traffic signals control. Firstly, the perturbation noise is generated based on the Discrete Carlin&Wagner attack, which is then added to the normal data to simulate the TSAD. Secondly, under different type of TSAD, the robustness of DRL models for traffic signals control is explored, which are demonstrated to have certain vulnerability, especially with high traffic flows. Finally, induction model based on reward detection(IMR) and mask the activation values of decision neurons (MVN) are proposed to effectively improve the robustness of DRL models for traffic signals control.