Traffic Signal Control Based on Deep Reinforcement Learning with Simplified State and Reward Definitions

Abstract

Traffic congestion has recently become a real issue especially within crowded cities and urban areas. Intelligent transportation systems (ITS) leveraged various advanced techniques aiming to optimize the traffic flow and subsequently alleviate the traffic congestion. In particular, traffic signal control TSC is one of the essential ITS techniques for controlling the traffic flow at intersections. Many research works have been proposed to develop algorithms and techniques which optimize TSC behavior. Recent works leverage Deep Learning (DL) and Reinforcement Learning (RL) techniques to optimize TSCs. However, most of Deep RL proposals are based on complex definitions of state and reward in the RL framework. In this work, we propose to use an alternative way of formulating the state and reward definitions. Basically, The basic idea is to define both state and reward in a simplified and straightforward manner rather than the complex design. We hypothesize that such a design approach simplifies the learning of the RL agent and hence provides a rapid convergence to optimal policies. For the agent architecture, we employ the double deep Q-Network (DDQN) along with prioritized experience replay (PER). We conduct the experiments using the Simulation of Urban MObility (SUMO) simulator interfaced with Python framework and we compare the performance of our proposal to traditional and learning-based techniques.