Speed planning for connected and automated vehicles in urban scenarios using deep reinforcement learning

Abstract

This paper proposed a deep reinforcement learning based reference speed planning strategy to co-optimize the fuel economy, driving safety, and travel efficiency of connected and automated hybrid electric vehicles in urban scenarios. Thus, in this work, a connected traffic environment is first developed based on Simulation Urban Mobility (SUMO) to simulate real-world urban scenarios. Then a deep reinforcement learning agent is designed based on a twin delayed deep deterministic policy gradient algorithm (TD3). It can quickly solve the reference speed according to the ego vehicle state and traffic information. In addition, by developing a rule-based safety module and integrating it into the reward function, the TD3 agent can be facilitated to learn safety car-following policies and adhere to traffic light rules. Finally, simulation results indicate that the proposed strategy can efficiently and safely control the ego vehicles in complex urban scenarios with signalized intersections. And the fuel consumption of the proposed strategy is reduced by about 15% compared with that of the intelligent driver model based speed planning strategy.