ES-DQN: A Learning Method for Vehicle Intelligent Speed Control Strategy Under Uncertain Cut-In Scenario

Abstract

Uncertain cut-in maneuver of vehicles from adjacent lanes makes it difficult for vehicle's automatic speed control strategy to make judgments and effective control decisions. In this paper, an intelligent speed control strategy for uncertain cut-in scenarios is established based on a basic autonomous driving system. This strategy judges cut-in maneuver from surrounding vehicles and outputs adaptive control action under current environment according to Q value of state-action pair based on a Q network. In addition, according to the analysis of cut-in scenarios, the Q network is trained based on a novel reinforcement learning method named as experience screening deep Q-learning network (ES-DQN). The proposed ES-DQN is an extension of double deep Q-learning network (DDQN) algorithm, and includes two parts: experience screening and policy learning. Based on the experience screened from the experience screening part, the proposed learning method can train an intelligent speed control strategy which has stronger adaptability and control effect in uncertain cut-in scenarios. According to simulation results, the proposed intelligent speed control strategy trained by ES-DQN has better performance under uncertain cut-in scenarios than DDQN method and traditional ACC strategy. Meanwhile, by adjusting weight value in reward function, the system can realize different control target.