A heuristic-action-involved safe lane-change of autonomous vehicles with multiple constraints on road centerline and speed under highway environment

Abstract

Lane-change (LC) is one of the most important topics in autonomous vehicles (AVs) on highways. To enhance the implementation of effective LC in AVs, this paper proposes a framework based on deep reinforcement learning, which takes into account heuristic actions and multiple constraints related to the centerline of the road and speed, to improve the overall performance of LC in AVs. Firstly, the influence of unreasonable vehicle actions on the algorithm training process is studied. To improve the rationality of the to-be-trained actions, a novel reasonable action screening mechanism is proposed. Secondly, to keep the vehicle on the centerline of the lane and avoid the collision with other vehicles, a method is designed to calculate the center position of the vehicle. Thirdly, a segmented speed reward mechanism is proposed to constrain vehicle speed. Subsequently, a dynamic reward function is established to train the control algorithm. Lastly, the proposed strategy is evaluated in two simulation scenarios of highways. The simulation results show that the proposed method can increase the number of reasonable actions by more than 30% and improve the success rate of obstacle avoidance with the increase of over 52% in both static and dynamic scenarios compared with the benchmark algorithms.