A Deep Reinforcement Learning Approach for Automated On-Ramp Merging

Abstract

Safe and efficient control for autonomous vehicles (AVs) at on-ramp merging sections is challenging. Previous work about on-ramp merging focuses on discrete behavioral decisions and longitudinal motion control, while the lane-changing (LC) behavior is usually oversimplified. To bridge the research gaps, an on-ramp merging optimization control framework (ORMOC) based on a deep reinforcement learning (DRL) approach is proposed in the paper, which jointly optimizes the control of lane-keeping (LK) and LC. In the framework, firstly, a LC agent based on DRL and quintic polynomial-based trajectory model is trained to generate a smooth LC trajectory. Then, a LK agent combined with the LC agent is trained by sharing learning, for longitudinal motion control LC decision execution. Finally, a priority-based safety supervisor is developed to enhance the safety of the control output. The proposed method is examined using SUMO simulation experiments, and the results demonstrate that compared to the TTC (time to collision)-based method, the proposed DRL-ORMOC framework optimizes the travel time by 8.3%.