Design of Obstacle Avoidance for Autonomous Vehicle Using Deep Q-Network and CARLA Simulator

Abstract

In this paper, we propose a deep Q-network (DQN) method to develop an autonomous vehicle control system to achieve trajectory design and collision avoidance with regard to obstacles on the road in a virtual environment. The intention of this work is to simulate a case scenario and train the DQN algorithm in a virtual environment before testing it in a real scenario in order to ensure safety while reducing costs. The CARLA simulator is used to emulate the motion of the autonomous vehicle in a virtual environment, including an obstacle vehicle parked on the road while the autonomous vehicle drives on the road. The target position, real-time position, velocity, and LiDAR point cloud information are taken as inputs, while action settings such as acceleration, braking, and steering are taken as outputs. The actions are sent to the torque control in the transmission system of the vehicle. A reward function is created using continuous equations designed, especially for this case, in order to imitate human driving behaviors. The results demonstrate that the proposed method can be used to navigate to the destination without collision with the obstacle, through the use of braking and dodging methods. Furthermore, according to the trend of DQN behavior, a better result can be obtained with an increased number of training episodes. This method is a non-global path planning method successfully implemented on a virtual environment platform, which is an advantage of this method over other autonomous vehicle designs, allowing for simulation testing and application with further experiments in future work.