Trajectory Planning of UAV in Unknown Dynamic Environment with Deep Reinforcement Learning

Abstract

Providing a collision-free, safe and efficient optimal trajectory for unmanned aerial vehicles (UAVs) in an unknown dynamic environment is one of the most important issues for researchers. In this paper, a trajectory planning approach for UAV in unknown dynamic environment based on deep reinforcement learning (DRL) is proposed. This study models trajectory planning of UAV as a discrete-time, discrete-action problem, and then proposes an improved deep Q network (IDQN) algorithm to solve it. The IDQN algorithm adds the track angle information of UAV to the reward function to speed up the learning process, furthermore, it also improves the action selection strategy and learning rate setting. Besides, in simulation, the paper considers the trajectory constraints of UAV in order to make the obtained trajectory have better practical availability. Simulation results demonstrate the effectiveness of the IDQN algorithm to implement UAV trajectory planning with constraints in unknown dynamic environments. Meanwhile, comparison with the classical DQN (CDQN) algorithm is conducted to further explore the advantage of the method.