Dynamic Scene Path Planning of UAVs Based on Deep Reinforcement Learning

Abstract

Traditional unmanned aerial vehicle path planning methods focus on addressing planning issues in static scenes, struggle to balance optimality and real-time performance, and are prone to local optima. In this paper, we propose an improved deep reinforcement learning approach for UAV path planning in dynamic scenarios. Firstly, we establish a task scenario including an obstacle assessment model and model the UAV’s path planning problem using the Markov Decision Process. We translate the MDP model into the framework of reinforcement learning and design the state space, action space, and reward function while incorporating heuristic rules into the action exploration policy. Secondly, we utilize the Q function approximation of an enhanced D3QN with a prioritized experience replay mechanism and design the algorithm’s network structure based on the TensorFlow framework. Through extensive training, we obtain reinforcement learning path planning policies for both static and dynamic scenes and innovatively employ a visualized action field to analyze their planning effectiveness. Simulations demonstrate that the proposed algorithm can accomplish UAV dynamic scene path planning tasks and outperforms classical methods such as A*, RRT, and DQN in terms of planning effectiveness.