Autonomous obstacle avoidance of UAV based on deep reinforcement learning1

Abstract

In the intelligent unmanned systems, unmanned aerial vehicle (UAV) obstacle avoidance technology is the core and primary condition. Traditional algorithms are not suitable for obstacle avoidance in complex and changeable environments based on the limited sensors on UAVs. In this article, we use an end-to-end deep reinforcement learning (DRL) algorithm to achieve the UAV autonomously avoid obstacles. For the problem of slow convergence in DRL, a Multi-Branch (MB) network structure is proposed to ensure that the algorithm can get good performance in the early stage; for non-optimal decision-making problems caused by overestimation, the Revise Q-value (RQ) algorithm is proposed to ensure that the agent can choose the optimal strategy for obstacle avoidance. According to the flying characteristics of the rotor UAV, we build a V-Rep 3D physical simulation environment to test the obstacle avoidance performance. And experiments show that the improved algorithm can accelerate the convergence speed of agent and the average return of the round is increased by 25%.