Path Planning based on Reinforcement Learning with Improved APF model for Synergistic Multi-UAVs

Abstract

As the emerging technology of Unmanned Aerial Vehicles (UAVs) becomes mature, UAVs are widely used in environmental monitoring, communication and other fields. In view of this, this paper analyzed the task of synergistic multi-UAVs exploration of unknown environments, and proposed a path planning method for them based on reinforcement learning. Firstly, the path planning task of the UAVs was divided into two parts: the path travel strategy module and the information exploration strategy module. Models of the two modules were based on the Deep Deterministic Policy Gradient algorithm (DDPG), and an improved Artificial Potential Field (APF) force traction mechanism was introduced in the path travel strategy module. Its aim was to assist in guiding the generation of UAV flight path trajectories. Also it could enhance the learning capability of the model. The path travel strategy module would generate the complete flight path of the whole cluster in a distributed manner. A series of temporary target points provided by the information exploration strategy module helped. In maps with 21.5%, 25.3% and 29.6% of obstacles, multi-UAVs could achieve 84.2%, 76.7% and 69.9% of environmental exploration by the designed method. Compared with the APF method, the A star method and the Breath First Search (BFS) method, the proposed method is not only able to plan feasible paths in a more complex map model, but also the curvature of the planned paths is smoother, thus achieving the goal of reducing the energy cost of UAVs.