For multi-obstacle complex scenarios, the traditional artificial potential field method suffers from the defects of potential field imbalance, its capability to easily fall into the local minima, and encounter unreachable targets in complex navigation environments. Therefore, this paper proposes a three-dimensional adaptive potential field algorithm (SAPF) based on multi-agent reinforcement learning. First, in this paper, the gravitational function in the artificial potential field (APF) is modified to weaken the gravitational effect on the UAV in the region far away from the target point in order to reduce the risk of collision between the UAV and the obstacles during the moving process. Second, in the region close to the target point, this paper improves the artificial potential field function to ensure that the UAV can reach the target point smoothly and realize path convergence by considering the relative distance between the UAV’s current position and the target point. Finally, for the characteristics of UAV trajectory planning, a 3D state space is designed based on the 3D coordinates of the UAV, the distance between the UAV and the nearest obstacle, and the distance between the UAV and the target point; an action space is designed based on the displacement increment of the UAV in the three coordinate axes; and the specific formulas for collision penalties and path optimization rewards are re-designed, which effectively avoids the UAV from entering the local minimal points. The experimental results show that the artificial potential field method designed with reinforcement learning can plan shorter paths and exhibit better planning results. In addition, the method is more adaptable in complex scenes and has better anti-interference.
Read full abstract