UAV Path Planning Based on Multi-Layer Reinforcement Learning Technique

Abstract

Unmanned aerial vehicles (UAVs) have been widely used in many applications due to its small size, swift mobility and low cost. Therefore, the study of guidance, navigation and control (GNC) system of UAV has becoming a popular research direction. Path planning plays an important role in the GNC system. In this paper, a multi-layer path planning algorithm based on reinforcement learning (RL) technique is proposed. Compared to the classic Q-learning, the proposed multi-layer algorithm has a distinct advantage that it collects both global and local information which greatly improves overall performance. The proposed RL algorithm has two layers, the higher layer deals with the local information, which could be considered as a short-term strategy. The lower layer deals with the global information, which could be considered as a long-term strategy. Both the higher layer and lower layer are working in harmony to plan a collision-free path. B-spline curve approach is applied for on-line path smoothing. Simulation results in different scenarios prove the effectiveness of multi-layer Q-learning algorithm.