Three-Dimensional Path Planning for Unmanned Helicopter Using Memory-Enhanced Dueling Deep Q Network

Abstract

Unmanned helicopter (UH) is often utilized for raid missions because it can evade radar detection by flying at ultra-low altitudes. Path planning is the key technology to realizing the autonomous action of UH. On the one hand, the dynamically changing radar coverage area and the mountains in the low airspace environment will seriously affect the flight safety of UH. On the other hand, the huge state space of the three-dimensional (3D) environment will also make traditional algorithms difficult to converge. To address the above problems, a memory-enhanced dueling deep Q-network (ME-dueling DQN) algorithm was proposed. First, a comprehensive reward function was designed, which can guide the algorithm to converge quickly and effectively improve the sparse reward problem. Then, we introduced a dual memory pool structure and proposed a memory-enhanced mechanism, which can reduce invalid exploration, further improve the learning efficiency of the algorithm, and make the algorithm more stable. Finally, the path planning ability of the proposed algorithm in multiple experimental environments was verified. Experiments showed that the proposed algorithm has good environmental adaptability and can help UH to accurately identify dangerous areas and plan a safe and reliable flight path.