Multi-UAV autonomous collision avoidance based on PPO-GIC algorithm with CNN–LSTM fusion network

Abstract

This paper is concerned with the autonomous effective collision avoidance strategy for multiple unmanned aerial vehicles (multi-UAV) in limited airspace under the framework of proximal policy optimization (PPO) algorithm. An end-to-end deep reinforcement learning (DRL) control strategy and a potential-based reward function are designed. Next, the CNN-LSTM (CL) fusion network is constructed by fusing the convolutional neural network (CNN) and the long short-term memory network (LSTM), which realizes the feature interaction among the information of multi-UAV. Then, a generalized integral compensator (GIC) is introduced into the actor-critic structure, and the CLPPO-GIC algorithm is proposed by combining CL and GIC. Finally, we validate the learned policy in various simulation environments by performance evaluation. The simulation results show that the introduction of the LSTM network and GIC can further improve the efficiency of collision avoidance, and the robustness and accuracy of the algorithm are verified in different environments.