Learning-Based UAV Coverage-Aware Path Planning in Large-scale Urban Environments

Abstract

In our research, the coverage-aware path planning problem of unmanned aerial vehicles (UAVs) in large-scale urban environments is studied. The goal of our research is to train the intelligent UAV to autonomously cover the area with satisfactory communication quality and finally arrive at the objective place without colliding with numerous skyscrapers. Such a multi-objective path planning problem is regularly non-convex and thus hard to converge. To this end, a distributed deep reinforcement learning (DRL)-based approach called Distributed-RPPO is developed to address the problem. To be specific, firstly we formulate our proposed problem as a Markov decision process (MDP). Since communication connectivity is the core constraint, a proximal policy optimization (PPO)-based SIR prediction network is designed for guiding the UAV to cover the area with superior communication connectivity, using only the UAV’s current location. Then we combine the PPO algorithm with the Long short-term memory (LSTM) structure to solve the basic navigation problem while only the historical interactive data is exploited for updating the parameter vector of the entire network. Simulation results demonstrate that our approach is capable of allowing the UAV to execute coverage-aware path planning tasks in large-scale urban environments. Furthermore, the learning efficiency is greatly improved compared to existing DRL-based approaches.