Cellular-Connected UAV Trajectory Design With Connectivity Constraint: A Deep Reinforcement Learning Approach

Abstract

Cellular-connected unmanned aerial vehicle (UAV) communication has attracted increasingly attention recently. We consider a cellular-connected UAV carried with limited on-board energy during the mission period requires flying from its initial location to a pre-determined location before the on-board energy runs out. The work aims to minimize the total outage duration by exploiting the UAV's mobility to optimize its trajectory. Traditional methods for solving such problems usually require an accurate and tractable communication model, which is difficult to be realized due to the complexity of the communication system. Even if this problem is solved, the formulated problem is hard to be addressed by standard optimization techniques because of the non-convexity. To this end, we propose a novel approach based on deep reinforcement learning (DRL), which only requires multiple interactions between the UAV and the environment, and can solve such problems well. Specifically, using dueling double deep Q network (dueling DDQN) with multi-step learning algorithm, the outage time of the UAV with different on-board energy and communication connectivity constrains is investigated to evaluate the effectiveness of our proposed algorithm. Numerical results demonstrate the proposed design with DRL contributing to significant performance enhancement compared with the benchmark.