Deep RL-based Trajectory Planning for AoI Minimization in UAV-assisted IoT

Abstract

Due to the flexibility and low deployment cost, unmanned aerial vehicles (UAVs) have been widely used to assist cellular networks in providing extended coverage for Internet of Things (IoT) networks. Existing throughput or delay-based UAV trajectory planning methods cannot meet the requirement of collecting fresh information from IoT devices. In this paper, by taking age-of-information (AoI) as a measure of information freshness, we investigate AoI-based UAV trajectory planning for fresh data collection. To model the complicated association and interaction pattern between UAV and IoT devices, the UAV trajectory planning problem is formulated as a Markov decision process (MDP) to capture the dynamics of UAV locations. As network topology and traffic generation pattern are unknown ahead, we propose an AoI-based trajectory planning (A-TP) algorithm using deep reinforcement learning (RL) technique. To accelerate the learning process during online decision making, the off-line pre-training of deep neural networks is performed. Extensive simulation results demonstrate that the proposed algorithm can significantly reduce the AoI of collected IoT data, as compared to other benchmark approaches.