Distributed UAV-BSs Trajectory Optimization for User-Level Fair Communication Service With Multi-Agent Deep Reinforcement Learning

Abstract

Unmanned Aerial Vehicles (UAVs) have attacted much attention in the field of wireless communication due to its agility and altitude. UAVs can be used as low-altitude aerial base stations (UAV-BSs) to provide communication services for ground devices (GDs) in various scenarios, such as emergency communication and traffic offloading in hotspots. However, due to the limited communication ranges and high prices of commercial UAV-BSs, covering a target area all the time with sufficient UAVs is quite challenging, especially under dynamic environment. We need to design the trajectory of the UAV-BSs to optimize system performance. Most existing works focus on the energy-efficient coverage and throughput maximization but ignore the fairness of communication service, especially the fairness at user-level. Besides, reinforcement learning is suitable for solving decision problems in dynamic environments. However, most existing works use centralized deep reinforcement learning (DRL) approaches. Due to the scalability and low time complexity, a distributed DRL approach is more suitable for multiple UAV-BSs communication system in dynamic environment. Unlike previous works, we characterize the fairness at user-level based on proportional fairness scheduling and formulate a weighted-throughput maximization problem via designing UAV-BSs’ trajectory. Then we model the dynamic deploymentproblem of UAV-BSs as a Markov game and propose a multi-agent deep reinforcement learning-based distributed UAV-BSs control approach named MAUC. MAUC approach adopts the framework of centralized training with distributed execution. Simulation results show that the MAUC can improve fairness of communication service by sacrificing a small amount of throughput.