Multi-Agent Deep Reinforcement Learning Based Transmission Latency Minimization for Delay-Sensitive Cognitive Satellite-UAV Networks

Abstract

With the ubiquitous deployment of a massive number of Internet-of-Things (IoT) devices, the satellite-aerial networks are becoming a promising candidate to provide flexible and seamless service for IoT applications. Concerning about the spectrum scarcity issue, we present a cognitive satellite-aerial network where the multiple unmanned aerial vehicles (UAVs) can share spectrum with the satellite without interfering with satellite communications. To further improve spectral efficiency, non-orthogonal multiple access (NOMA) technique is adopted in this network. Considering the delay-sensitive quality-of-service (QoS) requirement, a joint trajectory and power optimization problem is formulated to minimize the total transmission latency over a long-term task period. In order to reduce the computational complexity and ease the burden of information exchange by using the centralized DRL methods, we propose a multi-agent deep deterministic policy gradient (MADDPG) based algorithm which adopts the framework of centralized training with decentralized execution to solve the sophisticated problem. The simulation results show the proposed algorithm can achieve satisfactory performance through joint trajectory control and power allocation for the UAVs compared with other methods.