Decoupled Association With Rate Splitting Multiple Access in UAV-Assisted Cellular Networks Using Multi-Agent Deep Reinforcement Learning

Abstract

In unmanned aerial vehicles (UAVs) assisted cellular networks, user association plays an important role in interference control and spectrum efficiency. In this paper, we study the performance of uplink-downlink decoupled (UDDe) user association in a multi-UAV assisted network in which each user can associate with different UAVs or the macro base station (MBS) for uplink (UL) and downlink (DL) transmissions. Since some popular data may be requested by multiple users, grouping these users and applying multicasting can significantly improve spectral efficiency. Unlike traditional linear precoding that treats interference entirely as noise, we propose a rate-splitting multiple access (RSMA) policy that employs rate splitting at the transmitter and successive interference cancellation (SIC) at the receiver. To be specific, the transmitted signal is split into a common part and a private part, and the interference is partially decoded and partially treated as noise. In this context, we formulate a joint optimization problem of UL-DL association and beamforming for maximizing the sum-rate of users in UL and that of multicast groups in DL under the constraints of UAV backhaul capacity and power budget. Since the formulated problem is non-convex with intricate states and an individual UAV may not know the rewards of other UAVs, we convert it into a robust partially observable Markov decision process (POMDP). Then we resort to multi-agent deep reinforcement learning (MADRL) that enables each UAV to learn and optimize its policy in a distributed manner. To achieve an optimal policy, we further propose an improved clip and count-based proximal policy optimization (PPO) algorithm to train actor and critic networks. Simulation results demonstrate the superiority of the proposed decoupled association strategy with RSMA and the MADRL learning algorithm.