Multi-Agent Deep Reinforcement Learning for Interference-Aware Channel Allocation in Non-Terrestrial Networks

Abstract

Non-terrestrial network (NTN) services using low-Earth-orbit (LEO) satellites are expanding. Interference management of NTN services with other terrestrial wireless services is emerging as a critical issue owing to the inherent international and vast coverage nature of NTN. This study develops a multi-agent deep reinforcement learning (DRL) framework to establish a multi-beam uplink channel allocation strategy that minimizes interference with incumbent stations under the given quality of service (QoS) constraints. We propose a novel framework with the sequential training of agents in a specific order to overcome the inherent non-stationarity of multi-agent DRL. To improve learning efficiency, we design the training sequence in accordance with reward function and initial state. As a result, taking actions in the order of the largest interference to the incumbent station provides superior performance to taking actions in an arbitrary order. Moreover, the proposed channel allocation performs close to the optimal exhaustive search and outperforms conventional greedy graph coloring method.