Multi-Agent Reinforcement Learning for Wireless User Scheduling: Performance, Scalablility, and Generalization

Abstract

We propose a multi-agent reinforcement learning (MARL) solution for the user scheduling problem in cellular networks. Incorporating features of this particular use case, we cast the problem in a decentralized partially observable Markov decision process (Dec-POMDP) framework, and present a detailed design of MARL that allows for fully decentralized execution. The performance of MARL against both centralized RL and an engineering heuristic solution is comprehensively evaluated in a system-level simulation. In particular, MARL achieves almost the same total system reward as centralized RL, while enjoying much better scalability with the number of base stations. The transferability of both MARL and centralized RL to new environments is also investigated, and a simple fine-tuning approach based on a general model trained on a pool of environments is shown to have faster convergence while achieving comparable performance with individually trained RL agents, demonstrating its generalization capability.