A General DRL-based Optimization Framework of User Association and Power Control for HetNet

Abstract

Heterogeneous network (HetNet) and Non-Orthogonal Multiple Access (NOMA) have been seen as promising ways to improve the network capacity. However, the dense deployment of base stations (BSs) in HetNet causes unbalanced loads and large energy consumption. In this paper, we investigate the joint user association and power control problem with the objective of improving the sum rate and the energy efficiency in the Orthogonal Multiple Access (OMA)-enabled HetNet and NOMA-enabled HetNet, respectively. Particularly, the traditional joint user association and power control algorithms can only solve the problem for one network scenario, which are not applicable to the other networks. Specifically, this paper presents a novel deep reinforcement learning (DRL)-based general optimization framework, which is a unified solution for the user association and power control problem and can adapt to OMA-enabled and NOMA-enabled HetNet scenarios with relatively minor modifications. In the framework, a Deep Deterministic Policy Gradient Algorithm (DDPG)-based joint user association and power control algorithm is proposed that can learn to achieve load balance and improve energy efficiency by interacting with the environment. Specifically, different from the DDPG, the proposed algorithm can solve the discrete and continuous optimization problems together. Finally, simulation results demonstrate the proposed algorithm achieves a higher sum rate and energy efficiency than the traditional algorithms in both OMA-enabled HetNet and NOMA-enabled HetNet.