A Model-Driven Deep Reinforcement Learning Heuristic Algorithm for Resource Allocation in Ultra-Dense Cellular Networks

Abstract

Resource allocation in ultra dense network (UDN) is an multi-objective optimization problem since it has to consider the tradeoff among spectrum efficiency (SE), energy efficiency (EE) and fairness. The existing methods can not effectively solve this NP-hard nonconvex problem, especially in the presence of limited channel state information (CSI). In this paper, we investigate a novel model-driven deep reinforcement learning assisted resource allocation method. We first design a novel deep neural network (DNN)-based optimization framework consisting of a series of Alternating Direction Method of Multipliers (ADMM) iterative procedures, which makes the CSI as the learned weights. Then a novel channel information absent Q-learning resource allocation (CIAQ) algorithm is proposed to train the DNN-based optimization framework without massive labeling data, where the SE, the EE, and the fairness can be jointly optimized by adjusting discount factor. Our simulation results show that, the proposed CIAQ with rapid convergence speed not only well characterizes the extent of optimization objective with partial CSI, but also significantly outperforms the current random initialization method of neural network and the other existing resource allocation algorithms in term of the tradeoff among the SE, EE and fairness.