Deep Multi-Agent Reinforcement Learning for Resource Allocation in D2D Communication Underlaying Cellular Networks

Abstract

Device-to-device communications underlaying cellular networks have been recognized as one of the key technologies for the fifth generation (5G) cellular system to improve the spectrum efficiency and system capacity. In this paper, we investigate the potential of deep reinforcement learning (DRL) for joint subcarrier assignment and power allocation in a general form of D2D networks, where a subcarrier can be assigned to multiple D2D pairs and each D2D pair is permitted to utilize multiple subcarriers. We first formulate the above problem as a Markov decision process, and then propose a double deep Q-network (DQN)-based subcarrier-power allocation algorithm to learn the optimal policy in the absence of full instantaneous channel state information (CSI). Specifically, each D2D pair acts as a learning agent that adjusts its own subcarrier-power allocation strategy iteratively through interactions with the operating environment in a trial-and-error fashion. Simulation results confirm that the proposed algorithm achieves near optimal performance in real time. It is worth mentioning that the proposed algorithm is especially suitable for the case where the environmental dynamics is not accurate and the CSI delay cannot be ignored.