Energy Efficiency Optimization for SWIPT-Based D2D-Underlaid Cellular Networks Using Multiagent Deep Reinforcement Learning

Abstract

In this article, we study the optimization of energy efficiency in wireless device-to-device (D2D-underlaid cellular networks where multiple D2D pairs adopt simultaneous wireless information and power transfer functionality. We formulate the optimization problem, which is a NP-hard combinatorial problem with nonlinear constraints. First, we use optimization-based-iterative techniques such as exhaustive search (ES) and gradient search (GS) with barrier, which are generally used to obtain the global optimum and local optimum of the nonconvex optimization problem, respectively. Considering that these techniques require a centralized unit to share information with each other, we propose multiagent deep reinforcement learning to solve this optimization problem in a distributed manner, which provides optimal decision making together with efficient deep network training under inequality constraints including transmit power, power splitting ratio, and minimum requirement data rate for D2D and cellular users. In this proposed method, we consider the virtual environment in which each agent can train their model according to shared information, and then, we deploy the trained model into the actual environment where each agent can only know their channel gain, interference power, and required minimum throughput. Simulation results show that the proposed algorithm can afford a near-global-optimum solution with much lower computation complexity than ES and outperforms the GS.