An Efficient Reinforcement Learning for Device-to-device Communication Underlaying Cellular Network

Abstract

In this paper, a novel actor-critic reinforcement learning (RL) based on policy gradient is proposed to solve the channel and power allocation issues for a device-to-device (D2D) enabled cellular network when prior traffic information was not provided to the base station (BS). Furthermore, in this paper, we design a system to learn the optimal policy for resource and power allocation between cellular users (CUs) and D2D users, aimed at maximizing the sum rate of the overall system. Since the behavior of wireless channels, and the received reward in each state associated with the system, is stochastic in nature, the dynamic property of the environment allows us to apply an actor-critic RL technique to learn the best policy through continuous interaction with the surroundings. The policy-based approach is better than a value-based scheme, such as Q-learning, because it takes the help of policy space in order to maximize the expected throughput. The actor adopts a parameter-based stochastic policy for continuous actions, while the critic evaluates the policy through its overall performance, and criticizes the actor for the policy it follows. Through numerical simulations, we verify the performance of our proposed work with the existing methods.