Deep Reinforcement Learning Optimal Transmission Algorithm for Cognitive Internet of Things With RF Energy Harvesting

Abstract

Spectrum scarcity and energy limitation are becoming two critical issues in designing Internet of Things (IoT). As two promising technologies, cognitive radio (CR) and radio frequency (RF) energy harvesting can be used together to improve both energy and spectral efficiency. In this paper, an optimal transmission problem in a cognitive IoT (CIoT) with RF energy harvesting capability is investigated, where the optimization problem is formulated as a Markov decision process (MDP) without any priori-knowledge. Considering that the channel activity states of primary user network (PUN), RF energy arrival process and channel information are not available in advance, a deep reinforcement learning (DRL) based deep deterministic policy gradient (DDPG) algorithm is proposed to deal with the dynamic uplink access, working mode selection and continuous power allocation to maximize a long term uplink throughput. The simulation results show that the proposed algorithm is valid and efficient to achieve better performances when compared with deep <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">${Q}$ </tex-math></inline-formula> -network (DQN) based, myopic and random algorithms.