Smart Energy Borrowing and Relaying in Wireless-Powered Networks: A Deep Reinforcement Learning Approach

Abstract

Wireless energy harvesting (EH) communication has long been considered a sustainable networking solution. However, it has been limited in efficiency, which has been a major obstacle. Recently, strategies such as energy relaying and borrowing have been explored to overcome these difficulties and provide long-range wireless sensor connectivity. In this article, we examine the reliability of a wireless-powered communication network by maximizing the net bit rate. To accomplish our goal, we focus on enhancing the performance of hybrid access points and information sources by optimizing their transmit power. Additionally, we aim to maximize the use of harvested energy, by using energy-harvesting relays for both information transmission and energy relaying. However, this optimization problem is complex, as it involves non-convex variables and requires combinatorial relay selection indicator optimization for decode and forward (DF) relaying. To simplify this problem, we utilize the Markov decision process and deep reinforcement learning framework based on the deep deterministic policy gradient algorithm. This approach enables us to tackle this non-tractable problem, which conventional convex optimization techniques would have difficulty solving in complex problem environments. The proposed algorithm significantly improved the end-to-end net bit rate of the smart energy borrowing and relaying EH system by 13.22%, 27.57%, and 14.12% compared to the benchmark algorithm based on borrowing energy with an adaptive reward for Quadrature Phase Shift Keying, 8-PSK, and 16-Quadrature amplitude modulation schemes, respectively.