Optimal Resource Allocation for Reconfigurable Intelligent Surface Assisted Dynamic Wireless Network via Online Reinforcement Learning

Abstract

This paper investigates the problem of optimal resource allocation for reconfigurable intelligent surface (RIS) assisted dynamic wireless networks with uncertain time-varying wireless channels. Recently, RIS has been considered as one of the most promising techniques for enhancing dynamic wireless network quality, e.g. maximizing spectrum efficiency, etc., without increasing power consumption. However, conventional resource allocation algorithms cannot be directly utilized for RIS-assisted wireless networks especially when the wireless channels among base station (BS), RIS, and users (UEs) are uncertain and time varying. Hence, a novel online reinforcement learning based optimal resource allocation algorithm has been developed in this paper. Firstly, the RIS-assisted wireless communication network with dynamic wireless channels has been represented as a state-space model. Then, the optimal resource allocation problem can be formulated as a finite-horizon joint optimal control of users' transmit powers and RIS phase shifts problem. Next, since the wireless channel is time-varying and uncertain, a novel online reinforcement learning technique, i.e. Actor-Critic design, has been developed along with neural networks (NN) to learn the optimal resource allocation policies in real-time. Eventually, numerical simulations have been provided to demonstrate the effectiveness of the developed scheme.