Piecewise-DRL: Joint Beamforming Optimization for RIS-Assisted MU-MISO Communication System

Abstract

With the widespread connectivity of everyday devices realized by the advent of the Internet of Things (IoT), communication between users of different devices has become increasingly close. In practical scenarios, obstacles present between the transceiver may cause a deterioration in the quality of the received signals. Therefore, the reconfigurable intelligent surface (RIS) is employed to create virtual Line-of-Sight (LoS) channels in an IoT network. Specifically, this paper aims at maximizing the sum-rate of the RIS-assisted multi-user multiple input single output (MU-MISO) communication systems by jointly optimizing the phase shift matrix of the RIS and transmit beamforming. To solve the formulated non-convex problem, a piecewise-deep reinforcement learning (DRL) algorithm is proposed in this paper. Unlike the existing alternative optimization (AO) algorithms, the proposed algorithm avoids falling into the local optimal by using an exploration mechanism. Moreover, piecewise-DRL can reduce the action dimension, allowing the algorithm to obtain faster convergence. Simultaneously, this algorithm also ensures that the parameters of the two-part networks are updated to generate a larger system sum-rate by unsupervised joint optimization. Simulations in various circumstances reveal that the proposed approach is more robust and presents better stability and faster convergence than previous state-of-the-art algorithms while obtaining competitive performance.