Deep Reinforcement Learning Based End-to-End Multiuser Channel Prediction and Beamforming

Abstract

In this paper, reinforcement learning (RL) based end-to-end channel prediction (CP) and beamforming (BF) algorithms are proposed for multi-user downlink system. Different from the previous methods which either require perfect channel state information (CSI), or estimate outdated CSI and set constraints on pilot sequences, the proposed algorithms have no such premised assumptions or constraints. Firstly, RL is considered in channel prediction and the actor-critic aided CP algorithm is proposed at the base station (BS). With the received pilot signals and partial feedback information, the actor network at BS directly outputs the predicted downlink CSI without channel reciprocity. After obtaining the CSI, BS generates the beamforming matrix using zero-forcing (ZF). Secondly, we further develop a deep RL based two-layer architecture for joint CP and BF design. The first layer predicts the downlink CSI with the similar actor network as in the CP algorithm. Then, by importing the outputs of the first layer as inputs, the second layer is the actor-critic based beamforming layer, which can autonomously learn the beamforming policy with the objective of maximizing the transmission sum rate. Since the learning state and action spaces in the considered CP and BF problems are continuous, we employ the actor-critic method to deal with the continuous outputs. Empirical numerical simulations and the complexity analysis verify that the proposed end-to-end algorithms could always converge to stable states under different channel statistics and scenarios, and can beat the existing traditional and learning based benchmarks, in terms of transmission sum rate.