Intelligent Resource Allocation in Backscatter-NOMA Networks: A Soft Actor Critic Framework

Abstract

With the use of power domain non-orthogonal multiple access (NOMA) and backscatter communication (BAC), future sixth-generation ultra massive machine-type communications networks are expected to connect large-scale internet of things (IoT) devices. However, due to NOMA co-channel interference, the power allocation to large-scale IoT devices becomes critical. The existing convex optimization-based solutions are highly complex, and therefore it is difficult to find the optimal solution to the resource allocation problem in a highly dynamic environment. To alleviate this problem, this work develops an efficient model-free BAC approach with a NOMA system to assist the base station with complex resource scheduling tasks in a dynamic BAC-NOMA IoT network. The objective is to increase the sum rate of uplink backscatter devices. More specifically, we jointly optimize the transmit power of downlink IoT users and the reflection coefficient of uplink backscatter devices using a reinforcement learning algorithm, namely, the soft-actor critic (SAC) algorithm. With the advantage of entropy regularization, the SAC agent learns to explore and exploit the dynamic BAC-NOMA network efficiently. The proposed algorithm ensures the quality of service (QoS) requirements of downlink users while enhancing the sum rate of uplink backscatter devices. Numerical results reveal the superiority of the proposed algorithm over the conventional optimization (benchmark) approach in terms of the average sum rate of uplink backscatter devices. We show that the network with multiple downlink users obtained a higher reward with respect to a large number of iterations compared to episodes with a lower number of iterations. Moreover, the proposed algorithm outperforms the benchmark scheme and BAC with orthogonal multiple access in terms of the average sum rate with the different number of backscatter devices. Additionally, we show that our proposed algorithm enhances sum rate efficiency with respect to different self-interference coefficients and different noise levels. Finally, we evaluate and show the sum rate efficiency of the proposed algorithm with different QoS requirements and cell radii.