Reinforcement-Learning-Based Dynamic Spectrum Access for Software-Defined Cognitive Industrial Internet of Things

Abstract

The cognitive industrial Internet of Things (CIIoT) can improve transmission performance by utilizing the spectrum licensed to a primary user (PU), providing that the normal communication of the PU is not disturbed. However, the traditional spectrum access schemes for the CIIoT are difficult to adapt to the various communication environments. In this article, <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$Q$</tex-math></inline-formula> -learning-based dynamic spectrum access is proposed for the CIIoT to intelligently utilize the spectrum resources in three access scenarios: orthogonal multiple access (OMA), underlay spectrum access, and nonorthogonal multiple access (NOMA). In the OMA scheme, the CIIoT learns to access the idle channels to avoid distributing the PUs, but its communication continuity cannot be guaranteed when most of the channels are occupied by the PUs. In the underlay scheme, the CIIoT learns to utilize the busy channels to ensure the communication continuity by limiting its transmit power within the tolerance of the PU. However, the interference to the PU cannot be eliminated, which will decrease the PU’s throughput. In the NOMA scheme, however, the CIIoT can utilize the busy channels by canceling the interference to the PU with successive interference cancellation, which will guarantee the transmission performance of both the CIIoT and the PU. A <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$Q$</tex-math></inline-formula> -learning-based spectrum access algorithm is proposed to improve the transmission performance of the CIIoT in the three schemes. The simulation results have shown the advantages of the <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$Q$</tex-math></inline-formula> -learning-based NOMA scheme in terms of guaranteeing the throughput of the CIIoT nodes and decreasing the interference to the PUs.