Multi-Agent Learning and Bargaining Scheme for Cooperative Spectrum Sharing Process

Abstract

Recently, the lack of spectrum resources has become a key technical bottleneck to develop the Industrial Internet of Things (IIoT). Based on cognitive radio technology, the cognitive IIoT (CIIoT) paradigm can improve spectrum utilization via opportunistically accessing the idle spectrum bands. In this study, a novel cooperative spectrum sharing scheme is presented for the CIIoT system platform. The main challenge of our scheme is to effectively share the limited spectrum resource via cooperative sensing and dynamic accessing techniques. To achieve a mutually desirable solution for different CIIoT devices, we design a learning game model using the ideas of <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">multi-agent reinforcement learning (MARL)</i> and the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">negotiated aspirations bargaining solution (NABS)</i> . In the learning mechanism, individual CIIoT devices adaptively select their cooperative sensing policy according to the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">MARL</i> model. In the bargaining mechanism, the available spectrum resource is dynamically shared through the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">NABS</i> , which is obtained based on the devices’ selected sensing policy. By investigating the contribution of <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">MARL</i> to game theory, the proposed scheme can effectively guide intelligent CIIoT devices toward a socially optimal outcome. Numerical simulation results demonstrate that the normalized device payoff, CIIoT system throughput and device fairness of our approach are better than those of existing benchmark protocols. Finally, we present the key challenges and future direction of our research in the CIIoT system operations.