Fuzzy Reinforcement Learning for Robust Spectrum Access in Dynamic Shared Networks

Chaoqiong Fan,Chenglin Zhao,Shijian Bao,Yiwen Tao,Bin Li

doi:10.1109/access.2019.2939000

Abstract

The persistent increases of wireless terminals have brought about diverse shared networks, where robust and efficient spectrum reuse among heterogeneous users is of critical importance while still remains as a challenging task for practical application. In this paper, we study the problem of robust spectrum access (RSA) in a canonical wireless shared network (WSN) with fully considering the inherent dynamics of the wireless environment. The non-static features of WSNs result in uncertain channel state information (CSI) and complicated coupling interference, which can’t be directly formulated as the well-accepted crisp game model, rendering most existing perfect CSI relied approaches inefficient or even unfeasible. To address this, by interpreting the estimated CSI with uncertainty as fuzzy number, a novel framework referred to as a non-cooperative fuzzy game (NC-FG) is adopted, whereby the user utility is mapped as a fuzzy value via the user-defined fuzzy utility function. Based on the derived property of the NC-FG that fuzzy Nash equilibrium (FNE) exists, a fuzzy-logic inspired reinforcement learning (FLRL) algorithm is proposed to achieve the FNE solutions of the constructed NC-FG to obtain the RSA in dynamic WSN, with which both the iterative learning and decision making procedures are implemented in a fuzzy-space, thus the sensitiveness of our scheme to environmental variations is alleviated. Finally, numerical simulations are provided to demonstrate the convergence, effectiveness, and superiority of our proposed FLRL algorithm in dynamic WSNs.

Highlights

As a crucial driven force of the next-generation communication system development [1], the explosive increases of data traffic which are resulted from the unprecedented growth of mobile devices and versatile applications, pose serious challenges to the limited spectrum resource
We propose a fuzzy-logic inspired reinforcement learning [25] (FLRL) algorithm to achieve the equilibrium solution of our formulated non-cooperative fuzzy game (NC-FG), with which the decision is made in an uncertainty immune space, i.e. a fuzzy space, thereby the vulnerability of environmental variations can be remedied
(2) We develop a NC-FG to characterize the formulated optimization problem with the dynamic and uncertain information restriction, and study its property to demonstrate the existence of fuzzy Nash equilibrium (FNE)

Summary

Introduction

As a crucial driven force of the next-generation communication system development [1], the explosive increases of data traffic which are resulted from the unprecedented growth of mobile devices and versatile applications, pose serious challenges to the limited spectrum resource. The imminent spectrum shortage has produced a new impetus to seek practical solutions to improve the utilization efficiency of scarce spectrum resource in a shared manner. Of the Internet of Things (IoT) sector [3], continues to pursue advanced techniques of spectrum efficiency to fulfill the everincreasing wireless service requirements. SSA can enhance the utilization efficiency of scarce spectrum resources by enabling spectrum reuse among diverse heterogeneous users and support the proliferating wireless service demands of dramatically growing mobile terminals, plays a paramount role in alleviating the spectrum scarcity and facilitating the deployment of 5G system [9].

Objectives

Results

Conclusion