Reinforcement Learning for Routing and Spectrum Management in Cognitive Wireless Mesh Network

Abstract

Cognitive radio networks (CRNs) can provide a means for offering end-to-end Quality of Service (QoS) required by unlicensed users (secondary users. SUs). The authors consider the approach where licensed users (primary users, PUs) play the role of routers and lease spectrum with QoS guarantees for the SUs. Available spectrum is managed by the PU admission and routing policy. The main concern of the proposed policy is to provide end-to-end QoS connections to the SUs. Maximizing gain is the key objective for the PU. In this paper, the authors propose a novel resource management scheme where reinforcement learning (RL) is used to drive resource management scheme. The derived scheme helps PUs to adapt to the changes in the network conditions such as traffic load, spectrum cost, service reward, etc, so that PU's gain can continuously be optimized. The approach integrates spectrum adaptations with connection admission control and routing policies. Numerical analysis results show the ability of the proposed approach to attain the optimal gain under different conditions and constraints.