Spectrum Handoff Based on DQN Predictive Decision for Hybrid Cognitive Radio Networks.

Kaitian Cao,Ping Qian

doi:10.3390/s20041146

Abstract

Spectrum handoff is one of the key techniques in a cognitive radio system. In order to improve the agility and the reliability of spectrum handoffs as well as the system throughput in hybrid cognitive radio networks (HCRNs) combing interweave mode with underlay mode, a predictive (or proactive) spectrum handoff scheme based on a deep Q-network (DQN) for HCRNs is proposed in this paper. In the proposed spectrum handoff approach, spectrum handoff success rate is introduced into an optimal spectrum resource allocation model to ensure the reliability of spectrum handoff, and the closed-form expression for the spectrum handoff success rate is obtained based on the Poisson distribution. Furthermore, we exploit the transfer learning strategy to further improve the DQN learning process and finally achieve a priority sequence of target available channels for spectrum handoffs, which can maximize the overall HCRNs throughput while satisfying constraints on secondary users’ interference with primary user, limits on the spectrum handoff success rate, and the secondary users’ performance requirements. Simulation results show that the proposed spectrum handoff scheme outperforms the state-of-the-art spectrum handoff algorithms based on predictive decision in terms of the convergence rate, the handoff success rate and the system throughput.

Highlights

Cognitive radio networks (CRNs) have received great attention due to their potential to provide an efficient solution to the contradiction between spectrum scarcity and inefficient spectrum utilization, and improve system capacity via dynamic spectrum access (DSA) and spectrum management techniques [1,2]
The performance of transfer learning (TL)-PDSH spectrum handoff method based on deep Q-network (DQN) is verified through Monte-Carlo simulations, and it is compared with the PDSH algorithm, the proactive decision based-handoff scheme (PDBHS) algorithm [11], and the Q-PDSH method
It is assumed that there is only one primary users (PUs) accessing a single channel in the primary network, the transmitting power of primary base station (PBS) is 100 mW, the Gaussian noise power is 10 nW, and the signal-to-interference plus noise ratio (SINR) for the PU

Summary

Introduction

Cognitive radio networks (CRNs) have received great attention due to their potential to provide an efficient solution to the contradiction between spectrum scarcity and inefficient spectrum utilization, and improve system capacity via dynamic spectrum access (DSA) and spectrum management techniques [1,2]. In [11], a proactive decision based-handoff scheme (PDBHS) for CRNs is proposed, in which a hybrid handoff strategy is addressed by minimizing the number of handoffs such that the total service time is minimized. The above spectrum handoff methods based on predictive decision still have the following drawbacks: (1) data transmission only between a pair of sending and receiving SUs is considered, the impact of surrounding SUs’ behaviors on a SU is not taken into account;. In order to solve the shortcomings of the above existing spectrum handoff approaches, we propose a transfer learning (TL)-based predictive decision spectrum handoff (TL-PDSH) method by introducing a deep Q-network (DQN) [18], TL [19] strategy, and the handoff success rate in this paper. A DQN algorithm is used to obtain the optimal learning strategy and seek the target channel sequence for spectrum handoffs, and the TL strategy is introduced in our method to further improve the DQN learning process

System Model

TL-PDSH Spectrum Handoff Scheme Based on DQN

Simulations Results

Itcurve can be the number

Conclusions