Dynamic spectrum access based on double deep Q-network and convolution neural network

Abstract

As a new technology, deep reinforcement learning (DRL) has been applied in many fields of wireless communication in recent years. In this paper, we propose a centralized dynamic spectrum access (DSA) scheme based on DRL and convolutional neural network (CNN). The traditional distributed DSA scheme encounters two important issues, namely, interference suppression for primary users (PU) and interference coordination for secondary users (SU). In order to cope with these two issues, a centralized DSA scheme implemented with deep convolution Q-network is proposed in this paper. Operating with the proposed centralized deep convolution Q-network spectrum access (CDCQSA) algorithm, a central unit (CU) uses DRL to unify the spectrum access actions in the network, allowing multiple SU to share spectrum resources with multiple PU without requiring any prior information. Simulation results show that this proposed scheme improves the channel utilization and reduces the conflict probability between PU and SU while avoiding the collisions between SU as well.