Deep Q-learning multiple networks based dynamic spectrum access with energy harvesting for green cognitive radio network

Abstract

In cognitive radio (CR), dynamic spectrum access (DSA) can be used for secondary user (SU) to improve its spectrum efficiency under the limited spectrum resources. However, the SU has to consume more energy to sense the state of primary user (PU) before spectrum access. Therefore, high spectrum efficiency and lower energy consumption have become the requirements of CR. In this paper, we propose a sustainable green CR network combining DSA and energy harvesting, which can perform intelligent spectrum sensing, spectrum access and energy harvesting by a reward mechanism of deep Q-learning multiple networks (DQMN). The DQMN model consisting of multiple neural networks is designed to learn multiple learning objectives, including spectrum sensing, channel selection, communication power and access mode in DSA. A sleep mechanism is introduced to optimize the spectrum access mode and further reduce the energy consumption. The simulation results show that the DQMN-based DSA model can autonomously sense the channel state, harvest the energy, and then select the appropriate access strategy. Due to the energy harvesting and sleep mechanism, the proposed system model shows lower energy consumption and higher average effective throughput than the traditional models.