A Kind of Joint Routing and Resource Allocation Scheme Based on Prioritized Memories-Deep Q Network for Cognitive Radio Ad Hoc Networks.

Yihang Du,Lei Xue,Fan Zhang

doi:10.3390/s18072119

Abstract

Cognitive Radio (CR) is a promising technology to overcome spectrum scarcity, which currently faces lots of unsolved problems. One of the critical challenges for setting up such systems is how to coordinate multiple protocol layers such as routing and spectrum access in a partially observable environment. In this paper, a deep reinforcement learning approach is adopted for solving above problem. Firstly, for the purpose of compressing huge action space in the cross-layer design problem, a novel concept named responsibility rating is introduced to help decide the transmission power of every Secondary User (SU). In order to deal with problem of dimension curse while reducing replay memory, the Prioritized Memories Deep Q-Network (PM-DQN) is proposed. Furthermore, PM-DQN is applied to solve the joint routing and resource allocation problem in cognitive radio ad hoc network for minimizing the transmission delay and power consumption. Simulation results illustrates that our proposed algorithm can reduce the end-to-end delay, packet loss ratio and estimation error while achieving higher energy efficiency compared with traditional algorithm.

Highlights

With the explosion of wireless communication devices, the spectrum resource has become scarce and crowded
A Cognitive Radio Networks (CRN) with 10 Secondary User (SU) nodes and 5 primary users (PUs) channels deployed in a 300 m × 300 m region is considered, and the discrete transmission power of SU nodes is divided into
We have proposed a Prioritized Memories Deep Q-Network (PM-Deep Q-Network (DQN))-based joint routing and resource allocation scheme for cognitive radio ad hoc networks

Summary

Introduction

With the explosion of wireless communication devices, the spectrum resource has become scarce and crowded. CR is a promising technique that adopts DSA paradigm to enhance spectrum utilization, and it allows secondary users (SUs) to access the licensed frequency bands opportunistically in a dynamic and non-interfering manner. For the convenience thereader reader,with a brief to introduction reinforcement learning objective is just to provideofthe basic information understandofthe succeeding sections. This section is far from being an exhaustive survey, and its objective is just to Learning provide the reader with basic information to understand the succeeding sections. The RL problem is typically formulated as a finite-state MDP which comprises a discrete state space S appropriately by interacting with the environment only through trial and error [18]. At each problem is typically formulated as a finite-state MDP which comprises a discrete state space S and step action t, the space controller observes the system’s current state an action s St =and atime discrete

Methods

Results

Conclusion