Primary User Avoidance algorithm for CRAHNs using Reinforcement Learning

Abstract

Cognitive Radio Ad Hoc Networks (CRAHNs) are the need of the current time. There is a requirement of providing accessibility of network “everywhere”. This “everywhere” connectivity to all kinds of unlicensed devices is very hard to achieve in statically allocated spectrum bands. So CRAHNs were envisioned. In CRAHNs, it is very important to protect transmission of primary user. This paper tries to handle this concern. A reinforcement learning based algorithm is proposed and implemented which protects the transmission of primary user by avoiding secondary users to enter primary user coverage area, which is not known in advance. This algorithm will learn from its experience & once it has learnt from its environment, it always tries to avoid primary user within a region where primary user is operating on same channel as secondary user. This work includes Q learning along with neural networks (implemented in python). The experimentations results proved that this algorithm learns over time, because as the number of epochs are increased loss rate tends to decline. The algorithm is executed using neural networks with varying schemes. All of them proved that there is worthy amount of learning with decent accuracy. Results drawn from the proposed algorithm yield as output the distance covered by secondary user in each configuration, without hitting primary user and the mean loss rate for each configuration. The algorithm proposed in this paper is anticipated to be durable and robust. Moreover, it can work on large networks as well.