Reinforcement learning for cooperative sensing gain in cognitive radio ad hoc networks

Abstract

Spectrum sensing is a fundamental function in cognitive radio networks for detecting the presence of primary users in licensed bands. The detection performance may be considerably compromised due to multipath fading and shadowing. To resolve this issue, cooperative sensing is an effective approach to combat channel impairments by cooperation of secondary users. This approach, however, incurs overhead such as delay for reporting local decisions and the increase of control traffic. In this paper, a reinforcement learning-based cooperative sensing (RLCS) method is proposed to address the cooperation overhead problem and improve cooperative gain in cognitive radio ad hoc networks. The proposed algorithm is proven to converge and capable of (1) finding the optimal set of cooperating neighbors with minimum control traffic, (2) minimizings the overall cooperative sensing delay, (3) selecting independent users for cooperation under correlated shadowing, and (4) excluding unreliable users and data from cooperation. Simulation results show that the RLCS method reduces the overhead of cooperative sensing while effectively improving the detection performance to combat correlated shadowing. Moreover, it adapts to environmental change and maintains comparable performance under the impact of primary user activity, user movement, user reliability, and control channel fading.