Imperfect CSI Based Intelligent Dynamic Spectrum Management Using Cooperative Reinforcement Learning Framework in Cognitive Radio Networks

Abstract

The rapid development of wireless traffic pushed the wireless community to research different solutions towards the efficient utilization of the available radio spectrum. However, a recent study shows that most of the dynamically allocated spectrum bands (radio frequency resources), experience significant underutilization as cognitive radio (CR) technology still lacks intelligence. An intelligence in CRs can be incorporated with machine learning algorithms. Further, the perfect channel state information (CSI) is hardly obtained and CSI imperfections play a crucial role in Dynamic Spectrum Management. Thus, for efficient utilization of available spectrum, a decentralized Multi-Agent Reinforcement Learning based resource allocation scheme has been proposed. A robust resource allocation scheme is proposed which integrates machine learning and CR technology into a sophisticated multi-agent system (MAS). Moreover, assisted with cloud computing which provides a huge amount of storage space, reduces operating expenditures, and provides wider flexibility of cooperation. Hence, to foster the performance of the proposed scheme, a cooperative framework in MAS is introduced which enhances the performance of the proposed scheme in terms of network capacity, outage probability, and convergence speed. Numerical results verify the effectiveness of the proposed scheme and show the non-negligible impact of imperfect CSI, thus highlighting the importance of robust designs that maintains users’ QoS in practical wireless networks.