Improving the QoS in 5G HetNets Through Cooperative Q-Learning

Abstract

Heterogeneous networks are an integral part of the 5G cellular networks as they are one of the important enabling technologies for increased coverage and capacity. However, interferences in multi-tiered architecture bottleneck its performance. Although multiple schemes have been proposed for efficient radio resource management to handle the interferences in heterogeneous networks but provision of quality of service to macrocell and small cell user equipment simultaneously, is still an open research problem. Intelligent schemes for radio resource management in heterogeneous networks have proved their effectiveness due to their self-optimization capabilities. In this research article, a cooperative Q-Learning, algorithm is proposed for efficient joint radio resource management in ultra-dense heterogeneous networks to handle interferences by adaptive power allocation to small cell base stations while considering the minimum quality of service requirements. In this proposed cooperative Q-Learning algorithm, small cell base stations interacts with the neighboring small cell base stations to exchange information and performs self-optimization based on a joint reward function. The proposed solution not only provided significant improvement in the capacity of macrocell and small cell user equipment as compared to other state of art Q-Learning based radio resource management schemes but also ensure the provision of quality of service to all macrocell and small cell user equipment simultaneously in the cluster of 16 small cells. The proposed solution provided a minimum capacity of 2 b/s/Hz to macrocell and small cell user equipment which is 100% higher than the minimum quality of service requirements defined in literature where none of recently proposed solution could meet minimum quality of service requirements. The results analysis shows that cooperation among the small cells yields a significant improvement of 48% in capacity of small cell user equipment at the cost of a slight increase in computational time as compared to independent learning.