Markov game for autonomic joint radio resource management in a multi-operator scenario

Abstract

This article addresses the autonomy of joint radio resource management (JRRM) between heterogeneous radio access technologies (RATs) owned by multiple operators. By modeling the inter-operator competition as a general-sum Markov game, correlated-Q learning (CE-Q) is introduced to generate the operators' pricing and admission policies at the correlated equilibrium autonomically. The heterogeneity in terms of coverage, service suitability, and cell capacity amongst different RATs are considered in the input state space, which is generalized using multi-layer feed-forward neural networks for less memory requirement. Simulation results indicate that the proposed algorithm can produce rational JRRM polices for each network under different load conditions through the autonomic learning process. Such policies guide the traffic toward an optimized distribution and improved resource utilization, which results in the highest network profits and lowest blocking probability compared to other self-learning algorithms.