Embracing Complexity: Agent-Based Modeling for HetNets Design and Optimization via Concurrent Reinforcement Learning Algorithms

Abstract

Complexity is an inherent property in wireless heterogeneous networks (HetNets). In this paper, we investigate the application of the agent-based modeling (ABM) tool for optimization of complex and dynamic HetNets. The proposed framework contains a diversity of game-theoretic, machine learning, and rule-based algorithms within the same model. We present and analyze a HetNet ABM model that runs parallel reinforcement learning (RL) algorithms for spectrum deployment, interference management, resource allocation, and load balancing at both micro and macrocell levels. In our proposed model, two RL-based algorithms work jointly to manage the co-tier and cross-tier interferences. The macrocell runs the first algorithm to control the transmission power of the small cells. The second RL algorithm is run by small cells to assign the users to the sub-bands with less interference levels. Simultaneously, the user association is decided by the users depending on the available resources at the cells and user preferences. The model is then evaluated under various network load conditions to deduce relationships between the cell loads, aggregate bit rate, latency, and user association. Moreover, the system is assessed in a dynamic network scenario with moving users and is confirmed to possess the ability to attain convergence with sufficient performance levels.