Smart Multi-RAT Access Based on Multiagent Reinforcement Learning

Abstract

The ongoing increasing traffic in the era of big data yields unprecedented demands in user experience and network capacity expansion. The users of next generation mobile networks (5 G) should be able to use 3GPP, IEEE, and other technologies simultaneously. The integration of multiple radio access technologies (RATs) of licensed or unlicensed bands has been widely deemed as a cost-efficient way to greatly increase the network capacity. In this paper, we propose a smart aggregated RAT access (SARA) strategy with aim of maximizing the long-term network throughput while meeting diverse traffic quality of service (QoS) requirements. We consider the scenario that users with different QoS requirements access to a heterogeneous network with coexisting cellular-WiFi. In order to maximize system throughput while meeting diverse traffic QoS requirements in such a complex and dynamic environment, we exploit multiagent reinforcement learning to perform RAT selection in conjunction with resource allocation for individual user access requests, through sensing dynamic channel states and traffic QoS requirements. In SARA, we first use Nash Q-learning to provide a set of feasible RAT selection strategies while decreasing the strategy space in learning process, and then employ Monte Carlo tree search (MCTS) based Q-learning to perform resource allocation. Numerical results reveal that the network throughput can be maximized while meeting various traffic QoS requirements with limited number of searches by using our proposed SARA algorithm. For bulk arrival access requests, a suboptimal solution can be obtained as high computational complexity is incurred for achieving global optimality. Another attractive feature of SARA is that a tradeoff between the solution optimality and learning time can be readily made by terminating the search of MCTS according to the time constraint. Compared with traditional WiFi offloading schemes, SARA can significantly improve network throughput while guaranteeing traffic QoS requirements.