Reinforcement Learning With Network-Assisted Feedback for Heterogeneous RAT Selection

Abstract

Future wireless networks (e.g., 5G) will consist of multiple radio access technologies (RATs). In these networks, deciding which RAT users should connect to is not a trivial problem. Current fully distributed algorithms although guaranteeing convergence to equilibrium states, are often slow, require high exploration times and may converge to undesirable equilibria. To overcome these limitations, this paper develops a network feedback framework that uses limited network-assisted information to improve efficiency of distributed algorithms for RAT selection problem. We prove theoretically that a fully distributed algorithm developed within this framework is guaranteed to converge to a set of correlated equilibria. Our framework guarantees convergence in self-play even when only a single user applies the algorithm. Simulation results demonstrate that our solution: 1) is highly efficient with fast convergence time and low signaling overheads while achieving competitive, if not better, performance both in fairness and utility, as well as achieving lower per-user switchings than state-of-the-art algorithms; and 2) can flexibly support a wide range of network-assisted feedback. The simulations demonstrate the effectiveness of our solution in a heterogeneous environment, where users may potentially apply a number of different RAT selection procedures.