Efficient Online Learning Based Cross-Tier Uplink Scheduling in HetNets

Abstract

Heterogeneous cellular networks (HetNets), where low-power low-complexity base stations (Pico-BSs) are deployed inside the coverage of macro base stations (Macro-BSs), can significantly improve the spectrum efficiency by Pico- and Macro base station collaboration. Due to cross-tier interference, joint detection of uplink signals is widely adopted so that Pico-BS can either detect the uplink signals locally or forward them to Macro-BS for processing. The latter can achieve increased throughput at the cost of additional backhaul transmission. In this paper, we study the delay-optimal uplink scheduling problem in HetNets with limited backhaul capacity. Local signal detection or joint signal detection is scheduled in a unified delay-optimal framework. Specifically, we first prove that the problem is NP-hard and then formulate it as a Markov Decision Process. We propose an efficient algorithm, called <monospace xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">OLIUS</monospace> , that can deal with the exponentially growing state and action space. Furthermore, <monospace xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">OLIUS</monospace> is online learning-based which does not require any prior knowledge on user behavior or channel characteristics. We prove the convergence of <monospace xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">OLIUS</monospace> and derive an upper bound on its approximation error. Extensive experiments in various scenarios show our algorithm outperforms existing methods in reducing delay and power consumption.