Traffic-Aware Online Network Selection in Heterogeneous Wireless Networks

Abstract

The authors focus on the network selection problem in heterogeneous wireless networks. Many traditional approaches select the best network according to quality of service (QoS)-related criteria, which neglects diverse user demands. The authors aim to select networks maximizing the quality of experience (QoE) of users. When the availability and dynamics of network state information (NSI) are considered, most of the existing approaches cannot make effective selection decisions since they are vulnerable to the uncertainty in NSI. To address this issue, the authors introduce the idea of online learning for network selection. In this paper, the authors formulate the network selection problem as a continuous-time multiarmed bandit (CT-MAB) problem. A traffic-aware online network selection (ONES) algorithm is designed to match typical traffic types of users with respective optimal networks in terms of QoE. Moreover, the authors found that the correlation among multiple traffic network selections can be exploited to improve the learning capability. This motivates the authors to propose another two more efficient algorithms: the decoupled ONES (D-ONES) algorithm and the virtual multiplexing ONES (VM-ONES) algorithm. Simulation results demonstrate that the authors ONES algorithms attain around 10% gain in QoE reward rate over nonlearning-based algorithms and learning-based algorithms without QoE considerations.