Cross-Frame Resource Allocation With Context-Aware QoE Estimation for 360° Video Streaming in Wireless Virtual Reality

Abstract

Wireless virtual reality (VR), aiming to provide an untethered immersive experience through 360° videos, could be facilitated by viewport-guided streaming with the help of viewport prediction. Although many recent viewport predictors can output a series of predictions over upcoming frames, most existing work on video streaming does not fully utilize the capability of these predictors. In this paper, we investigate the problem of 360° video streaming by incorporating the complete series of viewport predictions for maximizing the quality of experience (QoE) through cross-frame resource allocation. To address the problem of viewport prediction errors that could result in erroneous estimation of QoE contribution of tiles in upcoming frames, we develop a novel approach based on contextual multi-armed bandit (CMAB) to “learn” <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">online</i> the viewing behavior of the user and the capability of the predictor such that resource can be preferentially allocated to tiles with significant QoE contribution. Further, to address the problem of transmission failures during wireless streaming, we formulate a constrained Markov decision process (CMDP) and apply model predictive control (MPC) to account for resource competition among reactive and proactive transmissions as well as retransmissions of tiles. The performance of the proposed streaming system is evaluated using a real-world VR dataset, state-of-the-art viewport predictors, and realistic mmWave channel models. An improvement of 10.2% in QoE and a reduction of 18.7% in resource waste are achieved across various videos, users, and predictors. Simulation results substantiate that the context-aware QoE learned by the proposed CMAB effectively addresses prediction errors for tiles with different temporal and spatial contexts, and the proposed CMDP can achieve the desired performance even under tight bandwidth constraint and severe channel condition.