Deeplive: QoE Optimization for Live Video Streaming through Deep Reinforcement Learning

Abstract

A new broad of video services that support live streaming has become tremendously popular in recent years. Compared with traditional video-on-demand (VOD) services, live video streaming has much higher requirements on Quality-of-Experience (QoE), including low rebuffering, high definition, low latency and low bitrate oscillations. While previous adaptive bitrate algorithms (ABR) solely optimize bitrate for ensuring QoE of VOD, live video streaming has a larger decision space, making the optimization problem more difficult to solve. We propose Deeplive, which maximizes QoE through deep reinforcement learning (DRL), so it does not rely on fixed rules. To accelerate the training process of Deeplive, we further propose optimization including window completion with historical data and quick-start with rate-based algorithm. We compare Deeplive with other advanced ABR algorithms in a frame-level dynamic adaptive video streaming simulator using different network traces, QoE definitions, and video categories. In all experiments, we find that Deeplive not only has significant improvement in training time, but also shows an average of 15-55% improvement on QoE than the state-of-the-art ABR algorithms.