Design, optimization and evaluation of a q-learning HTTP adaptive streaming client

Abstract

HTTP Adaptive Streaming (HAS) is becoming a key technology for audiovisual broadcasting, although the varying conditions of the networks imply an uneven video quality. So, there are currently many client players endowed with proprietary adaptation algorithms aiming at maximizing the perceived quality. Looking for the best user’s Quality of Experience (QoE), an adaptation algorithm based on the Q-Learning method is proposed. This analysis identifies the variables that best capture the system dynamics and establishes a formulation for the characteristic functions of this Reinforcement Learning method. In addition, suitable parameter configurations for the algorithm are analyzed together with its convergence. Experimental results confirm the ability of the proposed solution to control efficiently the selection of the segment qualities, so that quality switches are minimized and the occurrence of freezes is diminished. Moreover, a performance comparison with other strategies shows that this novel approach outperforms them. Finally, the adaptiveness of this strategy to changing environments is also assessed.