QoE-Based Performance Evaluation for Adaptive Media Playout Systems

Mingfu Li

doi:10.1155/2013/152359

Abstract

To improve the playout quality of video streaming services, several adaptive media playout (AMP) mechanisms were proposed in literature. However, all performance evaluations and comparisons for AMPs were made in terms of quality of service (QoS) metrics. As one knows, there may exist a trade-off between QoS metrics, such as buffer underflow and overflow performance. Thus, it is not sufficient to only evaluate the performance of AMPs in terms of QoS metrics. In this paper, we will evaluate and compare the performance of several AMPs from the aspect of quality of experience (QoE). Numerical results will show that some existing AMP systems do not perform better than the nonadaptive playout system from the point of view of overall QoE.

Highlights

Multimedia streaming applications such as IPTV [1] have been increasing rapidly due to the significant growth of bandwidth in access networks, such as xDSL, FTTH, 3G/4G, and WiMAX
This study introduces a quality of experience (QoE)-based evaluation method for adaptive media playout (AMP) systems
The overall QoE of an AMP system can be computed according to the proposed integrated QoE function with multivariate quality of service (QoS) metrics

Summary

Introduction

Multimedia streaming applications such as IPTV [1] have been increasing rapidly due to the significant growth of bandwidth in access networks, such as xDSL, FTTH, 3G/4G, and WiMAX. Several objective QoS metrics, such as underflow probability, overflow probability, variance of distortion of playout (VDoP) [3, 7], initial playout delay, playout curve, and mean playout rate have been used for evaluating the performance of AMP mechanisms. It is sometimes difficult to definitely judge whether an AMP scheme is better than the other one in terms of QoS metrics Most importantly, these objective QoS metrics cannot directly reflect the users’ perception on the playout quality of an AMP system. A real time and cost-effective QoE assessment scheme which takes the impact of AMP into account was proposed in [20]. The considered QoS metrics include the initial playout delay, packet loss rate, underflow time ratio, and playout rate These objective QoS metrics can be obtained by using network measurement equipment.

Overview of Adaptive Media

QoE Evaluation Method for AMP Systems

QoE Assessment Results of AMP Systems

Conclusions