Video Quality Of Experience Research Articles

ARBITER+: Adaptive Rate-Based InTElligent HTTP StReaming Algorithm for Mobile Networks

Dynamic adaptive streaming over HTTP (DASH) is widely adopted for video transport by major content providers. However, the inherent high variability in both encoded video and network rates represents a key challenge for designing efficient adaptation algorithms. Accommodating such variability in the adaptation logic design is essential for achieving a high user quality of Experience (QoE). In this paper, we present ARBITER+ as a novel adaptation algorithm for DASH. ARBITER+ integrates different components that are designed to ensure a high video QoE while accommodating inherent system variabilities. These components include a tunable adaptive target rate estimator, hybrid throughput sampling, controlled switching, and short-term actual video rate tracking. We extensively evaluate the streaming performance using real video and cellular network traces. We show that ARBITER+ components work in harmony to balance temporal and visual QoE aspects. Additionally, we show that ARBITER+ enjoys a noticeable QoE margin in comparison to state-of-the-art adaptation approaches in various operating conditions. Furthermore, we show that ARBITER+ also achieves the best application-level fairness when a group of mobile video clients shares a cellular base station.

Enhancing QoE for Mobile Users by Environment-Aware HTTP Adaptive Streaming.

HTTP adaptive streaming (HAS) has become a dominated media streaming paradigm in today’s Internet, which enriches the user’s experience by matching the video quality with the dynamic network conditions. A range of HAS mechanisms have been proposed to enhance the Quality of Experience (QoE). However, existing mechanisms ignore the environmental impact in the QoE evaluation of mobile users, while the popularity of mobile video allows users to watch videos in diversified scenarios. In this paper, we propose an environment-aware HAS scheme that fully concentrates on the different criteria for evaluating video QoE under different environments. Using the advantage of the sensors in mobile phones, the scheme constructs and validates a video QoE model based on environment perception and then designs a model-driven, environment-aware HAS rate adaptation algorithm. We also evaluate the scheme with an environment-aware DASH (Dynamic Adaptive Streaming over HTTP) player in real mobile environments. Compared to the benchmark HAS mechanism, the experimental results demonstrate that our scheme can provide appropriate differentiated rate adaptation for different environments, resulting in a higher QoE.

Two-phase rate adaptation strategy for improving real-time video QoE in mobile networks

With the popularity of smart handheld devices, mobile streaming video has multiplied the global network traffic in recent years. A huge concern of users' quality of experience (QoE) has made rate adaptation methods very attractive. In this paper, we propose a two-phase rate adaptation strategy to improve users' real-time video QoE. First, to measure and assess video QoE, we provide a continuous QoE prediction engine modeled by RNN recurrent neural network. Different from traditional QoE models which consider the QoE-aware factors separately or incompletely, our RNN-QoE model accounts for three descriptive factors (video quality, rebuffering, and rate change) and reflects the impact of cognitive memory and recency. Besides, the video playing is separated into the initial startup phase and the steady playback phase, and we takes different optimization goals for each phase: the former aims at shortening the startup delay while the latter ameliorates the video quality and the rebufferings. Simulation results have shown that RNN-QoE can follow the subjective QoE quite well, and the proposed strategy can effectively reduce the occurrence of rebufferings caused by the mismatch between the requested video rates and the fluctuated throughput and attains standout performance on real-time QoE compared with classical rate adaption methods.

Deep Learning Model for Multimedia Quality of Experience Prediction Based on Network Flow Packets

Quality of experience (QoE) is the overall acceptability of an application or service, as perceived subjectively by the end user. In particular, for video quality the QoE is dependent of video transmission parameters. To monitor and control these parameters is critical in modern network management systems, but it would be better to be able to monitor the QoE itself (in terms of both interpretation and accuracy) rather than the parameters on which it depends. In this article we present the first attempt to predict video QoE based on information directly extracted from the network packets using a deep learning model. The QoE detector is based on a binary classifier (good or bad quality) for seven common classes of anomalies when watching videos (blur, ghost, etc.). Our classifier can detect anomalies at the current time instant and predict them at the next immediate instant. This classifier faces two major challenges: first, a highly unbalanced dataset with a low proportion of samples with video anomaly, and second, a small amount of training data, since it must be obtained from individual viewers under a controlled experimental setup. The proposed classifier is based on a combination of a convolutional neural network (CNN), recurrent neural network, and Gaussian process classifier. Image processing, which is the common domain for a CNN, has been expanded to QoE detection. Based on a detailed comparison, the proposed model offers better performance metrics than alternative machine learning algorithms, and can be used as a QoE monitoring function in edge computing.

Feature-based prediction of streaming video QoE: Distortions, stalling and memory

Mobile streaming video data accounts for a large and increasing percentage of wireless network traffic. The available bandwidths of modern wireless networks are often unstable, leading to difficulties in delivering smooth, high-quality video. Streaming service providers such as Netflix and YouTube attempt to adapt their systems to adjust in response to these bandwidth limitations by changing the video bitrate or, failing that, allowing playback interruptions (stalling). Being able to predict end users’ quality of experience (QoE) resulting from these adjustments could lead to perceptually-driven network resource allocation strategies that would deliver streaming content of higher quality to clients, while being cost effective for providers. To this end, a number of QoE predictors have been developed, but they do not always capture the interplay between video quality and stalling.Towards more effectively predicting user QoE, we have developed a QoE prediction model called Video Assessment of TemporaL Artifacts and Stalls (Video ATLAS), which is a feature-based approach that combines a number of QoE-related features, including perceptually-relevant quality features, stalling-aware features and memory-driven features to make QoE predictions. We evaluated Video ATLAS on the recently designed LIVE-Netflix Video QoE Database which consists of practical playout patterns, where the videos are afflicted by both quality changes and stalling events, and found that it provides improved performance over state-of-the-art video quality metrics while generalizing well on a different dataset. The proposed algorithm is made publicly available at http://live.ece.utexas.edu/research/VideoATLAS/vatlas_index.html.

Recurrent and Dynamic Models for Predicting Streaming Video Quality of Experience

Streaming video services represent a very large fraction of global bandwidth consumption. Due to the exploding demands of mobile video streaming services, coupled with limited bandwidth availability, video streams are often transmitted through unreliable, low-bandwidth networks. This unavoidably leads to two types of major streaming-related impairments: compression artifacts and/or rebuffering events. In streaming video applications, the end-user is a human observer; hence being able to predict the subjective Quality of Experience (QoE) associated with streamed videos could lead to the creation of perceptually optimized resource allocation strategies driving higher quality video streaming services. We propose a variety of recurrent dynamic neural networks that conduct continuous-time subjective QoE prediction. By formulating the problem as one of time-series forecasting, we train a variety of recurrent neural networks and non-linear autoregressive models to predict QoE using several recently developed subjective QoE databases. These models combine multiple, diverse neural network inputs, such as predicted video quality scores, rebuffering measurements, and data related to memory and its effects on human behavioral responses, using them to predict QoE on video streams impaired by both compression artifacts and rebuffering events. Instead of finding a single time-series prediction model, we propose and evaluate ways of aggregating different models into a forecasting ensemble that delivers improved results with reduced forecasting variance. We also deploy appropriate new evaluation metrics for comparing time-series predictions in streaming applications. Our experimental results demonstrate improved prediction performance that approaches human performance. An implementation of this work can be found at https://github.com/christosbampis/NARX_QoE_release.

SenseVid : A traffic trace based tool for QoE Video transmission assessment dedicated to Wireless Video Sensor Networks

Video applications are being a key component to enhance traditional wireless sensor networks (WSN) applications. As a result, sensor network researchers need adequate and easy to use tools to assess the performances of their proposals. The existing transmission evaluation tools either consider video sequences along with codecs that are unsuitable for WSNs or make use of low cost compression methods for still images without inter-frame coding required for efficient video transmission. In this paper, we present SenseVid, an open source video transmission and evaluation tool that considers WSN specific characteristics. Besides low energy intra-frame compression based on fast pruned discrete cosine transform (DCT), a low complexity inter-frame encoding is provided to allow efficient support of video flows. A configurable fine-grain energy model is provided where both video capture and encoding cost are accounted for on a per frame basis. Video traffic differentiation based on priority levels is also provided. SenseVid adopts the video traffic traces approach, allowing its use in any simulation or real testbed environment. Using SenseVid, the user is able to reconstruct the received video considering lost packets during its transmission as well as estimating the achieved quality of service (QoS) and quality of experience (QoE).

A Quality-of-Experience Database for Adaptive Video Streaming

The dynamic adaptive streaming over HTTP provides an inter-operable solution to overcome the volatile network conditions, but its complex characteristic brings new challenges to objective video quality-of-experience (QoE) measurement. To test the generalizability and to facilitate the wide usage of QoE measurement techniques in real-world applications, we establish a new database named Waterloo Streaming QoE Database III (SQoE-III). Unlike existing databases constructed with hand-crafted test sequences, the SQoE-III database, so far the largest and most realistic of its kind, consists of a total of 450 streaming videos created from diverse source content and diverse distortion patterns, with six adaptation algorithms of diverse characteristics under 13 representative network conditions. All streaming videos are assessed by 34 subjects, and a comprehensive evaluation is conducted on the performance of 15 objective QoE models from four categories with regards to their efficacy in predicting subjective QoE. Detailed correlation analysis and statistical hypothesis testing are carried out. The results of this paper shed light on the future development of adaptive bitrate streaming algorithm and video QoE monitoring system. The subjective database is available online at https://ece.uwaterloo.ca/~zduanmu/tbc2018qoe/ .

Learning a Continuous-Time Streaming Video QoE Model

Over-the-top adaptive video streaming services are frequently impacted by fluctuating network conditions that can lead to rebuffering events (stalling events) and sudden bitrate changes. These events visually impact video consumers' quality of experience (QoE) and can lead to consumer churn. The development of models that can accurately predict viewers' instantaneous subjective QoE under such volatile network conditions could potentially enable the more efficient design of quality-control protocols for media-driven services, such as YouTube, Amazon, Netflix, and so on. However, most existing models only predict a single overall QoE score on a given video and are based on simple global video features, without accounting for relevant aspects of human perception and behavior. We have created a QoE evaluator, called the time-varying QoE Indexer, that accounts for interactions between stalling events, analyzes the spatial and temporal content of a video, predicts the perceptual video quality, models the state of the client-side data buffer, and consequently predicts continuous-time quality scores that agree quite well with human opinion scores. The new QoE predictor also embeds the impact of relevant human cognitive factors, such as memory and recency, and their complex interactions with the video content being viewed. We evaluated the proposed model on three different video databases and attained standout QoE prediction performance.

Video encoding adaptation for QoE maximization over 5G cellular networks

With the unprecedented growth of mobile video data traffic every year, increasing users' video quality of experience (QoE) under limited network radio resources becomes a critical issue in next-generation cellular systems. However, videos compressed with unsuitable video encoding parameters will waste the network radio resources and users even may not be able to get satisfied video QoE. Specifically, a video with higher bit rate requirement (determined by the video encoding parameters) does not imply that a user will have better QoE. This paper studies the QoE optimization problem via carefully determining a combination of video encoding parameters for each user under the radio resource block constraint. The objective is to maximize the total QoE of all users. We prove that our target problem is NP-hard and propose an algorithm based on dynamic programming to solve the problem. Then, we prove that our proposed algorithm is a pseudo-polynomial time optimal algorithm. We construct a series of simulations with realistic video sequences encoded by H.264 and network settings to evaluate the performance of our proposed algorithm. Compared with two baselines, the simulation results show that the proposed algorithm can significantly improve the total QoE of all users and indicate that the video encoding adaptation is an important issue in the QoE optimization problem.

Learning from users: a data-driven method of QoE evaluation for Internet video

Improving quality of experience (QoE) is increasingly significant for Internet video content providers. The essential issue is how to evaluate QoE under the complex circumstance of Internet video. Based on the massive user data extracted from a large scale Video-on-Demand (VoD) provider, we present a data-driven, comprehensive and extendible study on the problems of QoE evaluation. The main works of this paper include obtaining QoE-associated features via feature engineering and building an evaluation model on features of different aspects for Internet video QoE. Firstly, for feature engineering, we propose to introduce pattern features of user viewing behaviors that interact with user-perceived video quality. A new method of frequent time series pattern mining is proposed to find typical patterns. We correlate user experience with user-perceived quality features and user behavior pattern features, and consider the impact of confounding factors by applying them as context features into modeling. Secondly, interdependency among features is challenging for QoE evaluation modeling. And the high dimension of feature vector should be considered. To address these challenges, we develop an ensemble method to model the interactions between features and their intricate relationships to user experience. Experiments demonstrate that our approach could achieve sound results in comparison with other related works.

Modelling user quality of experience from objective and subjective data sets using fuzzy logic

One of the paramount research questions in the scientific community today is how to remotely assess user quality of experience (QoE) for a specific service. To this end, various user QoE assessment models have been developed; however, they are mostly based on the data gathered from controlled environment experimentation. The aim of this research was to model user QoE for the User Datagram Protocol-based video streaming service from the results of uncontrolled subjective tests. Specifically, using fuzzy logic, we have correlated the values of three objective network parameters (the packet loss rate and the number and duration of packet loss occurrences in one streaming session) with test subjects’ subjective perception about perceived quality distortions. The dependencies between different values of the parameters and the subjects’ perception of video quality were used to develop a no reference objective video quality assessment model for assessing user QoE. The key distinguishing feature of the developed model lays in the process of subjective evaluation, which was conducted with a panel of 602 test subjects who evaluated the quality of 1-h video in home environments. For this purpose, 72 different test sequences were prepared for rating. We showed that a strong positive linear relationship exists between the assessed QoE of the model and the Mean Opinion Scores of the subjects (a Pearson correlation coefficient equal to 0.8841).

Enhancing Transparency: Internet Video Quality Inference from Network Traffic

The 2017 FCC Restoring Internet Freedom Order removes the enhanced transparency obligations introduced by the 2015 Open Internet Order and aims to return net neutrality policy to transparency rules based on the 2010 Open Internet Order. The ruling states that the burden of additional network performance disclosures exceed the benefits, and that the most salient metrics to report are those that involve consumer quality of experience (QoE) for the applications that they commonly use. Unfortunately, however, internet service providers (ISPs) will typically have difficulty reporting on application performance and QoE metrics, both of which are notoriously difficult to estimate from network traffic. To address this shortcomings, we present the initial development of Network Microscope, a tool that estimates QoE for Internet video streaming from passively collected network traffic. Our system sits inline, on path, and analyzes traffic in real time as it traverses the network to (1) identify which traffic flows belong to a specific video streaming service; (2) estimate critical quality-of-experience metrics for streaming video such as: bitrate, changes in bitrate, startup delay, and rebuffering. When deployed on a commodity embedded device, the tools is suitable for deployment in consumer home networks, as well as near various network endpoints. Because Internet video traffic accounts for majority of the global internet traffic, this approach of passively observing traffic has two significant policy implications: - It reduces the administrative and operational burden on ISPs, because traffic collection and analysis is passive, in-line, and in homes, and does not introduce additional test traffic. - The approach offers application QoE metrics that are complementary to the lower-level network performance metrics that ISPs already collect. We discuss the capabilities of Network Microscope, its initial deployment to over 50 consumer homes, our initial findings concerning the reporting of application QoE metrics, and broader implications for policy surrounding ISP transparency reporting requirements. Network Microscope can provide a deeper understanding of several concepts often discussed both in the context of net neutrality and encouraging competitive forces in the market. First, such a tool can shed more light on the nature of streaming traffic from consumers, including which video streaming services are most popular with consumers, and how those popular services perform on different networks. Measurements based on service-specific usage is particularly meaningful to consumers because consumers often understand their network needs better when it is tied to the applications they use often. More specific information about the performance of popular video services can help consumers make more informed choices about the network services that they purchase. Second, the tool facilitates the analysis of application performance for network traffic at multiple locations along a single end-to-end path, enabling both consumers and regulators to independently verify ISP reports about application performance. Ultimately, the type of information about application performance that our tool exposes can affect consumer decision-making; we explore and discuss these effects, and how they may ultimately interact with switching costs, market competition, and other commercial considerations.

Assessing the Quality of Videoconferencing: from Quality of Service to Quality of Communication

Various approaches for the assessment of quality of experience for video conferencing exist that try to quantify the level of satisfaction of the user. In order for videoconferencing to really succeed as a substitute of face-to-face meetings, it is constructive to explore the associated quality of communication. One of the most significant factors to ascertain in this regard would be the assessment of the level of user satisfaction for a videoconference instead of a face-to-face meeting. Various recommendations of ITU-T related to this field (such as ITU-T Recommendation P.920) deal with subjective experiments that involve performing interactive tasks in order to quantify the impact of terminal and communication link performance. In addition to looking at different quality paradigms, in this paper, we review a number of subjective studies on videoconferences and investigate the possibilities of discovering more of the user feedback in order to assess the perceived quality of communication experienced in a videoconference. Based on the review, we present a set of future work items which can be useful in finding a comprehensive definition for quality of communication.

Quality of Experience (QOE) Aware Video Attributes Determination for Mobile Streaming Using Hybrid Profiling

<p>Today, consumers use a smartphone device to display the media contents for work and entertainment purposes, as well as watching online video. Online video streaming is the main cause that consume smartphone’s energy quickly. To overcome this problem, smartphone’s energy management is crucial. Thus, a hybrid energy-aware profiler is proposed. Basically, a profiler will monitor and manage the energy consumption in the smartphone devices. The hybrid energy-aware profiler will set up a protocol preference of both the user and the device. Then, it will estimates the energy consumption in smartphone. However, saving energy alone can contribute to the Quality of Experience (QoE) neglection, thus the proposed solution takes into account the client QoE. Even though there are several existing energy-aware profilers that have been developed to manage energy use in smartphones however, most energy-aware profilers does not consider QoE at the same time. The proposed solution consider both, the performance of the hybrid energy-aware profiler is compared with the baseline energy models against a variation of content adaptation according to the pre-defined variables. Three types of variables were determined; resolution, frame rate and energy consumption in smartphone devices. In this area, QoE subjective methods based on MOS (Mean Opinion Score) are the most commonly used approaches for defining and quantifying real video quality. Nevertheless, although these approaches have been established to consistently quantify users’ amounts of approval, they do not adequately realize which are the criteria of video attribute that important. In this paper, we conducted an experiment with a certain devices to measures user’s QoE and energy usage of video attribute in smartphone devices. Our results demonstrate that the list of possible solution is a relevant and useful video attribute that satify the users.</p>

Testing the Effects of Peak, End, and Linear Trend on Evaluations of Online Video Quality of Experience

While previous research has shown that the sequencing of good and bad experience is an important predictor of overall evaluations of a set of experiences, the impact of sequencing effects on the experience of viewing online video has yet to be established. The aim of this study was to determine whether different sequences of good (G), mediocre (M), and bad (B) quality videos in different blocks would influence overall ratings after viewing those blocks. Thirty-two participants each watched 10 blocks of 4 videos and provided ratings of technical quality (TQ), satisfaction, and frustration for each video in the block, as well as overall ratings for each block (as a whole). Sequences of G, M, and B videos were designed to test whether block characteristics (features), like the peak-end effect and effect of linear trend, influenced summary evaluations of the block service. The results of the experiment show that overall block TQ, satisfaction, and frustration ratings differed significantly by sequencing feature. Difference scores were used to determine whether the features had an effect on overall evaluations beyond what could be explained by the total number of bad videos in the block or the average ratings of the videos in the block. Results showed a significant end effect for negative ends of a block, and an effect of linear trend (both increasing and decreasing). There was no evidence of a peak effect or an end effect for positive ends. The presence of a negative end effect and effect of linear trend indicate that where possible service providers should avoid service sessions with poor service quality at the end, or sessions that have decreasing quality as the session progresses.

A token-based incentive mechanism for video streaming applications in peer-to-peer networks

Free-riding is one of the main challenges of Peer-to-Peer (P2P) streaming systems which results in reduction in video streaming quality. Therefore, providing an incentive mechanism for stimulating cooperation is one of the essential requirements to maintain video Quality of Experience (QoE) in such systems. Among the existing mechanisms, payment-based schemes are most suitable for streaming applications due to their low overhead. However, to date, no dynamic payment mechanism has been proposed which can take the stochastic dynamics of the video streaming ecosystem (e.g., the request arrival, demand submission, bandwidth availability, etc.) into account. In this paper, we propose a dynamic token-based payment mechanism in which each peer earns tokens by admitting other peers’ requests and spends tokens for submitting its demands to the others. This system allows the peers to dynamically adjust their income level in adaptation to changes in the system state. We propose a Constrained Markov Decision Process (CMDP) formulation in which the goal of each peer is to obtain a request admission policy which minimizes the expected cumulative cost of consumed bandwidth, while satisfying a long-term constraint on the Mean Opinion Score (MOS) of the users as the measure of QoE. The proposed admission policy is adaptive to the request arrival process, bandwidth state and the token bucket length of each peer. To make up for the lack of design-time knowledge of the system’s statistics, each individual peer is equipped with a model-free algorithm to learn its optimal admission policy over the course of real-time interaction with the system. Simulation results are presented to compare the performance of the proposed algorithm against baseline schemes such as: random, token-threshold, bandwidth-threshold and myopic algorithms.

Study of Temporal Effects on Subjective Video Quality of Experience.

HTTP adaptive streaming is being increasingly deployed by network content providers, such as Netflix and YouTube. By dividing video content into data chunks encoded at different bitrates, a client is able to request the appropriate bitrate for the segment to be played next based on the estimated network conditions. However, this can introduce a number of impairments, including compression artifacts and rebuffering events, which can severely impact an end-user's quality of experience (QoE). We have recently created a new video quality database, which simulates a typical video streaming application, using long video sequences and interesting Netflix content. Going beyond previous efforts, the new database contains highly diverse and contemporary content, and it includes the subjective opinions of a sizable number of human subjects regarding the effects on QoE of both rebuffering and compression distortions. We observed that rebuffering is always obvious and unpleasant to subjects, while bitrate changes may be less obvious due to content-related dependencies. Transient bitrate drops were preferable over rebuffering only on low complexity video content, while consistently low bitrates were poorly tolerated. We evaluated different objective video quality assessment algorithms on our database and found that objective video quality models are unreliable for QoE prediction on videos suffering from both rebuffering events and bitrate changes. This implies the need for more general QoE models that take into account objective quality models, rebuffering-aware information, and memory. The publicly available video content as well as metadata for all of the videos in the new database can be found at http://live.ece.utexas.edu/research/LIVE_NFLXStudy/nflx_index.html.

Continuous Prediction of Streaming Video QoE Using Dynamic Networks

Streaming video data accounts for a large portion of mobile network traffic. Given the throughput and buffer limitations that currently affect mobile streaming, compression artifacts and rebuffering events commonly occur. Being able to predict the effects of these impairments on perceived video quality of experience (QoE) could lead to improved resource allocation strategies enabling the delivery of higher quality video. Toward this goal, we propose a first of a kind continuous QoE prediction engine. Prediction is based on a nonlinear autoregressive model with exogenous outputs. Our QoE prediction model is driven by three QoE-aware inputs: An objective measure of perceptual video quality, rebuffering-aware information, and a QoE memory descriptor that accounts for recency. We evaluate our method on a recent QoE dataset containing continuous time subjective scores.

Scheduling for telemedicine traffic transmission over WLANs

The major disadvantage of the Enhanced Distributed Channel Access (EDCA, the contention-based channel access function of 802.11e) is that it is unable to guarantee priority access to higher priority traffic in the presence of significant traffic loads from low priority users. This problem is enhanced by the continuously growing number of multimedia applications and the popularity of Wireless Local Area Networks (WLANs). Hence, solutions in scheduling multimedia traffic transmissions need to take into account both the Quality of Service (QoS) requirements and the Quality of Experience (QoE) associated with each application, especially those of urgent traffic, like telemedicine, which carries critical information regarding the patients’ condition. In this work, we propose an easy-to-implement token-based and self policing-based scheduling scheme combined with a mechanism designed to mitigate congestion. Our approach is shown to guarantee priority access to telemedicine traffic, to satisfy its QoS requirements (delay, packet dropping) and to offer high telemedicine video QoE while preventing bursty video nodes from over-using the medium.