Multimedia Quality Of Experience Research Articles

New Approach for an Affective Computing- Driven Quality of Experience (QoE) Prediction

In human interactions, emotion recognition is crucial. For this reason, the topic of computer-vision approaches for automatic emotion recognition is currently being extensively researched. Processing multi-channel electroencephalogram (EEG) information is one of the most researched methods for automatic emotion recognition. This paper presents a new model for an affective computing-driven Quality of Experience (QoE) prediction. In order to validate the proposed model, a publicly available dataset is used. The dataset contains EEG, ECG, and respiratory data and is focused on a multimedia QoE assessment context. The EEG data are retained on which the differential entropy and the power spectral density are calculated with an observation window of three seconds. These two features were extracted to train several deep-learning models to investigate the possibility of predicting QoE with five different factors. The performance of these models is compared, and the best model is optimized to improve the results. The best results were obtained with an LSTM-based model, presenting an F1-score from 68% to 78%. An analysis of the model and its features shows that the Delta frequency band is the least necessary, that two electrodes have a higher importance, and that two other electrodes have a very low impact on the model's performances.

Multimedia Quality of Experience (QoE): Current Status and Future Direction

Quality of Experience (QoE) is becoming an important factor of User-Centred Design (UCD). The deployment of pure technical measures such as Quality of Service (QoS) parameters to assess the quality of multimedia applications is phasing out due to the failure of those methods to quantify true user satisfaction. Though significant research results and several deployments have occurred and been realized over the last few years, focusing on QoE-based multimedia technologies, several issues both of theoretical and practical importance remain open. Accordingly, the papers of this Special Issue are significant contribution samples within the general ecosystem highlighted above, ranging from QoE in the capture, processing and consumption of next-generation multimedia applications. In particular, a total of five excellent articles have been accepted, following a rigorous review process, which address many of the aforementioned challenges and beyond.

Comparison of multimedia communications QoE models by Bayesian networks and Bayesian statistics: a case study

This paper presents a comparison of Bayesian Network (BN) and Bayesian Statistics (BS) modeling for QoE (Quality of Experience) estimation and prediction in multimedia communications, with special attention to prediction. As an example of the comparison, we employ a haptic-audiovisual interactive communication system with guaranteed bandwidth. The QoE measure adopted here is subjects’ overall satisfaction (average score) of performing an interactive task under conditions specified by combinations of the video guaranteed bandwidth, video encoding bit rate, receiver’s playout buffering time and gender of each subject. For BN modeling, we utilize an R package bnlearn and create a discrete BN model of a directed acyclic graph with four nodes corresponding to the four parameters. For BS modeling, we build (1) a Bayesian hierarchical regression model with covariates of the four parameters and random effect terms reflecting users’ individualities and gender, and (2) a Bayesian regression model without the random effect terms. The two BS models are analyzed by Markov chain Monte Carlo (MCMC) simulation with the software OpenBUGS. We then find that the BN and BS models provide approximately the same estimates of the QoE measure. Regarding the prediction, however, the BS model with random effect terms outperforms the BN model and BS model without random effect terms. We thus learn that the random effect terms enhance the ability of Bayesian approaches in QoE prediction.

CrowdWatcher: an open-source platform to catch the eye of the crowd

This paper presents the open-source eye tracking platform CrowdWatcher. It enables researchers to measure gaze location and user engagement in a crowdsourcing context through traditional RGB webcams. The proposed platform particularly advances the field of Quality of Experience (QoE) research, as it allows the experimenter to collect remotely and with very limited effort novel information from crowds of participants, such as their commitment towards a task, attention and decision-making processes. Two different experiments are described that were conducted to demonstrate the platform’s potential. The first experiment addresses the measurement of participants’ behavior while performing a movie selection task. Results show that the platform provides complementary information to traditional self-reported data by taking gaze analysis into account. This is of particular relevance, since in a crowdsourcing context decision processes and attention are difficult to assess, and there is often limited control over the engagement of the test user with the task. A second experiment is conducted in the scenario of a multimedia QoE test. Prediction accuracy is compared to a professional infrared eye tracker. While CrowdWatcher performs less well than the professional eye tracker, it is still able to collect valuable gaze information in the far more challenging environment of crowdsourcing. As an outlook to further application domains, the usage of the platform to measure user engagement allows participants who do not pay attention to the task to be identified.

A survey of data-driven approach on multimedia QoE evaluation

With the development of mobile communication technology and the growth of mobile device, the requirements for user quality of experience (QoE) become higher and higher. Network operators and content providers are interested in QoE evaluation for improving users’ QoE. However, multimedia QoE evaluation faces severe challenges due to the subjective properties of the QoE. In this paper, we provide a survey of the state of the art about applying data-driven approach on QoE evaluation. Firstly, we describe the way to choose factors influencing QoE. Then we investigate and discuss the strengths and shortcomings of existing machine learning algorithms for modeling and predicting users’ QoE. Finally, we describe our research work on how to evaluate QoE in imbalanced dataset.

Collaborative Multimedia Source-Protocol Coordination: A Cross-Layer QoE Study in Modern Wireless Networks

Multiple image/video source neighbors in modern wireless systems such as device-to-device (D2D) networks can significantly improve multimedia quality of experience (QoE) in content sharing. However, the exploration of multisource diversity in multimedia communications has largely been overlooked in the past. In this paper, we propose a new scheme to improve user equipment's (UE) multimedia QoE by optimizing the source traffic dispatching and resource allocation strategies from multiple UE sources in cross-layer wireless D2D networks while assuring the latency constraint. This approach results in two major contributions. First, the proposed approach optimally controls the source server traffic for each multimedia packet with regard to distortion reduction (e.g., quality contribution) and the channel conditions. Second, optimal packet retry limits are adjusted on each direct D2D communication link to achieve effective but unequal resource allocation. Simulation results show that the cross-layer collaborative source-level traffic control and protocol-level resource optimization strategy improve D2D multimedia transmission quality.

A user-centered taxonomy of factors contributing to the listener experience of reproduced audio

The traditional paradigm for the assessment of audio quality is that of a listener positioned in the geometric center of a standardized loudspeaker setup, fully attending to the reproduced sound scene. However, this is not how listeners generally interact with audio technology. Audio is consumed in a variety of environments and situations, over devices with varying quality, and by listeners with different expectations and needs. Drawing on research from soundscapes, human computer interaction, and multimedia quality of experience, this paper proposes a user-centered taxonomy of factors that influence the listener experience of reproduced audio. The taxonomy is supported by data from recent research into the perception of complex reproduced sound scenes, and by new data from a web-based survey investigating the structure of experiences with audio technology. In this survey, participants were asked to consider previous experiences with audio technology, and data were collected on the experience itself (psychological need fulfillment and affect), perceptual attributes related to the reproduced audio, and the importance of audio quality to the experience. Results point toward a model of listener experience that can be used to profile listener experiences in different contexts and can be used as a measurement tool in future controlled experiments.

Guest editorial: Advances in tools, techniques and practices for multimedia QoE

It has been realized that the success of multimedia services or applications relies on the analysis of the entire user experience (UX). The relevance of this paradigm ranges from Internet protocol television to video-on-demand systems for distributing and sharing professional television (TV) and user-generated content that is consumed and produced ubiquitously. To obtain a pleasurable user experience, a large amount of aspects have to be taken into account. Major challenges in this context include the identification of relevant UX factors and the quantification of their influence on Quality of Experience (QoE). This special issue is dedicated to advances in, tools, techniques and practices for multimedia QoE that tackle several of the aforementioned challenges.

Interference Allocation Scheduler for Green Multimedia Delivery

One of the key challenges in wireless networking is how to allocate the available radio resources in order to maximise key service delivery parameters such as the aggregate throughput and the multimedia quality of experience (QoE). We propose a novel and effective scheduling policy that allocates resource blocks, such that interference power is shifted towards capacity-saturated users, while improving the throughput of unsaturated users. The highlight of the research is that the proposed scheme can dramatically improve the performance of cells that have a high discrepancy in its signal-to-noise-ratio (SNR) distribution, which is typical in urban areas. The results show that a \emph{free-lunch} solution is possible, whereby for a negligible performance degradation in the saturated users, a large improvement in the non-saturated users can be obtained. However, on average, the number of free-lunch user pairings are low. By relaxing the degradation constraints, the \emph{non-free-lunch} solution can yield a greater multi-user throughput gain. Motivated by surge in mobile multimedia traffic, we further demonstrate that the proposed scheduling may have a profound impact on both energy efficiency and QoE of multimedia service delivery.

Crowdsourcing Multimedia QoE Evaluation: A Trusted Framework

Crowdsourcing has emerged in recent years as a potential strategy to enlist the general public to solve a wide variety of tasks. With the advent of ubiquitous Internet access, it is now feasible to ask an Internet crowd to conduct QoE (Quality of Experience) experiments on their personal computers in their own residences rather than in a laboratory. The considerable size of the Internet crowd allows researchers to crowdsource their experiments to a more diverse set of participant pool at a relatively low economic cost. However, as participants carry out experiments without supervision, the uncertainty of the quality of their experiment results is a challenging problem. In this paper, we propose a crowdsourceable framework to quantify the QoE of multimedia content. To overcome the aforementioned quality problem, we employ a paired comparison method in our framework. The advantages of our framework are: 1) trustworthiness due to the support for cheat detection; 2) a simpler rating procedure than that of the commonly-used but more difficult mean opinion score (MOS), which places less burden on participants; 3) economic feasibility since reliable QoE measures can be acquired with less effort compared with MOS; and 4) generalizability across a variety of multimedia content. We demonstrate the effectiveness and efficiency of the proposed framework by a comparison with MOS. Moreover, the results of four case studies support our assertion that the framework can provide reliable QoE evaluation at a lower cost.

Multi-Resolution Multimedia QoE Models for IPTV Applications

Internet television (IPTV) is rapidly gaining popularity and is being widely deployed in content delivery networks on the Internet. In order to proactively deliver optimum user quality of experience (QoE) for IPTV, service providers need to identify network bottlenecks in real time. In this paper, we develop psycho-acoustic-visual models that can predict user QoE of multimedia applications inreal timebased on online network status measurements. Our models are neural network based and cater to multi-resolution IPTV applications that include QCIF, QVGA, SD, and HD resolutions encoded using popular audio and video codec combinations. On the network side, our models account for jitter and loss levels, as well as router queuing disciplines: packet-ordered and time-ordered FIFO. We evaluate the performance of our multi-resolution multimedia QoE models in terms of prediction characteristics, accuracy, speed, and consistency. Our evaluation results demonstrate that the models are pertinent for real-time QoE monitoring and resource adaptation in IPTV content delivery networks.

QoE-Driven Cross-Layer Optimization for High Speed Downlink Packet Access

This paper proposes a Quality of Experience (QoE) based cross-layer design (CLD) framework for High Speed Downlink Packet Access (HSDPA). The proposed scheme aims at maximizing the user satisfaction by taking advantage of the link adaptation mechanism of HSDPA and the rate adaptation capability of multimedia applications. The main contributions of the paper are as follows. First, we describe the multiuser rate region of HSDPA by constructing a long-term radio link layer model. Next, we formulate multimedia QoE by constructing long-term utility functions, describe the multiuser utility space and derive its properties. We show analytically that the maximization of the sum of utility (max-MOS) can be efficiently solved by a fast greedy algorithm which searches only through the boundary of the utility space. We investigate two alternatives to the max-MOS approach, which introduce additional fairness in the system. We compare our proposed QoE-based cross layer optimization schemes to a system that is configured to maximize the overall throughput. For the sake of completeness, we also compare our approaches to a non-optimized HSDPA system. The performance comparison is made by simulation using a software implementation of an actually deployed HSDPA system. Results show that our QoE-based approach leads to significantly improved user perceived quality compared to the other approaches.