Quality Of Experience Assessment Research Articles

User-centric QoE model of visual perception for mobile videos

It is crucial for service providers to improve user’s quality of visual perception for mobile users. Quality of experience (QoE) is an important perceptual visual metric. In this paper, we propose a user-centric QoE assessment model by joint considering technological-aware and psychology-aware parameters in the QoE communication ecosystem. For technological parameters, video encoding features are extracted from the video stream, and video content feature is estimated by video analysis. Moreover, user interests are also quantitatively collected as psychology parameters. Then, QoE model is developed by using support vector machine (SVM). Subjective tests have been performed. The collected data from subjective tests are used for training and validation of the proposed model. The experiment results show that the proposed user-centric QoE assessment model performs better in terms of high Pearson correlation coefficient (PCC) and low root-mean-square error (RMSE) compared with the conventional models.

A quantitative relationship between Application Performance Metrics and Quality of Experience for Over-The-Top video

Quality of Experience (QoE) is a measure of the overall level of customer satisfaction with a vendor. In telecommunications, consumer satisfaction is of great interest in the adoption of novel multimedia products and services. A number of factors can greatly influence the customer experience during a video session. Factors such as user perception, experience, and expectations are expressed by QoE while factors such as application and network performance are expressed by Quality of Service (QoS) parameters. This paper studies the relationship between QoS and QoE in a session-based mobile video streaming. Specific QoS Application Performance Metrics (APMs) are examined based on a QoE assessment database which is built for experimentation and contains 108 subjects. It is shown that these APMs are highly related to two QoE factors, Technical Quality (TQ) and Acceptability. Furthermore, Viewing Ration (VR) parameter and the corresponding Kendall correlation between VR and QoE factors proves that VR is a valuable metric for mapping QoS to QoE. We further generated the compacted decision tree to predict QoE factors through Rebuffering Ratio (RR), Non-interruption Content Viewing Ratio (VRc), and Non-interruption Viewing Ratio (VRs). Through extensive experimentation, a general relationship between APMs and QoE factors has been examined and a primary QoE model is proposed based on this relationship.

Toward the Assessment of Quality of Experience for Asymmetric Encoding in Immersive Media

The assessment of Quality of Experience (QoE) for stereoscopic 3-D video is a challenging task, especially in 3-D video compression and transmission applications. The focus of this contribution is the development of a QoE assessment framework, in line with the latest standardization progress in the field of QoE assessment, for understanding the visual effect of asymmetric and symmetric encoding for immersive media. Asymmetric stereoscopic video coding exploits the binocular suppression of the human vision system by representing one of the two views with a lower quality. This processing, while of limited effects on image quality, may influence the overall QoE. Many studies show that the QoE of immersive media such as 3-DTV can be thought as the combination of the perceived visual quality, the perceived depth quality, the visual fatigue, and visual discomfort. In this paper, we aim at: 1) exploiting the concept of preference of experience and protocols recently standardized for characterizing QoE; 2) conducting a case study using these standardized protocols to investigate the factors involving visual discomfort in stereoscopic video sequences with a focus on binocular rivalry; and 3) presenting the results of subjective experiments performed, by using the perceptual quality and preference of experience assessment protocols, for evaluating the impact of symmetrical, asymmetrical, and alternate coding schemes.

Modelling user quality of experience from objective and subjective data sets using fuzzy logic

One of the paramount research questions in the scientific community today is how to remotely assess user quality of experience (QoE) for a specific service. To this end, various user QoE assessment models have been developed; however, they are mostly based on the data gathered from controlled environment experimentation. The aim of this research was to model user QoE for the User Datagram Protocol-based video streaming service from the results of uncontrolled subjective tests. Specifically, using fuzzy logic, we have correlated the values of three objective network parameters (the packet loss rate and the number and duration of packet loss occurrences in one streaming session) with test subjects’ subjective perception about perceived quality distortions. The dependencies between different values of the parameters and the subjects’ perception of video quality were used to develop a no reference objective video quality assessment model for assessing user QoE. The key distinguishing feature of the developed model lays in the process of subjective evaluation, which was conducted with a panel of 602 test subjects who evaluated the quality of 1-h video in home environments. For this purpose, 72 different test sequences were prepared for rating. We showed that a strong positive linear relationship exists between the assessed QoE of the model and the Mean Opinion Scores of the subjects (a Pearson correlation coefficient equal to 0.8841).

Adaptive video streaming testbed design for performance study and assessment of QoE

SummaryHypertext Transfer Protocol adaptive streaming switches between different video qualities, adapting to the network conditions, and avoids stalling streamed frames over high‐oscillation client's throughput improving the users' quality of experience (QoE). Quality of experience has become the most important parameter to lead the service providers to know about the end‐user feedback. Implementing Hypertext Transfer Protocol adaptive streaming applications to find out QoE in real‐life scenarios of vast networks becomes more challenging and complex task regarding to cost, agile, time, and decisions. In this paper, a virtualized network testbed to virtualize various machines to support implementing experiments of adaptive video streaming has been developed. Within the test study, the metrics which demonstrate performance of QoE are investigated, respectively, including initial delay (ie, startup delay at the beginning of playback a video), frequency switches (ie, number of times the quality is changed), accumulative video time (ie, number and length of stalls), CPU usage, and battery energy consumption. Furthermore, the relation between effective parameters of QoS on the aforementioned metrics for different segment length is investigated. Experimental results show that the proposed virtualized system is agile, easy to install and use, and costs less than real testbeds. Moreover, the subjective and objective performance studies of QoE evaluation in the system have proven that the segment lengths of 6 to 8 seconds were faired and more efficient than others according to the investigated parameters.

E3DOAS: Balancing QoE and Energy-Saving for Multi-Device Adaptation in Future Mobile Wireless Video Delivery

Smart devices (e.g., smartphones, tablets, smart-home devices, etc.) have become important companions to most people in their daily activities, and are very much used for multimedia content exchange (i.e., video sharing, real-time/non-real-time multimedia streaming), contributing to the exponential increase in mobile traffic over the current wireless networks. While the next generation mobile networks will provide higher capacity than the current 4G systems, the network operators will face important challenges associated with the outstanding increase of both video traffic and user expectations in terms of their levels of perceived quality or quality of experience (QoE). Furthermore, the heterogeneity of mobile devices (e.g., screen resolution, battery life, and hardware performance) also impacts severely the end-user QoE. In this context, this paper proposes an evolved QoE-aware energy-saving device-oriented adaptive scheme (E3DOAS) for mobile multimedia delivery over future wireless networks. E3DOAS makes use of a coalition game-based rate allocation strategy within the multi-device heterogeneous environment, and optimizes the trade-off between the end-user perceived quality of the multimedia delivery and the mobile device energy-saving. Testing has involved a prototype of E3DOAS, a crowd-sourcing-based QoE assessment method to model non-reference perceptual video quality, and an energy measurement testbed introduced to collect power consumption parameters of the mobile devices. Simulation-based performance evaluation showed how E3DOAS outperformed other state of the art multimedia adaptive solutions in terms of energy saving, end-to-end quality of service metrics and end-user perceived quality.

Survey on QoE Assessment Approach for Network Service

With the development of converged all-IP networks, new network services—supporting, among others, image, audio, and video—with high diversity emerge, which brings huge pressure and fierce competition to service providers because of increasing choices for users. Assessing end user’s quality of experience (QoE) has become one of the main issues to meet high user requirements and satisfaction under any circumstances, which means receiving any service, anytime, anywhere, on any device, by any media and networking technology, across any operator domain. In this paper, we provide an overview of the field of QoE assessment approaches including three aspects: subjective assessment, objective assessment, and hybrid assessment. The discussion includes the definition, influence factors, and quantification approaches for QoE. Then, these three assessment methodologies are surveyed and compared across a number of factors. This paper also introduces challenges and opportunities in the area of QoE assessment.

Multimodal Physiological Quality-of-Experience Assessment of Text-to-Speech Systems

With the growing complexity of various text-to-speech systems, it is becoming more important to understand the underlying perceptual and judgement processes that drive user Quality-of-Experience (QoE) perception. Typical QoE assessment techniques, such as listening tests with self-report ratings, are useful but provide limited insight into these underlying processes. Recent advances in neuroimaging and physiological monitoring technologies, however, have opened new doors and allowed us to better understand and measure QoE perception. In this paper, we explore the use of two neuroimaging techniques, namely electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS), to better understand neuronal and cerebral haemodynamic changes resultant from synthesized speech of varying quality. Neural correlates of several QoE dimensions were derived and validated on the publicly available PhySyQX database. Fusion of EEG, fNIRS, and fNIRS-derived physiological parameters, combined with conventional features extracted from the synthesized speech signal showed to accurately represent several QoE dimensions, including those related to listener affective states. It is hoped that these findings will help researchers build better instrumental QoE models that incorporate technological, contextual, and human influence factors.

QoE in Video Transmission: A User Experience-Driven Strategy

The increasing popularity of video (i.e., audio-visual) applications or services over both wired and wireless links has prompted recent growing interests in the investigations of quality of experience (QoE) in online video transmission. Conventional video quality metrics, such as peak-signal-to-noise-ratio and quality of service, only focus on the reception quality from the systematic perspective. As a result, they cannot represent the true visual experience of an individual user. Instead, the QoE introduces a user experience-driven strategy which puts special emphasis on the contextual and human factors in addition to the transmission system. This advantage has raised the popularity and widespread usage of QoE in video transmission. In this paper, we present an overview of selected issues pertaining to QoE and its recent applications in video transmission, with consideration of the compelling features of QoE (i.e., context and human factors). The selected issues include QoE modeling with influence factors in the end-to-end chain of video transmission, QoE assessment (including subjective test and objective QoE monitoring) and QoE management of video transmission over different types of networks. Through the literature review, we observe that the context and human factors in QoE-aware video transmission have attracted significant attentions since the past two to three years. A vast number of high quality works were published in this area, and will be highlighted in this survey. In addition to a thorough summary of recent progresses, we also present an outlook of future developments on QoE assessment and management in video transmission, especially focusing on the context and human factors that have not been addressed yet and the technical challenges that have not been completely solved so far. We believe that our overview and findings can provide a timely perspective on the related issues and the future research directions in QoE-oriented services over video communications.

QoE Assessment of Fairness between Players in Networked Virtual 3D Objects Identification Game Using Haptic, Olfactory, and Auditory Senses

In this paper, we carry out QoE (Quality of Experience) assessment to investigate influences of olfactory and auditory senses on fairness for a networked virtual 3D object identification game with haptics. In the game, two players try to identify objects which are placed in a shared 3D virtual space. In the assessment, we carry out the game in four cases. Smells and sounds are presented in the first case, only sounds are done in the second case, and only smells are done in the third case. In the last case, we present neither smell nor sound. As a result, we demonstrate that the fairness deteriorates more largely as the difference in conditions between two users becomes larger.

Fairness Assessment in Networked Games with Olfactory and Haptic Senses

In this paper, we carry out QoE (Quality of Experience) assessment of fairness between players in a networked balloon bursting game with olfactory and haptic senses. We also make a comparison of the fairness among different types of networked games with olfactory and haptic senses in the assessment. Then, we clarify the differences owing to usage of olfactory and haptic senses among the games. As a result, we illustrate that the fairness is largely affected by the delay difference, and we also demonstrate that the allowable range depends on the type of games, type of senses which are employed in the games, and the play methods.

QoE-Based Analysis of DASH Streaming Parameters Over Mobile Broadcast Networks

Multimedia services over mobile networks are subject to multiple factors that affect the quality of experience (QoE) perceived by users. In a long term evolution (LTE) multimedia broadcast service, these factors are initial delay, technical quality of multimedia content, and service disruptions. However, there is a lack of QoE models to enable QoE measurement. We propose to use video mean opinion score as a new and objective QoE model that is in the process of standardization, and we show how it can be applied to LTE multimedia broadcast services for QoE assessment. The results obtained are useful to better understand the impact of several service parameters on perceived quality. By applying this assessment methodology, it is possible to determine the most adequate service parameters for a specific LTE broadcast deployment.

Research on HAS video QoE assessment technology

As internet video applications, especially HTTP Adaptive Streaming (HAS), gained their popularity all around the world, monitoring and assessment technologies of video-related quality of experience (QoE) were going hot during the last few years. The paper introduces the system architecture and procedures that support HAS video application, and analyses the difference of QoE technologies between traditional streaming video and HAS video. It also reviews the QoE models for current video services, and spots the possible future research directions and challenges, from point of view of client, network and server, respectively, on how HAS QoE would be built and applied in practical employment.

Research on HAS video QoE assessment technology

As internet video applications, especially HTTP Adaptive Streaming (HAS), gained their popularity all around the world, monitoring and assessment technologies of video-related quality of experience (QoE) were going hot during the last few years. The paper introduces the system architecture and procedures that support HAS video application, and analyses the difference of QoE technologies between traditional streaming video and HAS video. It also reviews the QoE models for current video services, and spots the possible future research directions and challenges, from point of view of client, network and server, respectively, on how HAS QoE would be built and applied in practical employment.

QoE assessment and prediction method for high-definition video stream using image damage accumulation

The accuracy of the traditional assessment method of the quality of experience (QoE) has been facing challenges with the growth of high-definition (HD) video streaming services. Image display-quality damage is the main factor that affects the QoE in HD video services through UDP network transmission. In this paper, we introduce a novel objective factor known as image damage accumulation (IDA) to assess user's QoE in HD video services. First, this paper quantitatively analyzed the effect on user quality of experience by IDA and established a mapping relationship between mean opinion scores and IDA. Furthermore, the probability of image damage caused by compression and transmission were analyzed. Based on this analysis, an objective QoE assessment and prediction method for HD video stream service that evaluated the user experience according to IDA are proposed. The proposed method can achieve assessment and prediction accuracy on three distinct subjective tests.

QoE rating performance evaluation of ITU-T recommended video quality metrics in the context of video freezes

In real-time video streaming, video quality can be degraded due to network performance issues. Among other artefacts, video freezing and video jumping are factors that influence user experience. Service providers, operators and manufacturers are interested in evaluating the quality of experience (QoE) objectively because subjective assessment of QoE is expensive and, in many user cases, subjective assessment is not possible to perform. Different algorithms have been proposed and implemented in this regard. Some of them are in the recommendation list of the ITU Telecommunication Standardization Sector (ITU-T). In this paper, we study the effect of the freezing artefact on user experience and compare the mean opinion score of these videos with the results of two algorithms, the perceptual evaluation of video quality (PEVQ) and temporal quality metric (TQM). Both metrics are part of the ITU-T Recommendation J.247 Annex B and C. PEVQ is a full-reference video quality metric, whereas TQM is a no-reference quality metric. Another contribution of this paper is the study of the impact of different resolutions and frame rates on user experience and how accurately PEVQ and TQM measure varying frame rates.

Best-effort service QoE assessment using visual information based Weber-Fechner law

Quality of experience (QoE) has been one of the most important factors for service providers and operators to develop and deploy services. Previous works have shown that Weber-Fechner law (WFL) can be used to promote the accuracy of best-effort (BE) services’ QoE assessment. In this paper, based on WFL, we take a deep insight into the visual information of BE services and its role in WFL, especially in web browsing services, and propose an enhanced QoE assessment model for BE services by taking the visual information into account. Both images and texts visual information are extracted and combined to get the visual information for web pages. Finally, a subjective experiment is performed to validate performance of proposed QoE assessment model, which shows that it performs better than traditional WFL based model.

Psychophysiology-Based QoE Assessment: A Survey

We present a survey of psychophysiology-based assessment for quality of experience (QoE) in advanced multimedia technologies. We provide a classification of methods relevant to QoE and describe related psychological processes, experimental design considerations, and signal analysis techniques. We summarize multimodal techniques and discuss several important aspects of psychophysiology-based QoE assessment, including the synergies with psychophysical assessment and the need for standardized experimental design. This survey is not considered to be exhaustive but serves as a guideline for those interested to further explore this emerging field of research.

Recommendations for Big Data in Online Video Quality of Experience Assessment

Real-time video application usage is increasing rapidly. Hence, accurate and efficient assessment of video Quality of Experience (QoE) is a crucial concern for end-users and communication service providers. After considering the relevant literature on QoS, QoE and characteristics of video trans-missions, this paper investigates the role of big data in video QoE assessment. The impact of QoS parameters on video QoE are established based on test-bed experiments. Essentially big data is employed as a method to establish a sensible mapping between network QoS parameters and the resulting video QoE. Ultimately, based on the outcome of experiments, recommendations/re- quirements are made for a Big Data-driven QoE model.

Modeling immersive media experiences by sensing impact on subjects

As immersive technologies target to provide higher quality of multimedia experiences, it is important to understand the quality of experience (QoE) perceived by users from various multimedia rendering schemes, in order to design and optimize human-centric immersive multimedia systems. In this study, various QoE-related aspects, such as depth perception, sensation of reality, content preference, and perceived quality are investigated and compared for presentation of 2D and 3D contents. Since the advantages of implicit over explicit QoE assessment have become essential, the way these QoE-related aspects influence brain and periphery is also investigated. In particular, two classification schemes using electroencephalography (EEG) and peripheral signals (electrocardiography and respiration) are carried out, to explore if it is possible to automatically recognize the QoE-related aspects under investigation. In addition, a decision-fusion scheme is applied to EEG and peripheral features, to explore the advantage of integrating information from the two modalities. The results reveal that the highest monomodal average informedness is achieved in the high beta EEG band (0.14 % ± 0.09, p < 0.01), when recognizing sensation of reality. The highest and significantly non-random multimodal average informedness is achieved when high beta EEG band is fused with peripheral features (0.17 % ± 0.1, p < 0.01), for the case of sensation of reality. Finally, a temporal analysis is conducted to explore how the EEG correlates for the case of sensation of reality change over time. The results reveal that the right cortex is more involved when sensation of reality is low, and the left when sensation of reality is high, indicating that approach and withdrawal-related processes occur during sensation of reality.