Objective Video Quality Assessment Models Research Articles

Practical Evaluation of VMAF Perceptual Video Quality for WebRTC Applications

WebRTC is the umbrella term for several emergent technologies aimed to exchange real-time media in the Web. Like other media-related services, the perceived quality of WebRTC communication can be measured using Quality of Experience (QoE) indicators. QoE assessment methods can be classified as subjective (users’ evaluation scores) or objective (models computed as a function of different parameters). In this paper, we focus on VMAF (Video Multi-method Assessment Fusion), which is an emergent full-reference objective video quality assessment model developed by Netflix. VMAF is typically used to assess video streaming services. This paper evaluates the use of VMAF in a different type of application: WebRTC. To that aim, we present a practical use case built on the top of well-known open source technologies, such as JUnit, Selenium, Docker, and FFmpeg. In addition to VMAF, we also calculate other objective QoE video metrics such as Visual Information Fidelity in the pixel domain (VIFp), Structural Similarity (SSIM), or Peak Signal-to-Noise Ratio (PSNR) applied to a WebRTC communication in different network conditions in terms of packet loss. Finally, we compare these objective results with a subjective evaluation using a Mean Opinion Score (MOS) scale to the same WebRTC streams. As a result, we found a strong correlation of the subjective video quality perceived in WebRTC video calls with the objective results computed with VMAF and VIFp in comparison with SSIM and PSNR and their variants.

Modelling user quality of experience from objective and subjective data sets using fuzzy logic

One of the paramount research questions in the scientific community today is how to remotely assess user quality of experience (QoE) for a specific service. To this end, various user QoE assessment models have been developed; however, they are mostly based on the data gathered from controlled environment experimentation. The aim of this research was to model user QoE for the User Datagram Protocol-based video streaming service from the results of uncontrolled subjective tests. Specifically, using fuzzy logic, we have correlated the values of three objective network parameters (the packet loss rate and the number and duration of packet loss occurrences in one streaming session) with test subjects’ subjective perception about perceived quality distortions. The dependencies between different values of the parameters and the subjects’ perception of video quality were used to develop a no reference objective video quality assessment model for assessing user QoE. The key distinguishing feature of the developed model lays in the process of subjective evaluation, which was conducted with a panel of 602 test subjects who evaluated the quality of 1-h video in home environments. For this purpose, 72 different test sequences were prepared for rating. We showed that a strong positive linear relationship exists between the assessed QoE of the model and the Mean Opinion Scores of the subjects (a Pearson correlation coefficient equal to 0.8841).

Hierarchical content importance-based video quality assessment for HEVC encoded videos transmitted over LTE networks

To improve the accuracy of assessment, many previous works take into account the video content. However, these previous works just only consider the video content, but do not consider the location and importance of the degraded content. Thus, this paper takes into account not only the video content, but also the location and importance of the degraded content, and proposes a hierarchical content importance-based video quality assessment. Firstly, we propose to use the hierarchical content importance-based frame degradation rate (HFDR) metric to quantify the importance of degraded content hierarchically. Secondly, we propose to use the intra random access point (IRAP) loss rate (ILR) metric to quantify the impact of IRAP. Finally, the proposed HFDR metric and ILR metric are subsequently used to develop an objective video quality assessment model. The experimental results show that the predicted mean opinion score (MOS) of the proposed method highly correlates with the actual MOS.

In-service Real-time and Continuous Objective Video Quality Assessment for DTV Broadcasting

This article presents a simple and reasonable in-service, real-time, and continuous single-ended objective video quality assessment model for DTV broadcasting using a multiburst signal at the bottom of the transient effect area, similar to Vertical Interval Test Signals. The issue of in-service video-quality monitoring in DTV broadcasting is addressed, and an effective method of quality monitoring is presented. The proposed method is also implemented and tested in a range of situations using a simulated HDTV broadcasting network.