Objective Video Quality Assessment Metrics Research Articles

Unsupervised deep learning for real-time assessment of video streaming services

Evaluating quality of experience in video streaming services requires a quality metric that works in real time and for a broad range of video types and network conditions. This means that, subjective video quality assessment studies, or complex objective video quality assessment metrics, which would be best suited from the accuracy perspective, cannot be used for this tasks (due to their high requirements in terms of time and complexity, in addition to their lack of scalability). In this paper we propose a light-weight No Reference (NR) method that, by means of unsupervised machine learning techniques and measurements on the client side is able to assess quality in real-time, accurately and in an adaptable and scalable manner. Our method makes use of the excellent density estimation capabilities of the unsupervised deep learning techniques, the restricted Boltzmann machines, and light-weight video features computed just on the impaired video to provide a delta of quality degradation. We have tested our approach in two network impaired video sets, the LIMP and the ReTRiEVED video quality databases, benchmarking the results of our method against the well-known full reference metric VQM. We have obtained levels of accuracy of at least 85% in both datasets using all possible cases.

A Comparative Study of Objective Video Quality Assessment Metrics

A Comparative Study of Objective Video Quality Assessment Metrics

VQAMap: A Novel Mechanism for Mapping Objective Video Quality Metrics to Subjective MOS Scale

The reliable estimation of video quality has become increasingly important with the proliferation of online video services and users becoming more quality aware. A multitude of objective video quality assessment (VQA) metrics with various performance and complexity have been proposed. However, their applicability in real-world scenarios is limited by the lack of clear interpretations of how the metric values reflect the subjective user-perceived video quality. This paper proposes a novel mechanism called VQAMap, that uses data from public VQA databases and enables to automatically create generic rules for mapping the values of objective VQA metrics to the subjective MOS scale (i.e., 1–bad, 2–poor, 3–fair, 4–good, and 5–excellent). An extensive evaluation study of VQAMap was conducted using data from three public VQA databases, considering six objective VQA metrics. The results analysis has shown that VQAMap provides mapping rules with quality estimation accuracy as high as 95%, while the variation in performance being caused by the varying accuracy of the different objective metrics.

Acceptability-Based QoE Models for Mobile Video

Quality of experience (QoE) measures the overall perceived quality of mobile video delivery from subjective user experience and objective system performance. Current QoE prediction models have two main limitations: (1) insufficient consideration of the factors influencing QoE, and (2) limited studies on QoE models for acceptability prediction. In this paper, a set of novel acceptability-based QoE models, denoted as A-QoE, is proposed based on the results of comprehensive user studies on subjective quality acceptance assessments. The models are able to predict users' acceptability and pleasantness in various mobile video usage scenarios. Statistical nonlinear regression analysis has been used to build the models with a group of influencing factors as independent predictors, which include encoding parameters and bitrate, video content characteristics, and mobile device display resolution. The performance of the proposed A-QoE models has been compared with three well-known objective Video Quality Assessment metrics: PSNR, SSIM and VQM. The proposed A-QoE models have high prediction accuracy and usage flexibility. Future user-centred mobile video delivery systems can benefit from applying the proposed QoE-based management to optimize video coding and quality delivery strategies.

Review of objective video quality metrics and performance comparison using different databases

The use of video-based applications has increased in recent years owing to the development of video technology as well as the widespread use of the Internet. Thus the evaluation of perceptual video quality has become very important and numerous video quality assessment (VQA) metrics have been developed over the past years. In this paper, we give a classification and a short review of objective VQA metrics, with a focus on the full reference metrics. With the aim of conducting a reliable test of the VQA metrics performances, we made two databases, each of them including 90 distorted video sequences. We carried out a subjective quality evaluation on these databases and the data were made available to the research community. Furthermore, we compared the performance of nine different, freely available, objective VQA metrics by using three different databases in different resolutions: LIVE Video Quality Database (768×432 resolution) and our two newly created databases for progressively scanned videos, i.e. ETFOS CIF Video Quality (ECVQ) database and ETFOS VGA Video Quality (EVVQ) database. Five different distortion types were used and the total number of 330 video sequences was evaluated. A comparison of metrics was done with respect to accuracy, monotonicity, stability, as well as complexity vs. accuracy criteria. The results show that the resolution, the content of the sequence and the distortion type have a significant influence on the performances of VQA metrics. Metrics that generally achieve a high correlation with subjective results for all databases and all distortion types are MOtion-based Video Integrity Evaluation (MOVIE) and Foveated Mean Squared Error (FMSE), but MOVIE has significantly higher complexity than FMSE. An exception is the distortion caused by IP transmission for which none of the analyzed metrics has shown satisfying accuracy and stability.

Subjective Assessment of BitDetect—A Mechanism for Energy-Aware Multimedia Content Adaptation

As mobile devices are becoming more compact and powerful and as they start to be increasingly used for accessing power-hungry multimedia streaming applications, there is an increasing need for mechanisms to efficiently manage the limited battery power resources. This is especially important as the battery capacity has not kept up with the power requirements of an increasing number of mobile device features and “always on” connected users. Adaptive multimedia-based power-saving mechanisms often decrease the clip bitrate to increase the mobile device battery life, without considering the effect of these degradations on the user-perceived quality. This paper proposes BitDetect, a mechanism that uses objective video quality assessment metrics to detect content-specific video bitrate levels that enable saving battery power while maintaining good user perceived quality. Results from a subjective study indicate that the recommended bitrate offers good user-perceived quality across different multimedia clips. Furthermore, experimental tests indicate that significant battery power can be saved by using the recommended bitrates when streaming multimedia clips to a mobile device.

Atherosclerotic Plaque Ultrasound Video Encoding, Wireless Transmission, and Quality Assessment Using H.264

We propose a unifying framework for efficient encoding, transmission, and quality assessment of atherosclerotic plaque ultrasound video. The approach is based on a spatially varying encoding scheme, where video-slice quantization parameters are varied as a function of diagnostic significance. Video slices are automatically set based on a segmentation algorithm. They are then encoded using a modified version of H.264/AVC flexible macroblock ordering (FMO) technique that allows variable quality slice encoding and redundant slices (RSs) for resilience over error-prone transmission channels. We evaluate our scheme on a representative collection of ten ultrasound videos of the carotid artery for packet loss rates up to 30%. Extensive simulations incorporating three FMO encoding methods, different quantization parameters, and different packet loss scenarios are investigated. Quality assessment is based on a new clinical rating system that provides independent evaluations of the different parts of the video (subjective). We also use objective video-quality assessment metrics and estimate their correlation to the clinical quality assessment of plaque type. We find that some objective quality assessment measures computed over the plaque video slices gave very good correlations to mean opinion scores (MOSs). Here, MOSs were computed using two medical experts. Experimental results show that the proposed method achieves enhanced performance in noisy environments, while at the same time achieving significant bandwidth demands reductions, providing transmission over 3G (and beyond) wireless networks.