Video Quality Metrics Research Articles

NR-Bitstream video quality metrics for SSIM using encoding decisions in AVC and HEVC coded videos

We propose a no-reference compressed video quality model to predict the full-reference SSIM metrics for AVC (Advanced Video Coding, H.264) and HEVC (High Efficiency Video Coding) videos. The model we use is support vector regression (SVR) model. We use only encoding decisions made during motion estimation to perform the prediction, and do not need the information from pixel domain. We show that the Block-Partition-related features have great importance in SSIM prediction, especially for HEVC videos, due to its partition decisions being more complex than those of AVC. The proposed SVR model trained by data of two different encoding configurations can predict SSIM well for AVC videos (0.78 correlation) and for HEVC videos (0.88 correlation). The proposed models are also compared with a state-of-the-art no-reference-bitstream-pixel SSIM prediction model. We show that the proposed methods provide higher prediction correlation (as high as 13.13% improvement in correlation) with much lower complexity.

Design of Objective Video Quality Metrics Using Spatial and Temporal Informations

Digital videos are subject to several types of distortions, which may occur during the processes of acquisition, processing, compression, storage and transmission of videos, resulting in loss of visual quality. Video quality assessment is important to set system parameters and propose the reduction of degradation, to improve the systems quality. The most appropriate way to assess the visual quality of a video sequence is subjective assessment. This study proposes new metrics for assessment of videos. The proposed metrics that uses spatial and temporal information to evaluate videos are more closely to the subjective assessment. For the evaluation of videos the BD-TPW-SSIM and TP-VQI metric obtained the best result for wireless degradation. For MPEG-2 encoded videos, T-PW-SSIM e T-MOVIE was the best, and IP systems the metric with the best result was the TPW-SSIM. Finally, the metric T-VimSSIM was the best for H.264 encoding.

Estimation of Degraded Video Quality of Mobile H.264/AVC Video Streaming

The problem of estimation of subjective video quality of distorted video sequences for mobile video streaming is addressed. Widely spreading mobile communication systems and increasing data transmission rates expand variety of multimedia services. One of such services is video straming. So it is important to asses quality of this service. Consumers of video streaming are humans, and quality assessment must account human perception characteristics. Most of the well-known video quality estimation methods, such as Peak Signal to Noise Ratio (PSNR), Structural Similarity (SSIM), Video Quality Metric (VQM), etc. are based on the presence of full or reduced reference video sequence. However, in order to estimate user experienced video quality, methods with no reference must be employed. Such existing methods as quality metrics usually use bit-error rate that has low correlation with by human perceived video quality. More advanced methods usually require too much processing power that cannot be obtained in handled mobile devices. It is shown that video quality estimation method with no reference and low computation complexity could be developed basing on H.264/AVC video stream structure. Such method is suitable for implementation in mobile devices with low processing power. Ill. 4, bibl. 11 (in English; summaries in English, Russian and Lithuanian).

A Novel No-Reference Video Quality Metric for Evaluating Temporal Jerkiness due to Frame Freezing

In this work, we propose a novel no-reference (NR) video quality metric that evaluates the impact of frame freezing due to either packet loss or late arrival. Our metric uses a trained neural network acting on features that are chosen to capture the impact of frame freezing on the perceived quality. The considered features include the number of freezes, freeze duration statistics, inter-freeze distance statistics, frame difference before and after the freeze, normal frame difference, and the ratio of them. We use the neural network to find the mapping between features and subjective test scores. We optimize the network structure and the feature selection through a cross validation procedure, using training samples extracted from both VQEG and LIVE video databases. The resulting feature set and network structure yields accurate quality prediction for both the training data containing 54 test videos and a separate testing dataset including 14 videos, with Pearson Correlation Coefficients greater than 0.9 and 0.8 for the training set and the testing set, respectively. Our proposed metric has low complexity and could be utilized in a system with realtime processing constraint.

Packet losses and objective video quality metrics in H.264 video streaming

Video quality is a key factor in modern video streaming systems. Video compression artifacts affect it but also delay, jitter and packet loss may compromise it. Objective metrics have been proposed to emulate the human visual system: several experimental works have evaluated their adherence to opinions expressed by real users. Instead of dealing with subjective tests, we focused on the effects of network packet losses on objective metrics: since quality metrics are generally computationally intensive, a convenient approach could consist in inferring information about transmission quality from network impairment statistics. We did experimental tests on two computer-animated videos, whose high color contrasts allow a fair comparison between content dependent and content independent metrics. We studied the encoding parameters that minimize/maximize the values of some metrics (PSNR, BI-PSNR, SSIM, 3SSIM, MSSSIM, VQM) for several packet loss percentages. We analyzed also the Empirical Cumulative Density Function of the degradations of quality metrics.

Full-reference audio-visual video quality metric

The goal of this work is to present a full-reference (FR) audio-visual quality metric. We performed three psychophysical experiments in order to obtain a better understanding of how audio and video components interact with each other and how these interactions affect the overall audio-visual quality. In experiment I, subjects evaluated the quality of videos (without any audio) compressed at different video bitrates. In experiment II, subjects evaluated the quality of audio (without any video) compressed at different audio bitrates. In experiment III, subjects evaluated the quality of videos (audio-visual signals), which had their audio and video components compressed at different bitrates. Based on the data gathered from these experiments, we obtain a set of subjective models for audio-visual quality. Inspired by these subjective models, we propose a set of FR audio-visual quality metrics composed of a combination of a video quality metric and an audio quality metric. The proposed metrics have good performance and present better results when compared to simple FR video quality metrics.

Video quality assessment using visual attention computational models

A recent development in the area of image and video quality consists of trying to incorporate aspects of visual attention in the design of visual quality metrics, mostly using the assumption that visual distortions appearing in less salient areas might be less visible and, therefore, less annoying. This research area is still in its infancy and results obtained by different groups are not yet conclusive. Among the works that have reported some improvements, most use subjective saliency maps, i.e., saliency maps generated from eye-tracking data obtained experimentally. Other works address the image quality problem, not focusing on how to incorporate visual attention into video signals. We investigate the benefits of incorporating bottom-up video saliency maps (obtained using Itti's computational model) into video quality metrics. In particular, we com- pare the performance of four full-reference video quality metrics with their modified versions, which had saliency maps incorporated into the algorithm. Results show that the addition of video saliency maps improve the per- formance of most quality metrics tested, but the highest gains were obtained for the metrics that only took into consideration spatial degradations. © 2014 SPIE and IS&T (DOI: 10.1117/1.JEI.23.6.061107)

Video quality assessment in video streaming services considering user preference for video content

In video streaming service, the user's Quality of Experience (QoE) is not only related to video signal quality received at consumer's devices, the users’ subjectivity must also be considered. In this context, a video quality assessment method that takes into account the user's preference for video content is proposed in this research. In order to perform this task, the users' profiles that include their preferences were stored in a video server. Then, subjective tests of video quality assessment were conducted, in which evaluators had different video content preferences. Results show that the evaluators' QoE is highly correlated with the user's preference for video content type. Based on these experimental results, a function named Preference Factor (PF) is defined and used to adjust the quality index values obtained by an objective video quality metric running in the end user's device. The PF function also depends on video content type and quality index score. Using the PF function, the enhanced Video streaming Quality Metric (e-VsQM) is proposed and the results of its performance evaluation demonstrate that PF improves an objective video quality metric. Furthermore, e-VsQM has low complexity and can be utilized in different video services. Thus, an application scenario is presented, in which the proposed video quality metric is implemented.

A study on quality assessment for medical ultrasound video compressed via HEVC.

The quality of experience and quality of service provided in the healthcare sector are critical in evaluating the reliable delivery of the healthcare services provided. Medical images and videos play a major role in modern e-health services and have become an integral part of medical data communication systems. The quality evaluation of medical images and videos is an essential process, and one of the ways of addressing it is via the use of quality metrics. In this paper, we evaluate the performance of seven state-of-the-art video quality metrics with respect to compressed medical ultrasound video sequences. We study the performance of each video quality metric in representing the diagnostic quality of the video, by evaluating the correlation of each metric with the subjective opinions of medical experts. The results indicate that the visual information fidelity, structural similarity index, and universal quality index metrics show good correlation with the subjective scores provided by medical experts. The tests also investigate the performance of the emerging video compression standard, high-efficiency video coding-HEVC, for medical ultrasound video compression. The results show that, using HEVC with the considered ultrasound video sequences, a diagnostically reliable compressed ultrasound video can be obtained for compression with values of the quantization parameter up to 35.

Optimal Rate Control in H.264 Video Coding Based on Video Quality Metric

The aim of this research is to find a method for providing better visual quality across the complete video sequence in H.264 video coding standard. H.264 video coding standard with its significantly improved coding efficiency finds important applications in various digital video streaming, storage and broadcast. To achieve comparable quality across the complete video sequence with the constrains on bandwidth availability and buffer fullness, it is important to allocate more bits to frames with high complexity or a scene change and fewer bits to other less complex frames. A frame layer bit allocation scheme is proposed based on the perceptual quality metric as indicator of the frame complexity. The proposed model computes the Quality Index ratio (QIr) of the predicted quality index of the current frame to the average quality index of all the previous frames in the group of pictures which is used for bit allocation to the current frame along with bits computed based on buffer availability. The standard deviation of the perceptual quality indicator MOS computed for the proposed model is significantly less which means the quality of the video sequence is identical throughout the full video sequence. Thus the experiment results shows that the proposed model effectively handles the scene changes and scenes with high motion for better visual quality.

Adaptation engine for a streaming service based on MPEG-DASH

HTTP Video streaming has become a strong candidate for video transmission on the Internet thanks to the abundance of web infrastructure. With the recent standardization of the new MPEG Dynamic Adaptive Streaming over HTTP (DASH), the flexibility and implantation of adaptive video systems has increased due to the fact that DASH can operate on a conventional web infrastructure. In this paper we propose an estimation and adaptation system for DASH. The proposed adaptive algorithm is based on client buffer threshold and smooth throughput measures (based on the throughput of previous segments). Before DASH, the standard of Scalable Video Coding (SVC) also arose from the idea of adaptation. Both systems (adaptive system based on SVC and the proposed system for DASH) are compared in terms of Video Quality (VQ) metrics. The results show that the proposed system reacts properly to changes in the network capacity, while maintaining a minimum level of segments in the buffer. The user-perceived quality is better than in the SVC-based adaptive system although the generated traffic is higher.

Perceptual video quality assessment in H.264 video coding standard using objective modeling.

Since usage of digital video is wide spread nowadays, quality considerations have become essential, and industry demand for video quality measurement is rising. This proposal provides a method of perceptual quality assessment in H.264 standard encoder using objective modeling. For this purpose, quality impairments are calculated and a model is developed to compute the perceptual video quality metric based on no reference method. Because of the shuttle difference between the original video and the encoded video the quality of the encoded picture gets degraded, this quality difference is introduced by the encoding process like Intra and Inter prediction. The proposed model takes into account of the artifacts introduced by these spatial and temporal activities in the hybrid block based coding methods and an objective modeling of these artifacts into subjective quality estimation is proposed. The proposed model calculates the objective quality metric using subjective impairments; blockiness, blur and jerkiness compared to the existing bitrate only calculation defined in the ITU G 1070 model. The accuracy of the proposed perceptual video quality metrics is compared against popular full reference objective methods as defined by VQEG.

No reference quality assessment for MPEG video delivery over IP

Video delivering over Internet protocol (IP)-based communication networks is widely used in the actual information sharing scenario. As well known, the best-effort Internet architecture cannot guarantee an errorless data delivering. In this paper, an objective no-reference video quality metric for assessing the impact of the degradations introduced by video transmission over heterogeneous IP networks is presented. The proposed approach is based on the analysis of the inter-frame correlation measured at the output of the rendering application. It does not require any information on errors, delays, and latencies affecting the links and on the countermeasures introduced by decoders in order to face the potential quality loss. Experimental results show the effectiveness of the proposed algorithm in approximating the assessments obtained by using human visual system (HVS)-inspired full reference metrics.

Full-reference video quality metric assisted the development of no-reference bitstream video quality metrics for real-time network monitoring

High-quality video is being increasingly delivered over Internet Protocol networks, which means that network operators and service providers need methods to measure the quality of experience (QoE) of the video services. In this paper, we propose a method to speed up the development of no-reference bitstream objective metrics for estimating QoE. This method uses full-reference objective metrics, which makes the process significantly faster and more convenient than using subjective tests. In this process, we have evaluated six publicly available full-reference objective metrics in three different databases, the EPFL-PoliMI database, the HDTV database, and the Live Video Wireless database, all containing transmission distortions in H.264 coded video. The objective metrics could be used to speed up the development process of no-reference real-time video QoE monitoring methods that are receiving great interest from the research community. We show statistically that the full-reference metric Video Quality Metric (VQM) performs best considering all the databases. In the EPFL-PoliMI database, SPATIAL MOVIE performed best and TEMPORAL MOVIE performed worst. When transmission distortions are evaluated, using the compressed video as the reference provides greater accuracy than using the uncompressed original video as the reference, at least for the studied metrics. Further, we use VQM to train a lightweight no-reference bitstream model, which uses the packet loss rate and the interval between instantaneous decoder refresh frames, both easily accessible in a video quality monitoring system.

New Stereo Video Quality Assessment Metric for Three Dimensional Video Systems

Since traditional two dimensional video quality metrics are not suitable for evaluating stereo video, a new stereo video quality metric based on support vector regression (SVR) for three dimensional video systems is proposed in this paper. Firstly, three factors affecting the quality of stereo video are analyzed, namely left-view video quality, right-view video quality and difference between two views. Then, each of these factors is calculated respectively with measuring the singular vectors’ similarity between original image blocks and distorted image blocks and with temporal weighting using fluctuations of video frames’ spatial quality. Then, SVR is used to learn the relationships between the factors and the overall perceived quality of stereo video. Finally, the relationship is applied to predict the objective quality of stereo video. By applying the proposed metric to stereo video database, Person’s linear correlation coefficient is higher than 0.82 for all types of distortions and 0.81 for individual distortion, which outperforms the other existing metrics. Experimental results also show that, the proposed metric can achieve stable performance for all kinds of distortions and predict human perception well.

A Framework to Measure and Estimate Video Quality in SVC Real-Time Adaptive Systems

Effectively adapting the content to network conditions in real-time is an important matter in best-effort networks like the Internet. Scalable Video Coding (SVC) is an interesting alternative to implement such systems. However, some problems of the performance evaluation of SVC based adaptive systems have not been solved. The authors review the main efforts directed to measure video quality on SVC related systems and discuss the limitations of each one. This paper elaborates a framework to measure video quality metrics in real adaptive SVC based streams. An estimation method for full reference video quality metrics is proposed. This method reduces reference information required and it is able to provide real-time accurate results simply using metadata regarding the video quality of the reference layers. The video quality of several streams that have been generated using a real-time adaptive system is first measured with the elaborated framework and then estimated with the proposed method.

On the performance of video quality assessment metrics under different compression and packet loss scenarios.

When comparing the performance of video coding approaches, evaluating different commercial video encoders, or measuring the perceived video quality in a wireless environment, Rate/distortion analysis is commonly used, where distortion is usually measured in terms of PSNR values. However, PSNR does not always capture the distortion perceived by a human being. As a consequence, significant efforts have focused on defining an objective video quality metric that is able to assess quality in the same way as a human does. We perform a study of some available objective quality assessment metrics in order to evaluate their behavior in two different scenarios. First, we deal with video sequences compressed by different encoders at different bitrates in order to properly measure the video quality degradation associated with the encoding system. In addition, we evaluate the behavior of the quality metrics when measuring video distortions produced by packet losses in mobile ad hoc network scenarios with variable degrees of network congestion and node mobility. Our purpose is to determine if the analyzed metrics can replace the PSNR while comparing, designing, and evaluating video codec proposals, and, in particular, under video delivery scenarios characterized by bursty and frequent packet losses, such as wireless multihop environments.

MDVQM: A novel multidimensional no-reference video quality metric for video transcoding

In this paper, we study the impact of quantization, frame dropping and spatial down-sampling on the perceived quality of compressed video streams. Based on the analysis of quality ratings obtained from extensive subjective tests, we propose a no-reference metric (named MDVQM) for video quality estimation in the presence of both spatial and temporal quality impairments. The proposed metric is based on the per-pixel bitrate of the encoded stream and selected spatial and temporal activity measures extracted from the video content. All the values required to compute the proposed video quality metric can be obtained without using the original reference video which makes the metric for instance useful for making transcoding decisions in a wireless video transmission scenario. Different from comparable metrics in the literature, we have also considered the case when both frame rate and frame size are changed simultaneously. The validation results show that the proposed metric provides more accurate estimation of the video quality than the state of the art metrics.

Factors affecting enhanced video quality preferences.

The development of video quality metrics requires methods for measuring perceived video quality. Most of these metrics are designed and tested using databases of images degraded by compression and scored using opinion ratings. We studied video quality preferences for enhanced images of normally-sighted participants using the method of paired comparisons with a thorough statistical analysis. Participants (n=40) made pair-wise comparisons of high definition video clips enhanced at four different levels using a commercially available enhancement device. Perceptual scales were computed with binary logistic regression to estimate preferences for each level and to provide statistical inference of the differences among levels and the impact of other variables. While moderate preference for enhanced videos was found, two unexpected effects were also uncovered: 1) participants could be broadly classified into two groups: a) those who preferred enhancement ("Sharp") and b) those who disliked enhancement ("Smooth") and 2) enhancement preferences depended on video content, particularly for human faces to be enhanced less. The results suggest that algorithms to evaluate image quality (at least for enhancement) may need to be adjusted or applied differentially based on video content and viewer preferences. The possible impact of similar effects on image quality of compressed video needs to be evaluated.

Despeckle Filtering Toolbox for Medical Ultrasound Video

Ultrasound medical video has the potential in differentiating between normal and abnormal tissue and structure. Ultrasound imaging is used in border identification and texture characterisation of the atherosclerotic carotid plaque in the common carotid artery (CCA), the identification and measurement of the intima-media thickness (IMT) and the lumen diameter that are very important in the assessment of cardiovascular disease. However, visual perception is reduced by speckle noise affecting the quality of ultrasound B-mode imaging. Noise reduction is therefore essential for increasing the visual quality or as a pre-processing step for further automated analysis, such as the video segmentation of the IMT and the atherosclerotic carotid plaque in ultrasound video sequences. In order to facilitate this analysis, the authors have developed a video analysis software toolbox based on MATLAB® that uses video despeckling, texture analysis and image quality evaluation techniques to automate the pre-processing and complement the disease evaluation in ultrasound CCA videos. The proposed software, which is based on a graphical user interface (GUI), incorporates video normalisation, 4 different despeckle filtering techniques (DsFlsmv, DsFhmedian, DsFkuwahara and DsFsrad), 65 texture features, 11 quantitative video quality metrics and objective video quality evaluation. The software was validated on 10 ultrasound videos of the CCA, by comparing its results with quantitative visual analysis performed by two medical experts. It was shown that the filters DsFlsmv, and DsFhmedian improved video quality perception (based on the expert’s assessment and the video quality metrics). It is anticipated that the system could help the physician in the assessment of cardiovascular video analysis. However, exhaustive evaluation of the despeckle filtering toolbox has to be carried out by more experts on more videos.