Perceptual Video Quality Assessment Research Articles

Perceptual video quality assessment: the journey continues!

Perceptual Video Quality Assessment (VQA) is one of the most fundamental and challenging problems in the field of Video Engineering. Along with video compression, it has become one of two dominant theoretical and algorithmic technologies in television streaming and social media. Over the last 2 decades, the volume of video traffic over the internet has grown exponentially, powered by rapid advancements in cloud services, faster video compression technologies, and increased access to high-speed, low-latency wireless internet connectivity. This has given rise to issues related to delivering extraordinary volumes of picture and video data to an increasingly sophisticated and demanding global audience. Consequently, developing algorithms to measure the quality of pictures and videos as perceived by humans has become increasingly critical since these algorithms can be used to perceptually optimize trade-offs between quality and bandwidth consumption. VQA models have evolved from algorithms developed for generic 2D videos to specialized algorithms explicitly designed for on-demand video streaming, user-generated content (UGC), virtual and augmented reality (VR and AR), cloud gaming, high dynamic range (HDR), and high frame rate (HFR) scenarios. Along the way, we also describe the advancement in algorithm design, beginning with traditional hand-crafted feature-based methods and finishing with current deep-learning models powering accurate VQA algorithms. We also discuss the evolution of Subjective Video Quality databases containing videos and human-annotated quality scores, which are the necessary tools to create, test, compare, and benchmark VQA algorithms. To finish, we discuss emerging trends in VQA algorithm design and general perspectives on the evolution of Video Quality Assessment in the foreseeable future.

Enhancing Perceptual Experience of Video Quality in Drone Communications by Using VPN Bonding

Recently, drones are increasingly being used as innovative IoT sensors. The application contexts of UAV systems where there is often a need to transmit time-critical data between the drone and the Internet are increasingly numerous and varied. Keeping this in mind and assuming that most professional UAVs are equipped with multiinterface routers and use virtual private networks (VPNs) to ensure reliable, robust, and secure transmission to the operation room, this letter presents a technique to enhance real-time and time-critical communications such as real-time video streaming. An embedded system, Raspberry-based gateway prototype, was used to create a testbed using a professional drone that captured and transmitted real-time video during a 30-min flight over a rural area characterized by poor mobile radio coverage. A perceptual video quality assessment tool, video multimethod assessment fusion (VMAF), has been employed to evaluate the improvement in terms of video quality perceived by the operator.

CONVIQT: Contrastive Video Quality Estimator.

Perceptual video quality assessment (VQA) is an integral component of many streaming and video sharing platforms. Here we consider the problem of learning perceptually relevant video quality representations in a self-supervised manner. Distortion type identification and degradation level determination is employed as an auxiliary task to train a deep learning model containing a deep Convolutional Neural Network (CNN) that extracts spatial features, as well as a recurrent unit that captures temporal information. The model is trained using a contrastive loss and we therefore refer to this training framework and resulting model as CONtrastive VIdeo Quality EstimaTor (CONVIQT). During testing, the weights of the trained model are frozen, and a linear regressor maps the learned features to quality scores in a no-reference (NR) setting. We conduct comprehensive evaluations of the proposed model against leading algorithms on multiple VQA databases containing wide ranges of spatial and temporal distortions. We analyze the correlations between model predictions and ground-truth quality ratings, and show that CONVIQT achieves competitive performance when compared to state-of-the-art NR-VQA models, even though it is not trained on those databases. Our ablation experiments demonstrate that the learned representations are highly robust and generalize well across synthetic and realistic distortions. Our results indicate that compelling representations with perceptual bearing can be obtained using self-supervised learning.

A content-oriented no-reference perceptual video quality assessment method for computer graphics animation videos

In this paper, we propose a content-oriented no-reference (NR) perceptual video quality assessment (VQA) method for computer graphics (CG) animation videos. First, we extract features in terms of spatiotemporal information and its visual perception from the videos as inputs of our proposed artificial neural network-based VQA model. Second, to facilitate the video quality evaluation, we apply a convolutional neural network (CNN) in the VQA model to generate weight factors for the input features adaptively according to the different types of CG content in videos. Third, we build a subjective CG video quality database for validation of VQA metrics. Experiments demonstrated that our method achieved superior performance in terms of evaluating the quality of CG animation videos. Both the code and proposed database are publicly available at https://github.com/WeizhiXian/CGVQA. The corresponding newly established database is available at https:// pan.baidu.com/s/1_P2ZNrLzJwZfG6xa6tKnDQ (password: cgvq).

Improved Temporal Pooling for Perceptual Video Quality Assessment Using VMAF

The Video Multimethod Assessment Fusion (VMAF) method, proposed by Netflix, offers an automated estimation of perceptual video quality for each frame of a video sequence. Then, the arithmetic mean of the per-frame quality measurements is taken by default, in order to obtain an estimate of the overall Quality of Experience (QoE) of the video sequence. In this paper, we validate the hypothesis that the arithmetic mean conceals the bad quality frames, leading to an overestimation of the provided quality. We also show that the Minkowski mean (appropriately parametrized) approximates well the subjectively measured QoE, providing superior Spearman Rank Correlation Coefficient (SRCC), Pearson Correlation Coefficient (PCC), and Root-Mean-Square-Error (RMSE) scores.

Motion silencing of flicker distortions on naturalistic videos

We study the influence of motion on the visibility of flicker distortions in naturalistic videos. A series of human subjective studies were executed to understand how motion silences the visibility of flicker distortions as a function of object motion, flicker frequency, and video quality. We found that flicker visibility is strongly reduced when the speed of coherent motion is large, and the effect is pronounced when video quality is poor. Based on this finding, we propose a model of flicker visibility on naturalistic videos. The target-related activation levels in the excitatory layer of neurons were estimated for a displayed video using a spatiotemporal backward masking model, and then the flicker visibility is predicted based on a learned model of neural flicker adaptation processes. Experimental results show that the prediction of flicker visibility using the proposed model correlates well with human perception of flicker distortions. We believe that sufficiently fast and coherent motion silences the perception of flicker distortions on naturalistic videos in agreement with a recently observed “motion silencing” effect on synthetic stimuli. We envision that the proposed model could be applied to develop perceptual video quality assessment algorithms that can predict “silenced” temporal distortions and account for them when computing quality judgments.

Perceptual video quality assessment in H.264 video coding standard using objective modeling.

Since usage of digital video is wide spread nowadays, quality considerations have become essential, and industry demand for video quality measurement is rising. This proposal provides a method of perceptual quality assessment in H.264 standard encoder using objective modeling. For this purpose, quality impairments are calculated and a model is developed to compute the perceptual video quality metric based on no reference method. Because of the shuttle difference between the original video and the encoded video the quality of the encoded picture gets degraded, this quality difference is introduced by the encoding process like Intra and Inter prediction. The proposed model takes into account of the artifacts introduced by these spatial and temporal activities in the hybrid block based coding methods and an objective modeling of these artifacts into subjective quality estimation is proposed. The proposed model calculates the objective quality metric using subjective impairments; blockiness, blur and jerkiness compared to the existing bitrate only calculation defined in the ITU G 1070 model. The accuracy of the proposed perceptual video quality metrics is compared against popular full reference objective methods as defined by VQEG.

Issues in vision modeling for perceptual video quality assessment

Lossy compression algorithms used in digital video systems produce artifacts whose visibility strongly depends on the actual image content. Simple error measures such as RMSE or PSNR, albeit popular, ignore this important fact and are only a mediocre predictor of perceived quality. Many applications require more reliable assessment methods. This paper discusses issues in vision modeling for perceptual video quality assessment (PVQA). Its purpose is not to describe a particular model or system, but rather to summarize and to provide pointers to up-to-date knowledge of important characteristics of the human visual system, to explain how these characteristics may be incorporated in vision models for PVQA, to give a brief overview of the state-of-the-art and current efforts in this field, and to outline directions for future research.

Subsidiary influence and autonomy in international firms

We study the influence of motion on the visibility of flicker distortions in naturalistic videos. A series of human subjective studies were executed to understand how motion silences the visibility of flicker distortions as a function of object motion, flicker frequency, and video quality. We found that flicker visibility is strongly reduced when the speed of coherent motion is large, and the effect is pronounced when video quality is poor. Based on this finding, we propose a model of flicker visibility on naturalistic videos. The target-related activation levels in the excitatory layer of neurons were estimated for a displayed video using a spatiotemporal backward masking model, and then the flicker visibility is predicted based on a learned model of neural flicker adaptation processes. Experimental results show that the prediction of flicker visibility using the proposed model correlates well with human perception of flicker distortions. We believe that sufficiently fast and coherent motion silences the perception of flicker distortions on naturalistic videos in agreement with a recently observed “motion silencing” effect on synthetic stimuli. We envision that the proposed model could be applied to develop perceptual video quality assessment algorithms that can predict “silenced” temporal distortions and account for them when computing quality judgments.

Contrast adaptation and contrast masking in human vision

After a preliminary study of visual evoked potentials (VEPS) to a test grating seen in the presence of masks at different orientations, psychophysical data are presented showing the effects of adaptation and of masking on thresholds for detecting the same test grating. The test is a vertical grating of spatial frequency 2 cycles per degree; adapting and masking gratings differ from the test either in orientation or in spatial frequency. The effects of adaptation and masking are explained by a single mechanism model that assumes: (i) adaptation and masking both alter the contrast response (or transducer) function of the mechanism that detects the test; (ii) masks, but not adaptors, stimulate the mechanism that detects the test; and (iii) a test is detectable when it raises response level by a constant amount. The model incorporates two distinct tuning functions, a broad adaptive contrast function and a narrow effective contrast function. It accounts adequately for all the data, including the location and size of the facilitative dip found in some masking functions, the constant slopes of the threshold elevation segments of adaptation functions and the varying slopes of masking functions. It also predicts the sometimes surprising joint effects of adaptation followed by masking and of two masks operating simultaneously.