Reduced-reference Research Articles

Bitstream-based Perceptual Quality Assessment of Compressed 3D Point Clouds.

With the increasing demand of compressing and streaming 3D point clouds under constrained bandwidth, it has become ever more important to accurately and efficiently determine the quality of compressed point clouds, so as to assess and optimize the quality-of-experience (QoE) of end users. Here we make one of the first attempts developing a bitstream-based no-reference (NR) model for perceptual quality assessment of point clouds without resorting to full decoding of the compressed data stream. Specifically, we first establish a relationship between texture complexity and the bitrate and texture quantization parameters based on an empirical rate-distortion model. We then construct a texture distortion assessment model upon texture complexity and quantization parameters. By combining this texture distortion model with a geometric distortion model derived from Trisoup geometry encoding parameters, we obtain an overall bitstream-based NR point cloud quality model named streamPCQ. Experimental results show that the proposed streamPCQ model demonstrates highly competitive performance when compared with existing classic full-reference (FR) and reduced-reference (RR) point cloud quality assessment methods with a fraction of computational cost.

Degraded Reference Image Quality Assessment.

In practical media distribution systems, visual content usually undergoes multiple stages of quality degradation along the delivery chain, but the pristine source content is rarely available at most quality monitoring points along the chain to serve as a reference for quality assessment. As a result, full-reference (FR) and reduced-reference (RR) image quality assessment (IQA) methods are generally infeasible. Although no-reference (NR) methods are readily applicable, their performance is often not reliable. On the other hand, intermediate references of degraded quality are often available, e.g., at the input of video transcoders, but how to make the best use of them in proper ways has not been deeply investigated. Here we make one of the first attempts to establish a new paradigm named degraded-reference IQA (DR IQA). Specifically, by using a two-stage distortion pipeline we lay out the architectures of DR IQA and introduce a 6-bit code to denote the choices of configurations. We construct the first large-scale databases dedicated to DR IQA and will make them publicly available. We make novel observations on distortion behavior in multi-stage distortion pipelines by comprehensively analyzing five multiple distortion combinations. Based on these observations, we develop novel DR IQA models and make extensive comparisons with a series of baseline models derived from top-performing FR and NR models. The results suggest that DR IQA may offer significant performance improvement in multiple distortion environments, thereby establishing DR IQA as a valid IQA paradigm that is worth further exploration.

Perceptual Hashing With Complementary Color Wavelet Transform and Compressed Sensing for Reduced-Reference Image Quality Assessment

Image quality assessment (IQA) is an important task of image processing and has diverse applications, such as image super-resolution reconstruction, image transmission and monitoring systems. This paper proposes a perceptual hashing algorithm with complementary color wavelet transform (CCWT) and compressed sensing (CS) for reduced-reference (RR) IQA. The CCWT is exploited to decompose input color image into different sub-bands. Since the calculation of CCWT uses all color channels without discarding any information, the distortions introduced by digital operations on color channels are preserved in the CCWT sub-bands. The block-based CS is used to extract features from the CCWT sub-bands. As the Euclidean distance between the block-based CS features is slightly influenced by content-preserving operations, perceptual features constructed by Euclidean distances are robust, discriminative and compact. Hash sequence is finally determined by quantifying the perceptual features. Effectiveness of the proposed hashing is verified by various experiments on four open image databases. Experimental results demonstrate that the proposed hashing is superior to some state-of-the-art algorithms in terms of classification and RR IQA application.

Comparative Perceptual Assessment of Visual Signals Using Free Energy Features

In this paper, we put forward the concept of comparative perceptual quality assessment (C-PQA), which refers to the judgment of relative qualities of two visual signals of the same content, but subject to different types and levels of distortions. While it is straightforward for human observers to fulfill the CPQA task in daily lives, it remains a difficult challenge for the current research of perceptual quality assessment (PQA). Among the existing PQA algorithms, the full-reference (FR) and reducedreference (RR) methods both need prior knowledge of the original images while the no-reference (NR) algorithms usually work with a single input image. C-PQA is inherently different from those existing methods in that it takes an image pair as input and predicts their relative quality without using any knowledge about the original image. In this paper, we propose a brain theory inspired approach to C-PQA that emulates the process of comparing the relative quality of two visual stimuli as performed by the human visual system (HVS) within the framework of free energy minimization. The brain's internal generative models initialized on the inputs are then used to explain both images. During the internal generative modeling, a group of features are extracted and then integrated to determine the relative quality of two images. We designed a dedicated image database to test the proposed C-PQA algorithm. Experimental results show that the proposed method achieves up to 98% prediction accuracy in line with the subjective ratings, outperforming many state of the art PQA algorithms.

Blind Stereo Image Quality Assessment Based on Binocular Visual Characteristics and Depth Perception

The quality prediction of stereo images has great challenges without reference images. In this paper, we propose a novel no-reference stereo image quality assessment (NR-SIQA) model based on binocular visual characteristics and depth perception, which can effectively evaluate the quality of symmetric distortion and asymmetric distortion images. To be specific, we discriminate the different binocular behaviors by analyzing binocular visual characteristics, and construct the corresponding cyclopean view instead of single cyclopean view to simulate different binocular behaviors. Then, we extract monocular and binocular visual features from the left view, the right view and the synthetic cyclopean view. Furthermore, in order to evaluate the depth quality of the stereo image accurately, we extract the depth perception features from the weighted disparity map and the longitudinal correlation coefficient map. Finally, we construct the mapping relationship model from quality perception feature domain to quality score domain by training an adaptive enhancement algorithm based on support vector regression (SVR). We evaluate the performance of the proposed algorithm on four stereo image databases. The experimental results show that compared with the state-of-the-art full reference(FR), reduced reference(RR) and NR-SIQA algorithms, the proposed algorithm achieves highly competitive performance for both symmetric and asymmetric distortions.

A comparative analysis of non­reference assessment methods

A comparative analysis of functions to assess image quality in the absence of a sample: no-reference (NR) measure or NR-type methods. The availability of NR-methods is very important for assessing the quality of streaming video such as television, game streaming, online conferences, web-chatting, etc. (because on the side of the recipient of the video there is no standard for quality comparison) and assessing the results of transformations aimed at improving video, and choosing the parameters of these transformations (brightness change, semitone and others). The human visual system (HVS) is able to visually assessing video quality, but If required to visually assess the quality of dozens or hundreds of videos or ranking them by quality level it will be needed a huge amount of time. Six types of experiments were performed to analyze the correlation of calculated quantitative estimates with visual assessments of the quality of the tested video files. Three of them are fundamentally new: comparing video after gamma correction and changing the contrast with different parameters, as well as blurring, which may be the result of defocusing the camcorder. A hybrid method (or reduced-reference (RR) measure) and a full-reference (FR) measure or FR-type method were also added for comparison. It has been experimentally shown that none of the studied non-reference methods of image quality assessment is universal, and the calculated assessment cannot be converted into a quality scale without taking into account the factors influencing the distortion of image quality. Moreover, all NR-type methods could not cope with the experiment of changing the contrast, believing that the best result is the most contrasting image but the original. Instead, the reference methods showed an excellent result (except one, which showed partial ineffectiveness). Also, it has been shown performance comparison between methods. It is shown that most of the studied methods calculate local estimates for each frame, and their arithmetic mean value is an estimate of the quality of the entire video file. If the video is dominated by large areas of uniform evaluation, methods of this type may give incorrect quality evaluations that do not coincide with the visual evaluations.

Spatial and Temporal Feature-Based Reduced Reference Quality Assessment for Rate-Varying Videos in Wireless Networks

For the impact of the bitrate change of video streaming services according to the available bandwidth on user satisfaction, in this paper, we propose a spatial and temporal feature-based reduced reference (RR) quality assessment for rate-varying videos in wireless networks called STRQAW. First, simulating the orientation selectivity mechanism of the human visual system (HVS), the histogram of the orientation selectivity-based visual pattern in each frame is extracted as the spatial feature. The histogram similarity between the rate-varying video and the original video is computed as the spatial metric. Second, we extract the temporal variation of the DCT coefficients of the consecutive frame differences as the temporal feature. The temporal variation similarity between the rate-varying video and the original video is calculated as the temporal metric. Finally, we take into account the recency effect and assess the overall quality by combining the temporal and spatial metric. The experimental results using the Laboratory for Image and Video Engineering (LIVE) mobile video quality assessment (VQA) database show that STRQAW is consistent with the subjective assessment results, which means it reflects human subjective feelings well and it provides an evaluation for adjusting compression-coding rates in real time. STRQAW can be used to guide video application providers and network operators working towards satisfying end-user experiences.

A no-reference framework for evaluating video quality streamed through wireless network

In this work, a no-reference framework is proposed for the video quality estimation streamed through the wireless network. The work presents a comprehensive survey of the existing full reference (FR), reduced reference (RR), and no-reference (NR) algorithms. A comparison has been made among existing algorithms, i.e. in terms of subjective correlation and feasibility to use these algorithms in wireless architecture, to describe the necessity of the proposed framework to overcome the limitations of the existing algorithms. A brief summary of our previously published algorithms, i.e. NR blockiness, NR blur, NR network, NR just noticeable distortion, and RR, has also been presented. These algorithms have also been used as function modules in the proposed framework. The proposed framework is able to measure the video quality by taking into account major spatial, temporal, network impairments, and human visual system effects for a comprehensive quality evaluation. The proposed framework is able to measure the video quality compressed by different codecs, i.e. MPEG x / H.264x, Motion JPEG/Motion, and JPEG2000, etc. The framework is able to work with two different kinds of received data, i.e. bit streams and decoded pixels. The framework is an integration of the RR and NR method, and can work in three different modes depending on the availability of the RR data, i.e. 1) only RR measurement, 2) hybrid of RR and NR measurement, and 3) only NR estimation. In addition, any individual function block, i.e. blurring, can also be used independently for particular specific distortion. A new subjective video quality database containing compressed and distorted videos (due to channel induced distortions) is also developed to test the proposed framework. The framework has also been tested on publicly available LIVE Video Quality Database. Overall test results show that our framework demonstrates a strong correlation with subjective evaluation of the two separate video databases as compared with other existing algorithms. The proposed framework also shows good results while working only in NR mode as compared with existing RR and FR algorithms. The proposed framework is more scalable and feasible to use in any kind of available network bandwidth as compared with other algorithms, as it can be used in different modes by using different function modules.

Reduced-reference image quality metric based on statistic model in complex wavelet transform domain

A new Reduced-Reference (RR) image quality metric based on statistical models in the complex wavelet transform domain is proposed. The magnitude and the relative phase information of the complex wavelet coefficients is modeled by using probability density function, and a strategy based on the information criterion is proposed to optimally approximate the distribution. To further improve the accuracy of the metric, a comparison of the candidate models is studied, and the inverse Gaussian distribution and the wrapped Cauchy distribution are selected to model the magnitude and the relative phase distributions, respectively. The Kullback–Leibler divergence between the distributions of the reference image and the distorted one serves as the RR feature to measure the distortion. Finally, a generalized regression neural network is employed to map the RR feature into an objective score. Experimental studies confirmed that the proposed RR image quality metric is quality-aware and highly correlated with the human visual system.

Blind Quality Metric for Contrast-Distorted Images Based on Eigendecomposition of Color Histograms

Although contrast is a major issue in overall quality assessment of an image, existing contrast evaluators with a reasonable performance are currently scarce. Here, we propose a learning-based blind/no-reference (NR) image quality assessment (IQA) model, a dubbed histogram eigen-feature-based contrast score (HEFCS) for evaluating image contrast. This research seeks for the inter-relationship between contrast degradation and relevant image histogram features. We introduce “eigen-histograms,” which are the eigenvectors of the set of image patches’ histograms. We found that the randomness of image eigen-histograms and the amplitude of corresponding eigenvalues can reliably reflect the changes in image contrast. Employing these characteristics leads to contrast-aware HEF vectors, which are used to compute the contrast score through a prediction model trained using support vector regression. Extensive analysis and cross validation are performed with five contrast relevant image databases, and the HEFCS performance results are compared with a collection of full-reference (FR), reduced-reference (RR), and NR measures. Despite its simplicity and low computational complexity, the HEFCS performs better than all competing NR-IQA models and also stands among the three best performers of FR and RR models.

An effective general-purpose NR-IQA model using natural scene statistics (NSS) of the luminance relative order

Blind/no-reference image quality assessment (NR-IQA) aims to assess the quality of an image without any reference image. In this paper, we propose an effective and efficient general-purpose NR-IQA model using natural scene statistics (NSS) of the luminance relative order, based on the observation that the variation of the marginal distribution of the relative order coefficients effectively reflect the degree of warping caused by different types of image distortions. In the literature, gradient-relevant methods have had a big success in full-reference (FR) IQA and reduced-reference (RR) IQA. Inspired by these, we extend it to NR-IQA in this paper. Notice that the NSS-based models usually extract their features derived from the spatial, wavelet, DCT and spectral domain etc. Unlike these metrics, the proposed method firstly extracts 32 natural scene statistics features of the luminance relative order, obtained from the log histograms of log horizontal, vertical, main-diagonal and secondary-diagonal derivatives, along with kurtosis, variance, differential entropy and entropy at two scales. Then a mapping is learned to predict the quality score using a support vector regression. The experimental results on several benchmark databases showed that the proposed method is comparable with the state-of-the-art methods and has a relatively low complexity.

Reduced-Reference Stereoscopic Image Quality Assessment Using Natural Scene Statistics and Structural Degradation

Perceptual stereo image quality assessment (SIQA) aims to design computational models to measure the stereo image quality in accordance with human opinions. In this paper, a novel reduced-reference (RR) SIQA is proposed by characterizing the statistical and perceptual properties of the stereo image in both the spatial and gradient domains. To be specific, in the spatial domain, we extract the parameters of the generalized Gaussian distribution fits of luminance wavelet coefficients to form the underlying features. In the gradient domain, the modified gradient magnitudes maps are generated by jointly considering human visual system’s contrast sensitivity and neighborhood gradient information to weight the gradient magnitudes in a locally adaptive manner. Afterward, perceptual features are extracted based on the entropy of discrete wavelet transform coefficients of modified gradient magnitudes. Furthermore, we consolidate the left and right features into a single set of features per stereo image pair. Finally, the qualities of both the spatial and gradient domains are combined to obtain the overall quality of stereo image. Extensive experiments performed on popular data sets demonstrate that the proposed RR-SIQA method achieves highly competitive performance as compared with the state-of-the-art RR-SIQA models as well as full-reference ones for both symmetric and asymmetric distortions.

Attended Visual Content Degradation Based Reduced Reference Image Quality Assessment

Reduced-reference (RR) image quality assessment (IQA), which aims to use a small amount of the reference information but achieve high accuracy, is greatly demanded in quality-orientated systems. In order to design a better RR IQA model which performs consistently with the subjective perception, the inner mechanism of the human visual system (HVS) is usually investigated and imitated. In this paper, the attention mechanism is thoroughly analyzed and used for RR IQA modeling. Generally, the HVS is more sensitive to the distortion on the attended region than that on the unattended region. Thus, the saliency of each region is calculated to highlight its importance, and a saliency weighted local structure (SWLS)-based histogram is created for visual structure degradation measurement. Meanwhile, the distortion may cause attention shift (changing the attended region). In other words, the difference of attention between the reference and distorted images can efficiently represent the quality degradation. Therefore, the attention distribution is analyzed with the salient map, and an orientation located global saliency (OLGS)-based histogram is built for attention shift measurement. Finally, combining the quality degradations from both SWLS and OLGS, a novel attended visual content degradation-based RR IQA method is introduced. 1 Experimental results demonstrate that the proposed method uses only several values (18 values) and performs consistently with the subjective perception. Moreover, the proposed attention procedure can be easily extended to the existing RR IQA models and improve their performances. 1 The source code of the proposed method will be available at http://web.xidian.edu.cn/wjj/en/index.html

Reduced-reference image quality assessment based on distortion families of local perceived sharpness

Previous research on reduced reference (RR) image quality assessment (IQA) suggested that appropriate RR features should provide efficient summaries of reference images and be sensitive to a variety of image distortions. The multi-scale local sharpness maps are effective RR features because they can capture smooth, edge, and textured areas of the reference image, and they are affected differently by different distortion types. Motivated by this observation, in this paper, we propose an efficient RR IQA algorithm using local sharpness. Our method, called S4RR, employs four sharpness maps (two FISH maps and two local standard deviation maps) to assess image quality via two main stages. The first stage soft-classifies the distorted image into eight distortion families based on an analysis of the different scatter-plot shapes of the sharpness map values of distorted image vs. reference image. The second st age performs distortion-family-specific quality assessment based on measuring the local sharpness variations between reference and distorted images by using seven types of local statistics and six distance measures. Finally, the soft-classification probabilities computed from the first stage are combined with the distortion-family-specific quality scores to yield a class-weighted average, which serves as the final S4RR quality index. Experiment results tested on various databases show that with less than 5% RR information, the proposed S4RR algorithm achieves better/competitive performance as compared to other state-of-the-art FR/RR IQA algorithms.

Perceptual Reduced-Reference Visual Quality Assessment for Contrast Alteration

In image/video systems, contrast adjustment which manages to enhance visual quality is nowadays an important research topic. Yet very limited struggles have been made to the exploration of visual quality assessment for contrast adjustment. To tackle the issue, this paper proposes a novel reduced-reference (RR) quality metric with the integration of bottom-up and top-down strategies. The former one stems from the recently revealed free energy principle that tells that the human visual system seeks to comprehend an input image via uncertainty removal, while the latter one is toward using the symmetric Kullback–Leibler divergence to compare the histogram of the contrast-altered image with that of the pristine image. The bottom-up and top-down strategies are lastly incorporated to derive the RR contrast-altered image quality measure. A comparison using numerous existing IQA models is carried out on five contrast related databases/subsets in CID2013, CCID2014, CSIQ, TID2008, and TID2013, and experimental results validate the superiority of the proposed technique.

Reduced-Reference Quality Assessment Based on the Entropy of DWT Coefficients of Locally Weighted Gradient Magnitudes.

Perceptual image quality assessment (IQA) attempts to use computational models to estimate the image quality in accordance with subjective evaluations. Reduced-reference (RR) image quality assessment (IQA) methods make use of partial information or features extracted from the reference image for estimating the quality of distorted images. Finding a balance between the number of RR features and accuracy of the estimated image quality is essential and important in IQA. In this paper we propose a training-free low-cost RRIQA method that requires a very small number of RR features (6 RR features). The proposed RRIQA algorithm is based on the discrete wavelet transform (DWT) of locally weighted gradient magnitudes.We apply human visual system's contrast sensitivity and neighborhood gradient information to weight the gradient magnitudes in a locally adaptive manner. The RR features are computed by measuring the entropy of each DWT subband, for each scale, and pooling the subband entropies along all orientations, resulting in L RR features (one average entropy per scale) for an L-level DWT. Extensive experiments performed on seven large-scale benchmark databases demonstrate that the proposed RRIQA method delivers highly competitive performance as compared to the state-of-the-art RRIQA models as well as full reference ones for both natural and texture images. The MATLAB source code of REDLOG and the evaluation results are publicly available online at https://http://lab.engineering.asu.edu/ivulab/software/redlog/.

Reorganized DCT-based image representation for reduced reference stereoscopic image quality assessment

In this paper, a novel reduced reference (RR) stereoscopic image quality assessment (SIQA) is proposed by characterizing the statistical properties of the stereoscopic image in the reorganized discrete cosine transform (RDCT) domain. Firstly, the difference image between the left and right view images is computed. Afterwards, the left and right view images, as well as the difference image, are decomposed by block-based discrete cosine transform (DCT). The DCT coefficients are further reorganized into a three-level coefficient tree, resulting in ten RDCT subbands. For each RDCT subband, the statistical property of the coefficient distribution is modeled by the generalized Gaussian density (GGD) function. And the mutual information (MI) and energy distribution ratio (EDR) are employed to depict the statistical properties across different RDCT subbands. Moreover, EDR can further model the mutual masking property of the human visual system (HVS). By considering the GGD modeling behavior within each RDCT subband and MI together EDR characterizing behavior across RDCT subbands, the statistical properties of the stereoscopic image are fully exploited, including the left view, right view, and the difference image. Experimental results demonstrated that the statistical properties of the difference image can well represent the perceptual quality of the stereoscopic image, which outperforms the representative RR quality metrics for stereoscopic image and even some full reference (FR) quality metrics. By considering the left view, right view, and difference image together, the performances of the proposed RR SIQA can be further improved, which presenting a more closely relationship between the quality metric output and human visual perception.

Visual structural degradation based reduced-reference image quality assessment

Reduced-reference (RR) image quality assessment (IQA) aims to use less reference data for quality evaluation. Global features are demanded to effectively express quality degradation caused by distortion. Since the human visual system (HVS) is highly sensitive to structure degradation, we suggest to represent the visual content of an image with several structural patterns. Also, image quality is measured based on the structural degradation on these patterns. Firstly, the spatial correlation of image structure is analyzed with the local binary pattern (LBP), and several representative structural patterns are extracted. Then, the energy changes on these patterns between the reference image and the distorted image are calculated. Finally, the support vector regression (SVR) procedure is adopted for feature pooling, and the quality score of the input image is returned. Experimental results for three publicly available databases demonstrate that the proposed RR IQA method is highly consistent with the human perception, which uses a small amount of reference data (20 values) and achieves a promising improvement on quality evaluation accuracy.

Binocular perception based reduced-reference stereo video quality assessment method

A new reduced-reference (RR) stereo video quality assessment method is proposed in this paper by considering temporal characteristics of video and binocular perception in human visual system (HVS). Firstly, motion intensity is utilized to extract RR frames for the purpose of temporal characteristics in stereo video. Secondly, according to internal generative mechanism of HVS, fusion and rivalry in the process of binocular perception is modeled, and the RR frames are divided into binocular fusion portion and binocular rivalry portion. Then, RR frame quality indicators are computed for these two portions. Finally, the RR frame quality indicators of the original and distorted frames are compared. A temporal pooling strategy is utilized on these quality indicators to obtain final stereo video quality score, where the motion intensity is used for toning the pooling parameters. Experimental results show that the proposed method has better performances when compared to other state-of-the-art quality assessment methods.

Orientation selectivity based visual pattern for reduced-reference image quality assessment

Image quality assessment (IQA) is in great demand for high quality image selection in the big data era. The challenge of reduced-reference (RR) IQA is how to use limited data to effectively represent the visual content of an image in the context of IQA. Research on neuroscience indicates that the human visual system (HVS) exhibits obvious orientation selectivity (OS) mechanism for visual content extraction. Inspired by this, an OS based visual pattern (OSVP) is proposed to extract visual content for RR IQA in this paper. The OS arises from the arrangement of the excitatory and inhibitory interactions among connected cortical neurons in a local receptive field. According to the OS mechanism, the similarity of preferred orientations between two nearby pixels is first analyzed. Then, the orientation similarities of pixels in a local neighborhood are arranged, and the OSVP is built for visual information representation. With the help of OSVP, the visual content of an image is extracted and mapped into a histogram. By calculating the changes between the two histograms of reference and distorted images, a quality score is produced. Experimental results on five public databases demonstrate that the proposed RR IQA method has performance consistent with the human perception under a small amount of reference data (only 9 values).