Reduced-reference Image Quality Assessment Research Articles

Super-resolution image visual quality assessment based on structure–texture features

Assessing the visual quality of super-resolution images (SRIs) is crucial for advancing algorithm development, but it remains an unsolved problem. In this paper, we present a novel reduced-reference image quality assessment (RR-IQA) method specifically suited for evaluating SRIs. Our approach leverages information from the input low-resolution (LR) image as a reference signal to extract features that are most relevant to modeling the visual quality of SRIs. We analyze the artifact characteristics of SRIs and demonstrate that features describing edge orientations, high frequency components, and textures are the most important for this task. To extract these features, we first perform structure–texture decompositions (STD) on both the SRI and its LR input, then obtain the edge orientation feature through a traditional hand-crafted approach, and deep neural networks to extract features related to high frequency components and textures. We employ a shallow multilayer perceptron (MLP) to predict an image quality score based on these quality-relevant features. To improve feature representation ability and prevent overfitting, we pretrain the feature extraction module using a large number of unlabeled samples, which accounts for over 99.8% of the total model parameters. We use precious samples with mean opinion score (MOS) labels to train a high-quality shallow MLP predictor. Our experimental results show that the proposed method outperforms classical and state-of-the-art models.

Degraded Reference Image Quality Assessment.

In practical media distribution systems, visual content usually undergoes multiple stages of quality degradation along the delivery chain, but the pristine source content is rarely available at most quality monitoring points along the chain to serve as a reference for quality assessment. As a result, full-reference (FR) and reduced-reference (RR) image quality assessment (IQA) methods are generally infeasible. Although no-reference (NR) methods are readily applicable, their performance is often not reliable. On the other hand, intermediate references of degraded quality are often available, e.g., at the input of video transcoders, but how to make the best use of them in proper ways has not been deeply investigated. Here we make one of the first attempts to establish a new paradigm named degraded-reference IQA (DR IQA). Specifically, by using a two-stage distortion pipeline we lay out the architectures of DR IQA and introduce a 6-bit code to denote the choices of configurations. We construct the first large-scale databases dedicated to DR IQA and will make them publicly available. We make novel observations on distortion behavior in multi-stage distortion pipelines by comprehensively analyzing five multiple distortion combinations. Based on these observations, we develop novel DR IQA models and make extensive comparisons with a series of baseline models derived from top-performing FR and NR models. The results suggest that DR IQA may offer significant performance improvement in multiple distortion environments, thereby establishing DR IQA as a valid IQA paradigm that is worth further exploration.

Perceptual Hashing With Complementary Color Wavelet Transform and Compressed Sensing for Reduced-Reference Image Quality Assessment

Image quality assessment (IQA) is an important task of image processing and has diverse applications, such as image super-resolution reconstruction, image transmission and monitoring systems. This paper proposes a perceptual hashing algorithm with complementary color wavelet transform (CCWT) and compressed sensing (CS) for reduced-reference (RR) IQA. The CCWT is exploited to decompose input color image into different sub-bands. Since the calculation of CCWT uses all color channels without discarding any information, the distortions introduced by digital operations on color channels are preserved in the CCWT sub-bands. The block-based CS is used to extract features from the CCWT sub-bands. As the Euclidean distance between the block-based CS features is slightly influenced by content-preserving operations, perceptual features constructed by Euclidean distances are robust, discriminative and compact. Hash sequence is finally determined by quantifying the perceptual features. Effectiveness of the proposed hashing is verified by various experiments on four open image databases. Experimental results demonstrate that the proposed hashing is superior to some state-of-the-art algorithms in terms of classification and RR IQA application.

A Novel Reduced Reference Image Quality Assessment Based on Formal Concept Analysis

Abstract The assessment of image quality provides the confidentiality, safety, quality and transparency of the obtained images. At the same time, there are many existing quality metric approaches that belong to the group of modification alteration procedures (pixel-based metric). These techniques have not been well-matched with perceptual image quality. The Perceptual Human Visual System (HVS) aspect drives our approach; The human visual system (HVS) is the best judge of image quality. This paper presents a new reduced reference image quality metric in the spatial domain that reduces the complexity of the image quality assessment. From the reference and distortion images, we extract four intrinsic features, namely, contrast, entropy, histogram and standard deviation. Then, we build the formal concept analysis matrix for reference and distortion images. Finally, we compare the obtained matrixes to evaluate the image quality. The performance of the proposed technique is assessed using LIVE, TID2013 and CSIQ datasets, and the obtained results are compared in terms of PSNR, SSIM and NCC metrics. Also, a comparison with more recent and relevant approaches is performed to highlight the superior performance of our proposed approach, the experimental results indicated that the proposed approach provides efficient performance among the compression, Gaussian blur, Contrast and add noise distortion types.

Disentangling Image distortions in deep feature space

Previous literature suggests that perceptual similarity is an emergent property shared across deep visual representations. Experiments conducted on a dataset of human-judged image distortions have proven that deep features outperform classic perceptual metrics. In this work we take a further step in the direction of a broader understanding of such property by analyzing the capability of deep visual representations to intrinsically characterize different types of image distortions. To this end, we firstly generate a number of synthetically distorted images and then we analyze the features extracted by different layers of different Deep Neural Networks. We observe that a dimension-reduced representation of the features extracted from a given layer permits to efficiently separate types of distortions in the feature space. Moreover, each network layer exhibits a different ability to separate between different types of distortions, and this ability varies according to the network architecture. Finally, we evaluate the exploitation of features taken from the layer that better separates image distortions for: i) reduced-reference image quality assessment, and ii) distortion types and severity levels characterization on both single and multiple distortion databases. Results achieved on both tasks suggest that deep visual representations can be unsupervisedly employed to efficiently characterize various image distortions.

Full and Reduced Reference Image Quality Assessment of Panoramic View using Novel Hybrid Image Stitching Method

Image Stitching is becoming more popular in field of computer vision because of rapid development of efficient algorithms that replaces the high cost wider lens cameras and commercial image stitching tools. The existing methods used global geometric transformation in registration stage and hence suffered from object deformation, parallax error, ghosting effect and motion blur in output result. In this paper, newly developed Hybrid Warping of weighted linearized homography matrix and similarity transform matrix is implemented over standard image stitching database. The visual quality of stitched image using proposed method has been examined in terms of performance metrics of Full Reference Image Quality Assessment (FRIQA) such as Root Mean Square Error (RMSE), Peak Signal-to-Noise Ratio (PSNR), Structural Similarity Index (SSIM) and Reduced Reference Image Quality Assessment (RRIQA). Also, the performance analysis of proposed method is compared against existing image stitching methods in terms of field of view and stitching time. This analysis has ascertained the outperformance of Novel Hybrid Image Stitching method.

Reduced Reference Image Quality Assessment Technique Based on DWT and Path Integral Local Binary Patterns

In the prevailing technological advancements, image and video services are one of the primary concerns of researches around the world. The transmission of multimedia contents involved various types of attenuations such as blurring, ringing, contrast, and blocking. To evaluate the quality of the image or video, researchers presented various quality assessment techniques, including subjective and objective assessments. In this article, a robust reduced reference image quality assessment technique based on the objective assessment is presented, which considers the extracted features of the underlying image to evaluate its quality. The proposed method uses a discrete wavelet transform-based path integral local binary pattern histogram as extracted features. These features of the reference and testing image are compared by a Euclidean distance formula to generate a single value to evaluate the quality of the image. The qualitative and quantitative analysis of the proposed method is carried on three different image datasets, namely LIVE, TID2013, and CSIQ. The experimental results of the proposed method indicate that it produces robust performance in terms of performance evaluation metrics as compared with state-of-the-art methods.

Just-noticeable difference binary pattern for reduced reference image quality assessment

Just-noticeable difference (JND) is defined as the smallest intensity change in an image that can be noticed by the human vision system (HVS). Any perceptible distortion level must be greater than the JND. Based on this observation, a local binary pattern (LBP) is developed for image quality assessment. First, the JND map of the image is computed. The spatial and relative intensity relationships among pixels in a local neighborhood are employed to generate the proposed LBP based on the JND map. Then, image contrast is used as a weighting factor for the LBP histogram generation to characterize the structural and contrast information of the image. Finally, the contrast and structure changes due to image distortion are measured by calculating the similarity between contrast-weighted histograms of the reference and distorted images. Support vector regression is employed to pool the similarity to predict the quality. Experimental results on benchmark databases demonstrate that the proposed LBP can effectively and accurately measure image quality, which is consistent with the HVS. The proposed method achieves high consistency with subjective perception using 18 reference values and performs better than other state-of-the-art reduced reference image quality assessment methods.

Multi-Channel Decomposition in Tandem With Free-Energy Principle for Reduced-Reference Image Quality Assessment

The visual quality of perceptions is highly correlated with the mechanisms of the human brain and visual system. Recently, the free-energy principle, which has been widely researched in brain theory and neuroscience, is introduced to quantize the perception, action, and learning in human brain. In the field of image quality assessment (IQA), on one hand, the free-energy principle can resort to the internal generative model to simulate the visual stimulus of the human beings. On the other hand, abundant psychological and neurobiological studies reveal that different frequency and orientation components of one visual stimulus arouse different neurons in the striate cortex, and the striate cortex processes visual information in the cerebral cortex. Motivated by these two aspects, a novel reduce-reference IQA metric called the multi-channel free-energy based reduced-reference quality metric is proposed in this paper. First, a two-level discrete Haar wavelet transform is used to decompose the input reference and distorted images. Next, to simulate the generative model in the human brain, the sparse representation is leveraged to extract the free-energy-based features in subband images. Finally, the overall quality metric is obtained through the support vector regressor. Extensive experimental comparisons on four benchmark image quality databases (LIVE, CSIQ, TID2008, and TID2013) demonstrate that the proposed method is highly competitive with the representative reduced-reference and classical full-reference models.

A reduced-reference quality assessment metric for super-resolution reconstructed images with information gain and texture similarity

Super-resolution (SR) image reconstruction has been extensively studied in recent years due to its broad uses in machine vision, medical imaging, remote sensing and monitoring systems. However, evaluating the performance of SR algorithms is still an ongoing problem. A number of image quality metrics have been reported in recent years, however, they are not specifically designed for SR reconstructed images, so they are usually limited when assessing SR images. Here, we propose a reduced-reference image quality assessment (IQA) metric for SR images. First, saliency detection is used on the high-resolution (HR) images, and low-resolution (LR) images are used to generate the corresponding saliency maps. Second, the information gain and texture similarity between the HR images and the LR images are calculated to quantify the image quality degradation. Finally, the information gain and the texture similarity are weighted to predict the quality of SR images. Extensive experiments illustrate that the proposed metric has better performance for SR images than the existing state-of-the-art IQA algorithms.

Effective quality metric for contrast-distorted images based on SVD

Proper contrast usually makes an object distinguishable, while contrast degradation will affect our understanding of the object. Many experiments found that the human visual system is more sensitive to contrast than to luminance in distinguishing an object. However, few studies have been conducted to assess the quality of contrast change effectively. In our study, we first reveal some characteristics of contrast-distorted images by analyzing image data. Based on these characteristics, we then present two novel and fast full-reference image quality metrics: FC-SVD for a full-reference model and RC-SVD for a reduced-reference image quality assessment model, both using singular value decomposition (SVD). The proposed methods are evaluated through four public contrast-related databases (TID2013, TID2008, CCID2014, and CSIQ). The results show that they have excellent performance in prediction accuracy and much lower computation complexity than other state-of-the-art image quality assessment methods.

Reduced-reference image quality assessment method based on wavelet feature extraction and fusion

Virtual reality has many characteristics such as multi-sensation, interactivity, and presence. Virtual reality images, also known as 360-degree panoramic images, provide users with an immersive experience and have been widely used in education, medical, military and many other fields in recent years. 360-degree panoramic images have high resolution and are subject to varying degrees of distortion during transmission and storage. Therefore, the evaluation of virtual reality images has practical significance. To this end, we propose a method for evaluating the quality of images in the new CVIQD database. The proposed image quality assessment model is developed based on Haar wavelet and Db4 wavelet feature extraction and fusion. The method is compared with a single wavelet feature extraction method. Experiments show that the method has better performance.

Reduced-reference quality assessment of multiply-distorted images based on structural and uncertainty information degradation

The majority of existing objective Image Quality Assessment (IQA) methods are designed for evaluation of images corrupted by single distortion types. However, images may be degraded with multiple distortions during processing stages. In this paper, we propose a reduced-reference IQA algorithm to predict the quality of multiply-distorted images. An image is first decomposed into predicted and disorderly portions based on the internal generative mechanism theory. The structural information is captured from the predicted image by using a shearlet representation and Rényi directional entropy is deployed to measure the disorderly information changes. Finally, we introduce the application of a framework namely Learning Using Privileged Information (LUPI) to build a quality model and obtain quality scores. During training, the LUPI framework utilizes a set of additional privileged data to learn an improved quality model. Experimental results on multiply-distorted image datasets (MLIVE and MDID2015) confirm the effectiveness of the proposed IQA model.

Variational based smoke removal in laparoscopic images

BackgroundIn laparoscopic surgery, image quality can be severely degraded by surgical smoke, which not only introduces errors for the image processing algorithms (used in image guided surgery), but also reduces the visibility of the observed organs and tissues. To overcome these drawbacks, this work aims to remove smoke in laparoscopic images using an image preprocessing method based on a variational approach.MethodsIn this paper, we present the physical smoke model where the degraded image is separated into two parts: direct attenuation and smoke veil and propose an efficient variational-based desmoking method for laparoscopic images. To estimate the smoke veil, the proposed method relies on the observation that smoke veil has low contrast and low inter-channel differences. A cost function is defined based on this prior knowledge and is solved using an augmented Lagrangian method. The obtained smoke veil is then subtracted from the original degraded image, resulting in the direct attenuation part. Finally, the smoke free image is computed using a linear intensity transformation of the direct attenuation part.ResultsThe performance of the proposed method is evaluated quantitatively and qualitatively using three datasets: two public real smoked laparoscopic datasets and one generated synthetic dataset. No-reference and reduced-reference image quality assessment metrics are used with the two real datasets, and show that the proposed method outperforms the state-of-the-art ones. Besides, standard full-reference ones are employed with the synthetic dataset, and indicate also the good performance of the proposed method. Furthermore, the qualitative visual inspection of the results shows that our method removes smoke effectively from the laparoscopic images.ConclusionAll the obtained results show that the proposed approach reduces the smoke effectively while preserving the important perceptual information of the image. This allows to provide a better visualization of the operation field for surgeons and improve the image guided laparoscopic surgery procedure.

Combined Blockiness, Blurriness and White Noise Distortion Meter

Due to discrete cosine transform (DCT) based compression, the loss appears in the form of blocky and blurry distortions in coded pictures. One of the degradation is white noise which corrupts nearly all pixels in the picture. In image transmission system, to assess the image quality at receiver end, lesser image features are sent, instead of the whole reference original picture to accommodate the available bandwidth. The purpose of this research is to propose a procedure which quantifies or estimates the objective picture quality automatically. The presented combined white noise image quality assessment meter (CWNIQAM) functions in the frequency domain for computation of three artefacts blockiness, blurriness and white noise in the corrupted coded images. The designed quality meter at first converts RGB pictures into YCbCr, gray scaled pictures and then edge detection method is applied. Afterward the picture is divided into blocks of 32 × 32 pixels to get local level blockiness, blurriness and white noise value. Next the picture is transformed from spatial to frequency domain and different features called reduced reference parameters are estimated. The composite magnitudes strength through horizontal and vertical harmonics is obtained for estimation of blocky and blurry artefacts in the coded images, while the strength of all ac coefficients and dc component is obtained for computation of white noise. The reduced reference features are computed and compared for reference and coded picture for estimating the particular type of distortion. The uniqueness of the proposed CWNIQAM meter is that, it can estimate the quality of corrupted images by estimating the combination of blockiness, blurriness and white noise through this single meter. The LIVE database2 having 174 different white noise (wn) corrupted images are used for measuring combined objective value of the three artefacts using the designed reduced reference image quality assessment meter algorithm. The objective results are correlated with subjective wn, DMOS LIVE scores. The correlation of about 98% is attained which indicate that much better picture quality index is achieved by the offered reduced reference combined white noise image quality evaluation meter in the frequency domain.

Perceptual Image Hashing with Weighted DWT Features for Reduced-Reference Image Quality Assessment

Perceptual Image Hashing with Weighted DWT Features for Reduced-Reference Image Quality Assessment

A new reduced-reference image quality assessment based on the SVD signal projection

A new image quality metric is proposed in this paper based on the degradation of structural information. It uses the singular value decomposition (SVD) as a structural projection tool called SVD-based reduced-reference image quality evaluator (SBR-IQE). This method employs the SVD signal projection to factorize the reference image matrix as well as its distorted version into their components including singular vectors and values. The singular vectors contain structural information and singular values determine the importance of each singular vector in the image structure. Thus, the minutiae perceptual information could be eliminated by singular values diagnosis. The remaining components are considered as the image features. The task of similarity measurement includes three comparisons based on luminance, contrast and structure. The overall quality evaluation is obtained according to these three comparisons. Experimental results have demonstrated that the proposed metric outperforms the state-of-the-art RR-IQA metrics and enjoys lower computational cost as well.

Reduced-Reference Image Quality Assessment in Free-Energy Principle and Sparse Representation

The free-energy principle in recent studies of brain theory and neuroscience models the perception and understanding of the outside scene as an active inference process, in which the brain tries to account for the visual scene with an internal generative model. Specifically, with the internal generative model, the brain yields corresponding predictions for its encountered visual scenes. Then, the discrepancy between the visual input and its brain prediction should be closely related to the quality of perceptions. On the other hand, sparse representation has been evidenced to resemble the strategy of the primary visual cortex in the brain for representing natural images. With the strong neurobiological support for sparse representation, in this paper, we approximate the internal generative model with sparse representation and propose an image quality metric accordingly, which is named FSI (free-energy principle and sparse representation-based index for image quality assessment). In FSI, the reference and distorted images are, respectively, predicted by the sparse representation at first. Then, the difference between the entropies of the prediction discrepancies is defined to measure the image quality. Experimental results on four large-scale image databases confirm the effectiveness of the FSI and its superiority over representative image quality assessment methods. The FSI belongs to reduced-reference methods, and it only needs a single number from the reference image for quality estimation.

A Prediction Backed Model for Quality Assessment of Screen Content and 3-D Synthesized Images

In this paper, we address problems associated with free-energy-principle-based image quality assessment (IQA) algorithms for objectively assessing the quality of Screen Content (SC) and three-dimensional (3-D) synthesized images and also propose a very fast and efficient IQA algorithm to address these issues. These algorithms separate an image into predicted and disorder residual parts and assume disorder residual part does not contribute much to the overall perceptual quality. These algorithms fail for quality estimation of SC images as information of textual regions in SC images are largely separated into the disorder residual part and less information in the predicted part and subsequently, given a negligible emphasis. However, this is in contrast with the characteristics of human vision. Since our eyes are well trained to detect text in daily life. So, our human vision has prior information about text regions and can sense small distortions in these regions. In this paper, we proposed a new reduced-reference IQA algorithm for SC images based upon a more perceptually relevant prediction model and distortion categorization, which overcomes problems with existing free-energy-principle-based predictors. From experiments, it is validated that the proposed model has a better capability of efficiently estimating the quality of SC images as compared to the recently developed reduced-reference IQA algorithms. We also applied the proposed algorithm to judge the quality of 3-D synthesized images and observed that it even achieves better performance than the full-reference IQA metrics specifically designed for the 3-D synthesized views.

Attended Visual Content Degradation Based Reduced Reference Image Quality Assessment

Reduced-reference (RR) image quality assessment (IQA), which aims to use a small amount of the reference information but achieve high accuracy, is greatly demanded in quality-orientated systems. In order to design a better RR IQA model which performs consistently with the subjective perception, the inner mechanism of the human visual system (HVS) is usually investigated and imitated. In this paper, the attention mechanism is thoroughly analyzed and used for RR IQA modeling. Generally, the HVS is more sensitive to the distortion on the attended region than that on the unattended region. Thus, the saliency of each region is calculated to highlight its importance, and a saliency weighted local structure (SWLS)-based histogram is created for visual structure degradation measurement. Meanwhile, the distortion may cause attention shift (changing the attended region). In other words, the difference of attention between the reference and distorted images can efficiently represent the quality degradation. Therefore, the attention distribution is analyzed with the salient map, and an orientation located global saliency (OLGS)-based histogram is built for attention shift measurement. Finally, combining the quality degradations from both SWLS and OLGS, a novel attended visual content degradation-based RR IQA method is introduced. 1 Experimental results demonstrate that the proposed method uses only several values (18 values) and performs consistently with the subjective perception. Moreover, the proposed attention procedure can be easily extended to the existing RR IQA models and improve their performances. 1 The source code of the proposed method will be available at http://web.xidian.edu.cn/wjj/en/index.html