Full-reference Image Quality Assessment Metrics Research Articles

No-Reference Hyperspectral Image Quality Assessment via Ranking Feature Learning

In hyperspectral image (HSI) reconstruction tasks, due to the lack of ground truth in real imaging processes, models are usually trained and validated on simulation datasets and then tested on real measurements captured by real HSI imaging systems. However, due to the gap between the simulation imaging process and the real imaging process, the best model validated on the simulation dataset may fail on real measurements. To obtain the best model for the real-world task, it is crucial to design a suitable no-reference HSI quality assessment metric to reflect the reconstruction performance of different models. In this paper, we propose a novel no-reference HSI quality assessment metric via ranking feature learning (R-NHSIQA), which calculates the Wasserstein distance between the distribution of the deep features of the reconstructed HSIs and the benchmark distribution. Additionally, by introducing the spectral self-attention mechanism, we propose a Spectral Transformer (S-Transformer) to extract the spatial-spectral representative deep features of HSIs. Furthermore, to extract quality-sensitive deep features, we use quality ranking as a pre-training task to enhance the representation capability of the S-Transformer. Finally, we introduce the Wasserstein distance to measure the distance between the distribution of the deep features and the benchmark distribution, improving the assessment capacity of our method, even with non-overlapping distributions. The experimental results demonstrate that the proposed metric yields consistent results with multiple full-reference image quality assessment (FR-IQA) metrics, validating the idea that the proposed metric can serve as a substitute for FR-IQA metrics in real-world tasks.

An Optimization-Based Family of Predictive, Fusion-Based Models for Full-Reference Image Quality Assessment.

Given the reference (distortion-free) image, full-reference image quality assessment (FR-IQA) algorithms seek to assess the perceptual quality of the test image. Over the years, many effective, hand-crafted FR-IQA metrics have been proposed in the literature. In this work, we present a novel framework for FR-IQA that combines multiple metrics and tries to leverage the strength of each by formulating FR-IQA as an optimization problem. Following the idea of other fusion-based metrics, the perceptual quality of a test image is defined as the weighted product of several already existing, hand-crafted FR-IQA metrics. Unlike other methods, the weights are determined in an optimization-based framework and the objective function is defined to maximize the correlation and minimize the root mean square error between the predicted and ground-truth quality scores. The obtained metrics are evaluated on four popular benchmark IQA databases and compared to the state of the art. This comparison has revealed that the compiled fusion-based metrics are able to outperform other competing algorithms, including deep learning-based ones.

No-reference perceptual CT image quality assessment based on a self-supervised learning framework

Accurate image quality assessment (IQA) is crucial to optimize computed tomography (CT) image protocols while keeping the radiation dose as low as reasonably achievable. In the medical domain, IQA is based on how well an image provides a useful and efficient presentation necessary for physicians to make a diagnosis. Moreover, IQA results should be consistent with radiologists’ opinions on image quality, which is accepted as the gold standard for medical IQA. As such, the goals of medical IQA are greatly different from those of natural IQA. In addition, the lack of pristine reference images or radiologists’ opinions in a real-time clinical environment makes IQA challenging. Thus, no-reference IQA (NR-IQA) is more desirable in clinical settings than full-reference IQA (FR-IQA). Leveraging an innovative self-supervised training strategy for object detection models by detecting virtually inserted objects with geometrically simple forms, we propose a novel NR-IQA method, named deep detector IQA (D2IQA), that can automatically calculate the quantitative quality of CT images. Extensive experimental evaluations on clinical and anthropomorphic phantom CT images demonstrate that our D2IQA is capable of robustly computing perceptual image quality as it varies according to relative dose levels. Moreover, when considering the correlation between the evaluation results of IQA metrics and radiologists’ quality scores, our D2IQA is marginally superior to other NR-IQA metrics and even shows performance competitive with FR-IQA metrics.

Perceptual quality assessment for fine-grained compressed images

Recent years have witnessed the rapid development of image storage and transmission systems, in which image compression plays an important role. Generally speaking, image compression algorithms are developed to ensure good visual quality at limited bit rates. However, due to the different compression optimization methods, the compressed images may have different levels of quality, which needs to be evaluated quantificationally. Nowadays, the mainstream full-reference (FR) metrics are effective to predict the quality of compressed images at coarse-grained levels (the bit rates differences of compressed images are obvious), however, they may perform poorly for fine-grained compressed images whose bit rates differences are quite subtle. Therefore, to better improve the Quality of Experience (QoE) and provide useful guidance for compression algorithms, we propose a full-reference image quality assessment (FR-IQA) method for compressed images of fine-grained levels. Specifically, the reference images and compressed images are first converted to YCbCr color space. The gradient features are extracted from regions that are sensitive to compression artifacts. Then we employ the Log-Gabor transformation to further analyze the texture difference. Finally, the obtained features are fused into a quality score. The proposed method is validated on the fine-grained compression image quality assessment (FGIQA) database, which is especially constructed for assessing the quality of compressed images with close bit rates. The experimental results show that our metric outperforms mainstream FR-IQA metrics on the FGIQA database. We also test our method on other commonly used compression IQA databases and the results show that our method obtains competitive performance on the coarse-grained compression IQA databases as well.

Evaluation of motion artifacts in brain magnetic resonance images using convolutional neural network-based prediction of full-reference image quality assessment metrics.

Purpose: Motion artifacts in magnetic resonance (MR) images mostly undergo subjective evaluation, which is poorly reproducible, time consuming, and costly. Recently, full-reference image quality assessment (FR-IQA) metrics, such as structural similarity (SSIM), have been used, but they require a reference image and hence cannot be used to evaluate clinical images. We developed a convolutional neural network (CNN) model to quantify motion artifacts without using reference images. Approach: The brain MR images were obtained from an open dataset. The motion-corrupted images were generated retrospectively, and the peak signal-to-noise ratio, cross-correlation coefficient, and SSIM were calculated. The CNN was trained using these images and their FR-IQA metrics to predict the FR-IQA metrics without reference images. Receiver operating characteristic (ROC) curves were created for binary classification, with artifact scores indicating the need for rescanning. ROC curve analysis was performed on the binary classification of the real motion images. Results: The predicted FR-IQA metric having the highest correlation with the subjective evaluation was SSIM, which was able to classify images requiring rescanning with a sensitivity of 89.5%, specificity of 78.2%, and area under the ROC curve (AUC) of 0.930. The real motion artifacts were classified with the AUC of 0.928. Conclusions: Our CNN model predicts FR-IQA metrics with high accuracy, which enables quantitative assessment of motion artifacts in MR images without reference images. It enables classification of images requiring rescanning with a high AUC, which can improve the workflow of MR imaging examinations.

Adaptive total variation L1 regularization for salt and pepper image denoising

In this article, we propose an adaptive total variation (TV) regularization model for salt and pepper denoising in digital images. The adaptive TV denoising method is developed based on the general regularized image restoration model with L1 fidelity for handling salt and pepper noise model. An estimation for regularization parameter is also proposed based on the characteristics of the salt and pepper noise. We implement the proposed adaptive TV-L1 regularization model efficiently for image denoising using the primal dual gradient method. In the experiments, the full-reference image quality assessment metrics are used for evaluating denoising quality across various noise levels in different synthetic and real images. The denoising results are compared to other similar salt and pepper image denoising methods and our results indicate we obtain artifact free edge preserving restorations.

Pre-Attention and Spatial Dependency Driven No-Reference Image Quality Assessment

The excessive emulation of the human visual system and the lack of connection between chromatic data and distortion have been the major bottlenecks in developing image quality assessment. To address this issue, we develop a new no-reference (NR) image quality assessment (IQA) metric that accounts for the impact of pre-attention and spatial dependency on the perceived quality of distorted images. The resulting model, dubbed the Pre-attention and Spatial-dependency driven Quality Assessment (PSQA) predictor, introduces the pre-attention theory to emulate early phase visual perception by refining luminance-channel data. Chromatic data are also processed concurrently by transforming images from RGB to the perceptually optimized SCIELAB color space. Considering that the gray-tone spatial dependency matrix conveys important texture properties that are closely related to visual quality, this matrix, as a mathematical solution for subsequent visual process emulation, is calculated along with its statistical features on both gray and color channels. To clarify the influence of different regression procedures on model output, support vector regression and AdaBoosting Back Propagation (BP) neural networks are adopted separately to train the prediction models. We thoroughly evaluated PSQA on four public image quality databases: LIVE, TID2013, CSIQ, and VCL. The experimental results show that PSQA delivers highly competitive performance compared with top-rank NR and full-reference IQA metrics.

A No-Reference Image Quality Assessment Metric by Multiple Characteristics of Light Field Images

Evaluation of light field image (LFI), especially micro-lens camera light field (LF), is a new and challenging work. The development of image quality assessment (IQA) metric of LFIs relies on the subjective quality assessment database. In this paper, we establish a perceptual quality assessment dataset consisting of 240 distorted images from 8 source images with five distortion types. Furthermore, a no-reference IQA metric is proposed by combining 2D and 3D characteristics of LFI with the Support Vector Regression (SVR) model. The performance of the proposed metric is demonstrated by comparing with some classical full reference IQA metrics both on the presented dataset and a third-party dataset. The experiment results show that our method has a better performance than others.

Full-Reference 3-D Video Quality Assessment Using Scene Component Statistical Dependencies

In this letter, we present a full-reference (FR) quality assessment algorithm to assess the perceptual quality of natural stereoscopic three-dimensional (S3-D) videos. Toward the end, we rely on an empirical model for the joint statistics of motion and depth subband coefficients of an S3-D video frame. Specifically, we use the recently proposed bivariate generalized Gaussian distribution (BGGD) model for the joint statistics. In this letter, we show that the coherence of the covariance matrix of the BGGD varies in proportion with the perceptual video quality. We compute the coherence scores from the eigenvalues of the covariance matrix to estimate the amount of directional dependency between the motion and depth components. To estimate the overall spatial quality score, we apply off-the-shelf 2-D FR image quality assessment metrics on a frame-by-frame basis on both the views and average the frame wise scores. Finally, we pool the coherence and spatial quality scores to derive the overall quality for the S3-D video. An evaluation of the proposed algorithm on the IRCCYN, WaterlooIVC Phase I, and LFOVIA S3D video databases demonstrates its robust performance. The proposed algorithm is called depth- and motion-based 3-D video quality evaluator.

A Prediction Backed Model for Quality Assessment of Screen Content and 3-D Synthesized Images

In this paper, we address problems associated with free-energy-principle-based image quality assessment (IQA) algorithms for objectively assessing the quality of Screen Content (SC) and three-dimensional (3-D) synthesized images and also propose a very fast and efficient IQA algorithm to address these issues. These algorithms separate an image into predicted and disorder residual parts and assume disorder residual part does not contribute much to the overall perceptual quality. These algorithms fail for quality estimation of SC images as information of textual regions in SC images are largely separated into the disorder residual part and less information in the predicted part and subsequently, given a negligible emphasis. However, this is in contrast with the characteristics of human vision. Since our eyes are well trained to detect text in daily life. So, our human vision has prior information about text regions and can sense small distortions in these regions. In this paper, we proposed a new reduced-reference IQA algorithm for SC images based upon a more perceptually relevant prediction model and distortion categorization, which overcomes problems with existing free-energy-principle-based predictors. From experiments, it is validated that the proposed model has a better capability of efficiently estimating the quality of SC images as compared to the recently developed reduced-reference IQA algorithms. We also applied the proposed algorithm to judge the quality of 3-D synthesized images and observed that it even achieves better performance than the full-reference IQA metrics specifically designed for the 3-D synthesized views.

Knowledge-based Taxonomic Scheme for Full-Reference Objective Image Quality Measurement Models

In this paper, we propose a knowledge-based taxonomic scheme of the objective image quality assessment metrics including the key concepts involved for each approach. Our classification is constructed according to six criteria based on the information available at each stage of the design process. The novelty of the present classification scheme is that the six layers are linked via a single concept where each layer represents a single type of knowledge about: 1) the reference image, 2) the degradation type, 3) the visual perception field, 4) the human visual physiology and psychophysical mechanisms, 5) the processes of the visual information analysis, and finally 6) knowledge about perceptual image representation and coding. The first layer helps delineate boundaries between full-reference image quality assessment metrics, that are further classified trough layers 2 to 6, and other families (reduced-reference and no-reference). In addition, gradual degrees are considered for knowledge about specific areas related to visual quality evaluation processes. The proposed taxonomic framework is intended to be stepwise, to help sorting out the fundamental ideas behind the development of objective image quality metrics often working on the luminance channel or marginally on the RGB channels. The aim is to congregate the already published classification schemes and to methodologically expand new aspects according to which an efficient and straightforward classification of the image quality assessment algorithms becomes possible. This is significant because of the increasing number of developed metrics. Furthermore, a systematic summarisation is necessary in order to facilitate the search and application of image quality techniques.

Blind Quality Metric for Multidistortion Images Based on Cartoon and Texture Decomposition

In this letter, a no-reference (NR) hybrid image quality assessment (IQA) metric based on cartoon-texture decomposition (CTD) is presented. Focusing on images distorted by both blur and noise, the method takes properties of CTD to separate image into cartoon part with salient edges and texture part with noises. Then, the blur degree and noise level can be estimated separately from different image parts, combine with a joint effect prediction between blur and noise distortions, we present a cartoon-texture decomposition-based blind metric (CTDBBM). Comparative studies with classical full-reference IQA metrics and state-of-the-art NR metrics are conducted on multidistortion image database: LIVEMD. Experimental results show that the CTDBBM performs well and has a high consistency with the human opinions given in the database.

Correlation between subjective and objective assessment of magnetic resonance (MR) images

Medical Image Quality Assessment (IQA) plays an important role in assisting and evaluating the development of any new hardware, imaging sequences, pre-processing or post-processing algorithms. We have performed a quantitative analysis of the correlation between subjective and objective Full Reference - IQA (FR-IQA) on Magnetic Resonance (MR) images of the human brain, spine, knee and abdomen. We have created a MR image database that consists of 25 original reference images and 750 distorted images. The reference images were distorted with six types of distortions: Rician Noise, Gaussian White Noise, Gaussian Blur, DCT compression, JPEG compression and JPEG2000 compression, at various levels of distortion. Twenty eight subjects were chosen to evaluate the images resulting in a total of 21,700 human evaluations. The raw scores were then converted to Difference Mean Opinion Score (DMOS). Thirteen objective FR-IQA metrics were used to determine the validity of the subjective DMOS. The results indicate a high correlation between the subjective and objective assessment of the MR images. The Noise Quality Measurement (NQM) has the highest correlation with DMOS, where the mean Pearson Linear Correlation Coefficient (PLCC) and Spearman Rank Order Correlation Coefficient (SROCC) are 0.936 and 0.938 respectively. The Universal Quality Index (UQI) has the lowest correlation with DMOS, where the mean PLCC and SROCC are 0.807 and 0.815 respectively. Student's T-test was used to find the difference in performance of FR-IQA across different types of distortion. The superior IQAs tested statistically are UQI for Rician noise images, Visual Information Fidelity (VIF) for Gaussian blur images, NQM for both DCT and JPEG compressed images, Peak Signal-to-Noise Ratio (PSNR) for JPEG2000 compressed images.

Blind image quality assessment using statistical independence in the divisive normalization transform domain

We present a general purpose blind image quality assessment (IQA) method using the statistical independence hidden in the joint distributions of divisive normalization transform (DNT) representations for natural images. The DNT simulates the redundancy reduction process of the human visual system and has good statistical independence for natural undistorted images; meanwhile, this statistical independence changes as the images suffer from distortion. Inspired by this, we investigate the changes in statistical independence between neighboring DNT outputs across the space and scale for distorted images and propose an independence uncertainty index as a blind IQA (BIQA) feature to measure the image changes. The extracted features are then fed into a regression model to predict the image quality. The proposed BIQA metric is called statistical independence (STAIND). We evaluated STAIND on five public databases: LIVE, CSIQ, TID2013, IRCCyN/IVC Art IQA, and intentionally blurred background images. The performances are relatively high for both single- and cross-database experiments. When compared with the state-of-the-art BIQA algorithms, as well as representative full-reference IQA metrics, such as SSIM, STAIND shows fairly good performance in terms of quality prediction accuracy, stability, robustness, and computational costs.

Full-Reference Image Quality Metrics Performance Evaluation Over Image Quality Databases

A quantitative predictive performance evaluation of 18 well-known and commonly used full-reference image quality assessment metrics has been conducted in the present work. The process has been run over six publicly available and subjectively rated image quality databases for four degradation types namely JPEG and JPEG2000 compression, noise and Gaussian blur. Results show that the existing predictive performance evaluation tools of the different full-reference image quality metrics are significantly impacted by the choice of the image quality database. Three of them, namely Toyama, LIVE and TID, have been found to give different assessment results. The visual information fidelity (VIF) quality metric has been found to have superior predictive capabilities to its counterparts. MS-SSIM (multi-scale structural similarity index), MSSIM (modified SSIM) and VIFP (pixel-based VIF) have also closer performances in terms of their correlation to the subjective human ratings, accuracy and monotonicity to the VIF model.