Image Quality Assessment Performance Research Articles

Evaluating the Stability of Deep Image Quality Assessment with Respect to Image Scaling

Image quality assessment (IQA) is a fundamental metric for image processing tasks (e.g., compression). With full-reference IQAs, traditional IQAs, such as PSNR and SSIM, have been used. Recently, IQAs based on deep neural networks (deep IQAs), such as LPIPS and DISTS, have also been used. It is known that image scaling is inconsistent among deep IQAs, as some perform down-scaling as pre-processing, whereas others instead use the original image size. In this paper, we show that the image scale is an influential factor that affects deep IQA performance. We comprehensively evaluate four deep IQAs on the same five datasets, and the experimental results show that image scale significantly influences IQA performance. We found that the most appropriate image scale is often neither the default nor the original size, and the choice differs depending on the methods and datasets used. We visualized the stability and found that PieAPP is the most stable among the four deep IQAs.

A novel algorithm for comprehensive quality assessment of clinical magnetic resonance images based on natural scene statistics in spatial domain

BackgroundA magnetic resonance imaging (MRI)-specific objective image quality assessment (IQA) algorithm, the quality evaluation using multidirectional filters for MRI (QEMDIM), was previously reported. QEMDIM requires a set of reference images to calculate the quality score (SQ) for an assessed image. SQ may be affected by the quality of the reference set owing to the calculation procedure. PurposeTo propose a modified version of the IQA algorithm and compare the IQA performance of the original and modified algorithms. AssessmentBrain axial T1- and T2-weighted spin-echo images of varying quality levels (noise and blurring) were acquired from seven healthy men. Subjective IQA (paired comparisons) was conducted on the images, and subjective quality scores were obtained. With reference sets of various quality levels, QEMDIM and modified IQA were applied to the same images that underwent the subjective IQA. The correlation of each SQ and modified score (Smod) with the subjective scores was evaluated for content-related subsets of assessed images and for each reference set. The effect of the reference-set quality on the distribution of the correlation coefficients (CCs) was statistically evaluated for SQ and Smod using a one-way analysis of variance test with a significance level of 0.05. We also evaluated the variation in Smod for images with almost the same qualities using the standard deviation (SD). ResultsThe CCs of SQ varied significantly with the quality of the reference set, whereas that of Smod did not. The SD of Smod for almost-same-quality images was less than that corresponding to the confidence interval of the subjective scores. ConclusionOur modified algorithm was superior to QEMDIM in terms of IQA performance in clinical practice, especially in terms of accuracy, robustness, and reproducibility.

Lung nodule image quality assessment under iterative model reconstruction

Aiming at the problem that lung nodule image quality assessment (IQA) when lung is constructed by a matrix iterative model (IMR), we in this paper proposes a novel IQA model based on multi-modal visual features highly representative to lung nodules. In order to achieve a competitive IQA performance, our proposed model adopted a medical-record-guided deep architecture as a significant part to provide additional information for our lung nodule IQA architecture. In the first place, such significant deep model constructs feature vector for each lung nodule image input by the model based on Bi-LSTM and Seq2Seq. Subsequently, the deep features was fed into to a carefully-trained decoder to reconstruct the original input data. Next, the deep feature associated with the lung nodule images are inputted into an end-to-end IQA deep model based on the 3D CNN directly. Comprehensive experimental results on the medical image sets collected from two hospitals in mainland China have shown that the designed deep model can reflect the subjective quality of lung nodule image accurately, and achieves better overall medical IQA assessment performance than the other well-known and non-reference IQA methods.

MMMNet: An End-to-End Multi-Task Deep Convolution Neural Network With Multi-Scale and Multi-Hierarchy Fusion for Blind Image Quality Assessment

As the evaluation of image quality depends on the human visual system (HVS), many existing image quality assessment (IQA) methods focus on modeling the HVS to account for subjective perception. The visual attention of the HVS makes humans more sensitive to distortion on the attended regions than on regions which are not the focus of attention. Therefore, we propose an end-to-end multi-task deep convolution neural network with multi-scale and multi-hierarchy fusion (MMMNet), in which the IQA and saliency subtasks are jointly optimized to improve saliency-guided IQA performance. Particularly, the incorporation of saliency information is achieved by fusing saliency features with IQA features hierarchically to progressively improve the IQA features over network depth. A multi-scale feature extraction module (MSFE) is proposed to provide effective saliency features for the IQA network. Based on the saliency fusion, MMMNet introduces an auxiliary saliency task, achieving the multi-task learning to improve the generalization of the IQA task. Experimental results show that MMMNet achieves state-of-the-art performance and strong generalization ability on IQA databases.

Interobserver variability in quality assessment of magnetic resonance images

BackgroundThe perceptual quality of magnetic resonance (MR) images influences diagnosis and may compromise the treatment. The purpose of this study was to evaluate how the image quality changes influence the interobserver variability of their assessment.MethodsFor the variability evaluation, a dataset containing distorted MRI images was prepared and then assessed by 31 experienced medical professionals (radiologists). Differences between observers were analyzed using the Fleiss’ kappa. However, since the kappa evaluates the agreement among radiologists taking into account aggregated decisions, a typically employed criterion of the image quality assessment (IQA) performance was used to provide a more thorough analysis. The IQA performance of radiologists was evaluated by comparing the Spearman correlation coefficients, ρ, between individual scores with the mean opinion scores (MOS) composed of the subjective opinions of the remaining professionals.ResultsThe experiments show that there is a significant agreement among radiologists (κ=0.12; 95% confidence interval [CI]: 0.118, 0.121; P<0.001) on the quality of the assessed images. The resulted κ is strongly affected by the subjectivity of the assigned scores, separately presenting close scores. Therefore, the ρ was used to identify poor performance cases and to confirm the consistency of the majority of collected scores (ρmean = 0.5706). The results for interns (ρmean = 0.6868) supports the finding that the quality assessment of MR images can be successfully taught.ConclusionsThe agreement observed among radiologists from different imaging centers confirms the subjectivity of the perception of MR images. It was shown that the image content and severity of distortions affect the IQA. Furthermore, the study highlights the importance of the psychosomatic condition of the observers and their attitude.

Objective image quality assessment based on image color appearance and gradient features

With the rapid development of color image contents and imaging devices in various kinds of multimedia communication systems, conventional grayscale counterparts are replaced by chromatic ones. Under such a transition, the image quality assessment (IQA) model needs to be built by subjective visual measurement, designed in accordance with the results, and applied to the related practical problems. Based on the visual perception characteristics, chromaticity and the structure feature information are quantified, and an objective IQA model combining the color appearance and the gradient image features is proposed in this paper, namely color appearance and gradient similarity(CAGS) model. Two new color appearance indices, vividness and depth, are selected to build the chromatic similarity map. The structure information is characterized by gradient similarity map. Vividness map plays two roles in the proposed model. One is utilized as feature extractor to compute the local quality of distorted image, and the other is as a weight part to reflect the importance of local domain. To quantify the specific parameters of CAGS, Taguchi method is used and four main parameters, i.e., &lt;i&gt;K&lt;/i&gt;&lt;sub&gt;&lt;i&gt;V&lt;/i&gt;&lt;/sub&gt;, &lt;i&gt;K&lt;/i&gt;&lt;sub&gt;&lt;i&gt;D&lt;/i&gt;&lt;/sub&gt;, &lt;i&gt;K&lt;/i&gt;&lt;sub&gt;&lt;i&gt;G&lt;/i&gt;&lt;/sub&gt; and &lt;i&gt;α&lt;/i&gt;, of this model are determined based on the statistical correlation indices. The optimal parameters of CAGS are &lt;i&gt;K&lt;/i&gt;&lt;sub&gt;&lt;i&gt;V&lt;/i&gt;&lt;/sub&gt; = &lt;i&gt;K&lt;/i&gt;&lt;sub&gt;&lt;i&gt;D&lt;/i&gt;&lt;/sub&gt; = 0.02, &lt;i&gt;K&lt;/i&gt;&lt;sub&gt;&lt;i&gt;G&lt;/i&gt;&lt;/sub&gt; = 50, and &lt;i&gt;α&lt;/i&gt; = 0.1. Furthermore, the CAGS is tested by utilizing 94 reference images and 4830 distorted images from the four open image databases (LIVE, CSIQ, TID2013 and IVC). Additionally, the influences of the 35 distortion types on IQA are analyzed. Massive experiments are performed on four publicly available benchmark databases between CAGS and other 10 state-of-the-art and recently published IQA models, for the accuracy, complexity and generalization performance of IQA. The experimental results show that the accuracy PLCC of the CAGS model can achieve 0.8455 at lowest and 0.9640 at most in the four databases, and the results about commonly evaluation criteria prove that the CAGS performs higher consistency with the subjective evaluations. Among the 35 distortion types, the two distortion types, namely contrast change and change of color saturation, CAGS and mostly IQA models have the worst influence on IQA, and the CAGS yields the highest top three rank number. Moreover, the SROCC values of CAGS for other distortion types are all larger than 0.6 and the number of SROCC value larger than 0.95 is 14 times. Besides, the CAGS maintains a moderate computational complexity. These results of test and comparison above show that the CAGS model is effective and feasible, and the corresponding model has an excellent performance.

SPSIM: A Superpixel-Based Similarity Index for Full-Reference Image Quality Assessment.

Full-reference image quality assessment algorithms usually perform comparisons of features extracted from square patches. These patches do not have any visual meanings. On the contrary, a superpixel is a set of image pixels that share similar visual characteristics and is thus perceptually meaningful. Features from superpixels may improve the performance of image quality assessment. Inspired by this, we propose a new superpixel-based similarity index by extracting perceptually meaningful features and revising similarity measures. The proposed method evaluates image quality on the basis of three measurements, namely, superpixel luminance similarity, superpixel chrominance similarity, and pixel gradient similarity. The first two measurements assess the overall visual impression on local images. The third measurement quantifies structural variations. The impact of superpixel-based regional gradient consistency on image quality is also analyzed. Distorted images showing high regional gradient consistency with the corresponding reference images are visually appreciated. Therefore, the three measurements are further revised by incorporating the regional gradient consistency into their computations. A weighting function that indicates superpixel-based texture complexity is utilized in the pooling stage to obtain the final quality score. Experiments on several benchmark databases demonstrate that the proposed method is competitive with the state-of-the-art metrics.

DeepSim: Deep similarity for image quality assessment

This paper studies one interesting problem: how does the deep neural network (DNN) architecture affect the image quality assessment (IQA) performance? In order to find the answer, we propose a novel full-reference IQA framework, codenamed deep similarity (DeepSim). In DeepSim, we first measure the local similarities between the features (produced by a DNN model) of the test image and those of the reference image; Afterwards, the local quality indices are gradually pooled together to estimate the overall quality score. In addition, various factors that may affect the IQA performance are investigated. Thorough experiments conducted on standard databases show that: (1) DeepSim can accurately predict human perceived image quality and outperforms previous state-of-the-art; (2) mid-level representations are most effective for quality prediction; and (3) preprocessing, the restricted linear units and max-pooling operations are beneficial for the IQA performance.

Analysis of Distortion Distribution for Pooling in Image Quality Prediction

Image quality assessment (IQA) has been an active research area during last decades. Many existing objective IQA models share a similar two-step structure with measuring local distortion before pooling. Compared with the rapid development for local distortion measurement, seldom effort has been made dedicated to effective pooling schemes. In this paper, we design a new pooling model via the analysis of distortion distribution affected by image content and distortion. That is, distributions of distortion position, distortion intensity, frequency changes, and histogram changes are comprehensively considered to infer an overall quality score. Experimental results conducted on four large-scale image quality databases (LIVE, TID2008, CSIQ, and CCID2014) concluded with three valuable findings. First, the proposed technique leads to consistent improvement in the IQA performance for studied local distortion measures. Second, relative to the traditional pooling, the performance gain of our algorithm is beyond 15% on average. Third, the best overall performance made by the proposed strategy outperforms state-of-the-art competitors.

Information-based reduced reference image quality assessment incorporating non-tensor product wavelet filter banks

To achieve the better performance of image quality assessment, this paper develops an informationbased scheme. First, visual maps that contain visual information are extracted by non-tensor product wavelet filter banks at different scales. Subsequently, to model visual difference (VD) caused by image degradations, the variation of information entropy between distorted and reference images is employed. Then the final visual quality is calculated with the VDs at each scale. In experiments, evaluation results show that it performs very well in terms of high accuracy of image quality measurement as well as low data rate.