Blind Image Quality Assessment Metrics Research Articles

Blind cartoon image quality assessment based on local structure and chromatic statistics

With the help of computer-assisted systems, cartoon synthesis has become convenient and efficient by reusing existing cartoon materials. However, quality evaluation of the obtained cartoon image still relies on labor-intensive subjective judgment. This accordingly raises an urgent demand for effective quality evaluation methods to automatically select a cartoon image of high quality from a set of candidates with different parameter settings. In this paper, a new blind image quality assessment metric is developed for evaluating the perceptual quality of cartoon images by considering structure and chromatic distortions. The extracted gradient-based local structure features and multiscale chromatic statistical features are integrated into one representation for an overall perceptual quality prediction. Experimental results on two benchmark cartoon image datasets, i.e., NBU-CIQAD and HFUT-CID, indicate that the proposed metric outperforms both the state-of-the-art blind quality evaluation methods designed for natural images or synthetic images.

Blind CT Image Quality Assessment Using DDPM-derived Content and Transformer-based Evaluator.

Lowering radiation dose per view and utilizing sparse views per scan are two common CT scan modes, albeit often leading to distorted images characterized by noise and streak artifacts. Blind image quality assessment (BIQA) strives to evaluate perceptual quality in alignment with what radiologists perceive, which plays an important role in advancing low-dose CT reconstruction techniques. An intriguing direction involves developing BIQA methods that mimic the operational characteristic of the human visual system (HVS). The internal generative mechanism (IGM) theory reveals that the HVS actively deduces primary content to enhance comprehension. In this study, we introduce an innovative BIQA metric that emulates the active inference process of IGM. Initially, an active inference module, implemented as a denoising diffusion probabilistic model (DDPM), is constructed to anticipate the primary content. Then, the dissimilarity map is derived by assessing the interrelation between the distorted image and its primary content. Subsequently, the distorted image and dissimilarity map are combined into a multi-channel image, which is inputted into a transformer-based image quality evaluator. By leveraging the DDPM-derived primary content, our approach achieves competitive performance on a low-dose CT dataset.

A dual-stream hybrid model for blind image quality assessment

Blind image quality assessment (BIQA) is a fundamental task in computer vision. Humans can evaluate image quality from local and global aspects without information on reference images. Inspired by this, we propose a BIQA method named DS-IQA by mimicking human visual system (HVS). A dual-stream hybrid module is established to get dual-stream quality-aware features. A CNN branch is used to mimic the active inference process of HVS to extract local quality-aware features. An enhanced Transformer-branch is used to extract global quality-aware features by modeling nonlocal relations of image patches. Finally, a quality evaluator based on Transformer layers is developed to map the dual-stream features and output the final quality score. The proposed approach is evaluated on five databases. The PLCC of DS-IQA reaches 0.975, 0.938, and 0.963 respectively on synthetic databases (LIVE, TID2013, CSIQ), and individual distortion experimental on TID2013 shows that DS-IQA outperforms in 8 of the 24 distortion categories on TID2013. On authentic databases (LIVEC, KonIQ-10k), the PLCC of DS-IQA reaches 0.887, 0.918, experiments show the superiority of the proposed method over other state-of-the-art BIQA metrics.

Bridge the gap between full-reference and no-reference: A totally full-reference induced blind image quality assessment via deep neural networks

Blind image quality assessment (BIQA) is of fundamental importance in low-level computer vision community. Increasing interest has been drawn in exploiting deep neural networks for BIQA. Despite of the notable success achieved, there is a broad consensus that training deep convolutional neural networks (DCNN) heavily relies on massive annotated data. Unfortunately, BIQA is typically a small sample problem, resulting the generalization ability of BIQA severely restricted. In order to improve the accuracy and generalization ability of BIQA metrics, this work proposed a totally opinion-unaware BIQA in which no subjective annotations are involved in the training stage. Multiple full-reference image quality assessment (FR-IQA) metrics are employed to label the distorted image as a substitution of subjective quality annotation. A deep neural network (DNN) is trained to blindly predict the multiple FR-IQA score in absence of corresponding pristine image. In the end, a self-supervised FR-IQA score aggregator implemented by adversarial auto-encoder pools the predictions of multiple FR-IQA scores into the final quality predicting score. Even though none of subjective scores are involved in the training stage, experimental results indicate that our proposed full reference induced BIQA framework is as competitive as state-of-the-art BIQA metrics.

Deep Neural Network for Blind Visual Quality Assessment of 4K Content

The 4K content can deliver a more immersive visual experience to consumers due to the huge improvement in spatial resolution. However, the high spatial resolution brings a great challenge for video transmission and storage. Therefore, it is necessary to compress or downscale the 4K content before transmitting it to end-users. Existing blind image quality assessment (BIQA) methods are not suitable for 4K contents due to the high spatial resolution and specific distortions caused by upscaling methods. In this paper, we propose a deep learning-based BIQA model for 4K content, aiming to recognize true and pseudo 4K content and meanwhile evaluate their visual quality. Considering the characteristic that high spatial resolution can represent more abundant high-frequency information, we first propose a Grey-level Co-occurrence Matrix (GLCM) based texture complexity measure to select three representative image patches from a 4K image, which can reduce the computational complexity and is proven to be very effective for the overall quality prediction. Then, we extract various visual features from the intermediate layers of the convolutional neural network (CNN) and integrate them into the quality-aware feature representation. Finally, two multilayer perception (MLP) networks are utilized to map the quality-aware features into the class probability and the quality score of each patch respectively. The overall quality index is obtained through averaging the results of all patches. The proposed model is trained via the multi-task learning manner and the uncertainty principle is introduced to balance the losses of the classification and regression tasks. The experimental results show that the proposed model outperforms all compared BIQA metrics on four 4K content quality assessment databases.

Blind Image Quality Assessment for Authentic Distortions by Intermediary Enhancement and Iterative Training

With the boom of deep neural networks, blind image quality assessment (BIQA) has achieved great processes. However, the current BIQA metrics are limited when evaluating low-quality images as compared to medium-quality and high-quality images, which restricts their applications in real world problems. In this paper, we first identify that two challenges caused by distribution shift and long-tailed distribution lead to the compromised performance on low-quality images. Then, we propose an intermediary enhancement-based bilateral network with iterative training strategy for solving these two challenges. Drawing on the experience of transitive transfer learning, the proposed metric adaptively introduces enhanced intermediary images to transfer more information to low-quality images for mitigating the distribution shift. Our metric also adopts an iterative training strategy to deal with the long-tailed distribution. This strategy decouples feature extraction and score regression for better representation learning and regressor training. It not only transfers the knowledge learned from the earlier stage to the latter stage, but also makes the model pay more attention to long-tailed low-quality images. We conduct extensive experiments on five authentically distorted image quality datasets. The results show that our metric significantly improves the evaluating performance on low-quality images and delivers state-of-the-art intra-dataset results. During generalization tests, our metric also achieves the best cross-dataset performance.

Application of blind image quality assessment metrics to pulsed thermography

ABSTRACT This paper explores the application of Blind Image Quality Assessment (BIQA) metrics to pulsed thermography. Two BIQA were used to subsample a sequence of images acquired using Pulse Thermography (PT). The experiments show that the sequences subsampled using BIQA significantly improve the results when applied to metallic samples but fail to capture informative features on composite materials. On metallic samples, when the PCT is applied, an average improvement of 139% of the Contrast to Noise Ratio (CNR) score is observed, on the sequences subsampled, compared with the entire sequence. Nonetheless, the CNR shows that processing the entire data sequence offers 240% improvement compared with subsampled sequences on composite materials.

Weighted differential evolution algorithm based pansharpening

ABSTRACT Imaging Satellites acquire multispectral images, MIs, in low resolution, LR, and panchromatic images, PANs, in high resolution, HR, due to some advantages provided in satellite design. The pansharpening, PS, is a super resolution image synthesis method that is used to generate the pansharpened image, PI, by fusion of PAN and MI. The use of PS process is unavoidable in applications such as efficient use of communication-bandwidth of imaging satellites and fusion of images derived from various image sensors. The PS process is a multi-step process consisting of various complex image processing stages, such as registration, resampling, synthesis, and fusion. The Weighted Differential Evolution Algorithm-based PS method, WDEPS, has been proposed in this paper. The WDEPS uses WDE to synthesize the intensity image, which is the blended-image of MI -bands. WDE has been used to compute the relevant image-blending weights, efficiently. In the experiments, several satellite images (QuickBird-2, Ikonos-2, and GeoEye-1) with different spatial resolutions were used. The WDEPS’s experimental results have been compared with 17 well-known PS methods by using 9 full-reference and 3 blind image quality assessment metrics. The experimental results exposed that WDEPS generates high-quality PIs than traditional PS methods used in the experimental aspect of qualitative and quantitative assessment.

BGT: A blind image quality evaluator via gradient and texture statistical features

Blind image quality assessment (BIQA) aims to design a model that can accurately evaluate the quality of the distorted image without any information about its reference image. Previous studies have shown that gradients and textures of image is widely used in image quality evaluation tasks. However, few studies used the joint statistics of gradient and texture information to evaluate image quality. Considering the visual perception characteristics of the human visual system, we develop a novel general-purpose BIQA model via two sets of complementary perception features. Specifically, the joint statistical histograms of gradient and texture are extracted as the first set of features, and the second set of features is extracted using the local binary pattern (LBP) operator. After extracting two groups of complementary quality-aware features, the feature vectors are sent to the support vector regression machine to establish the nonlinear relationship between quality-aware features and quality scores. A large number of experiments on seven large benchmark databases show that the proposed BIQA model has higher accuracy, better generalization properties and lower computational complexity than the relevant state-of-the-art BIQA metrics.

Blind Image Quality Assessment With Active Inference.

Blind image quality assessment (BIQA) is a useful but challenging task. It is a promising idea to design BIQA methods by mimicking the working mechanism of human visual system (HVS). The internal generative mechanism (IGM) indicates that the HVS actively infers the primary content (i.e., meaningful information) of an image for better understanding. Inspired by that, this paper presents a novel BIQA metric by mimicking the active inference process of IGM. Firstly, an active inference module based on the generative adversarial network (GAN) is established to predict the primary content, in which the semantic similarity and the structural dissimilarity (i.e., semantic consistency and structural completeness) are both considered during the optimization. Then, the image quality is measured on the basis of its primary content. Generally, the image quality is highly related to three aspects, i.e., the scene information (content-dependency), the distortion type (distortion-dependency), and the content degradation (degradation-dependency). According to the correlation between the distorted image and its primary content, the three aspects are analyzed and calculated respectively with a multi-stream convolutional neural network (CNN) based quality evaluator. As a result, with the help of the primary content obtained from the active inference and the comprehensive quality degradation measurement from the multi-stream CNN, our method achieves competitive performance on five popular IQA databases. Especially in cross-database evaluations, our method achieves significant improvements.

No Reference Quality Assessment for Screen Content Images Using Stacked Autoencoders in Pictorial and Textual Regions.

Recently, the visual quality evaluation of screen content images (SCIs) has become an important and timely emerging research theme. This article presents an effective and novel blind quality evaluation metric for SCIs by using stacked autoencoders (SAE) based on pictorial and textual regions. Since the SCI consists of not only the pictorial area but also the textual area, the human visual system (HVS) is not equally sensitive to their different distortion types. First, the textual and pictorial regions can be obtained by dividing an input SCI via an SCI segmentation metric. Next, we extract quality-aware features from the textual region and pictorial region, respectively. Then, two different SAEs are trained via an unsupervised approach for quality-aware features that are extracted from these two regions. After the training procedure of the SAEs, the quality-aware features can evolve into more discriminative and meaningful features. Subsequently, the evolved features and their corresponding subjective scores are input into two regressors for training. Each regressor can obtain one output predictive score. Finally, the final perceptual quality score of a test SCI is computed by these two predicted scores via a weighted model. Experimental results on two public SCI-oriented databases have revealed that the proposed scheme can compare favorably with the existing blind image quality assessment metrics.

A Highly Efficient Blind Image Quality Assessment Metric of 3-D Synthesized Images Using Outlier Detection

With multitudes of image processing applications, image quality assessment (IQA) has become a prerequisite for obtaining maximally distinctive statistics from images. Despite the widespread research in this domain over several years, existing IQA algorithms have a number of key limitations concerning different image distortion types and algorithms’ computational efficiency. Images that are synthesized using depth image-based rendering have applications in various disciplines, such as free viewpoint videos, which enable synthesis of novel realistic images in the referenceless environment. In the literature, very few no-reference (NR) quality assessment metrics of three-dimensional (3-D) synthesized images are proposed, and most of them are computationally expensive, which makes it difficult for them to be deployed in real-time applications. In this paper, we attribute the geometrically distorted pixels as outliers in 3-D synthesized images. This assumption is validated using the three $sigma$ rule-based robust outlyingness ratio. We propose a novel fast and accurate blind IQA metric of 3-D synthesized images using nonlinear median filtering since the median filtering has the capability of identifying and removing outliers. The advantages of the proposed algorithm are twofold. First, it uses a simple technique, i.e., median filtering, to capture the level of geometric and structural distortions (up to some extend). Second, the proposed algorithm has higher computational efficiency. Experiments show the superiority of the proposed NR IQA algorithm over existing state-of-the-art full-, reduced-, and NR IQA methods, in terms of both predicting accuracy and computational complexity.

Blind image quality assessment utilising local mean eigenvalues

Eigenvalues are intrinsic and representative values of a square matrix. They have thus been used in many image processing areas due to their important application value, but not in the image quality assessment (IQA) field. In this Letter, the authors study the correlation between local mean eigenvalues (LMEs) and perceptual quality of images, and demonstrate the applicability of LMEs in IQA. The LMEs are related to structural complexity of images. The LMEs and natural scene statistics features are utilised for a sparse dictionary learning. Experimental results conducted on promising IQA databases show their method's superiority in comparison with top-performing blind IQA metrics.

Blind quality assessment for screen content images by orientation selectivity mechanism

Currently, most objective image quality assessment (IQA) methods designed for screen content images (SCIs) require reference information, and existing blind IQA metrics cannot obtain consistent results with subjective scores. In this study, we propose a novel blind IQA method for SCIs based on orientation selectivity mechanism. First, we extract orientation features to compute the visual distortion of degraded SCIs by orientation selectivity mechanism. The statistical orientation features are further obtained by the histogram of orientation features. Second, the statistical structure features are calculated as the complementary information of orientation features for quality prediction of degraded SCIs. Finally, we employ support vector regression (SVR) as the mapping function from these extracted statistical features to quality scores. Experimental results show that the proposed method can obtain better performance of visual quality prediction for SCIs than other existing related methods.

Blind Image Quality Assessment Based on Rank-Order Regularized Regression

Blind image quality assessment (BIQA) aims to estimate the subjective quality of a query image without access to the reference image. Existing learning-based methods typically train a regression function by minimizing the average error between subjective opinion scores and model predictions. However, minimizing average error does not necessarily lead to correct quality rank-orders between the test images, which is a highly desirable property of image quality models. In this paper, we propose a novel rank-order regularized regression model to address this problem. The key idea is to introduce a pairwise rank-order constraint into the maximum margin regression framework, aiming to better preserve the correct perceptual preference. To the best of our knowledge, this is the first attempt to incorporate rank-order constraints into margin-based quality regression model. By combing with a new local spatial structure feature, we achieve highly consistent quality prediction with human perception. Experimental results show that the proposed method outperforms many state-of-the-art BIQA metrics on popular publicly available IQA databases (i.e., LIVE-II, TID2013, VCL@FER, LIVEMD, and ChallengeDB).

Blind Image Quality Assessment Using Statistical Structural and Luminance Features

Blind image quality assessment (BIQA) aims to develop quantitative measures to automatically and accurately estimate perceptual image quality without any prior information about the reference image. In this paper, we introduce a novel BIQA metric by structural and luminance information, based on the characteristics of human visual perception for distorted image. We extract the perceptual structural features of distorted image by the local binary pattern distribution. Besides, the distribution of normalized luminance magnitudes is extracted to represent the luminance changes in distorted image. After extracting the features for structures and luminance, support vector regression is adopted to model the complex nonlinear relationship from feature space to quality measure. The proposed BIQA model is called no-reference quality assessment using statistical structural and luminance features (NRSL). Extensive experiments conducted on four synthetically distorted image databases and three naturally distorted image databases have demonstrated that the proposed NRSL metric compares favorably with the relevant state-of-the-art BIQA models in terms of high correlation with human subjective ratings. The MATLAB source code and validation results of NRSL are publicly online at http://www.ntu.edu.sg/home/wslin/Publications.htm .

Toward a Blind Deep Quality Evaluator for Stereoscopic Images Based on Monocular and Binocular Interactions.

During recent years, blind image quality assessment (BIQA) has been intensively studied with different machine learning tools. Existing BIQA metrics, however, do not design for stereoscopic images. We believe this problem can be resolved by separating 3D images and capturing the essential attributes of images via deep neural network. In this paper, we propose a blind deep quality evaluator (DQE) for stereoscopic images (denoted by 3D-DQE) based on monocular and binocular interactions. The key technical steps in the proposed 3D-DQE are to train two separate 2D deep neural networks (2D-DNNs) from 2D monocular images and cyclopean images to model the process of monocular and binocular quality predictions, and combine the measured 2D monocular and cyclopean quality scores using different weighting schemes. Experimental results on four public 3D image quality assessment databases demonstrate that in comparison with the existing methods, the devised algorithm achieves high consistent alignment with subjective assessment.

Blind Image Quality Assessment Based on Multichannel Feature Fusion and Label Transfer

In this paper, we propose an efficient blind image quality assessment (BIQA) algorithm, which is characterized by a new feature fusion scheme and a ${k}$ -nearest-neighbor (KNN)-based quality prediction model. Our goal is to predict the perceptual quality of an image without any prior information of its reference image and distortion type. Since the reference image is inaccessible in many applications, the BIQA is quite desirable in this context. In our method, a new feature fusion scheme is first introduced by combining an image’s statistical information from multiple domains (i.e., discrete cosine transform, wavelet, and spatial domains) and multiple color channels (i.e., Y, Cb, and Cr). Then, the predicted image quality is generated from a nonparametric model, which is referred to as the label transfer (LT). Based on the assumption that similar images share similar perceptual qualities, we implement the LT with an image retrieval procedure, where a query image’s KNNs are searched for from some annotated images. The weighted average of the KNN labels (e.g., difference mean opinion score or mean opinion score) is used as the predicted quality score. The proposed method is straightforward and computationally appealing. Experimental results on three publicly available databases (i.e., LIVE II, TID2008, and CSIQ) show that the proposed method is highly consistent with human perception and outperforms many representative BIQA metrics.

Blind image quality assessment using statistical independence in the divisive normalization transform domain

We present a general purpose blind image quality assessment (IQA) method using the statistical independence hidden in the joint distributions of divisive normalization transform (DNT) representations for natural images. The DNT simulates the redundancy reduction process of the human visual system and has good statistical independence for natural undistorted images; meanwhile, this statistical independence changes as the images suffer from distortion. Inspired by this, we investigate the changes in statistical independence between neighboring DNT outputs across the space and scale for distorted images and propose an independence uncertainty index as a blind IQA (BIQA) feature to measure the image changes. The extracted features are then fed into a regression model to predict the image quality. The proposed BIQA metric is called statistical independence (STAIND). We evaluated STAIND on five public databases: LIVE, CSIQ, TID2013, IRCCyN/IVC Art IQA, and intentionally blurred background images. The performances are relatively high for both single- and cross-database experiments. When compared with the state-of-the-art BIQA algorithms, as well as representative full-reference IQA metrics, such as SSIM, STAIND shows fairly good performance in terms of quality prediction accuracy, stability, robustness, and computational costs.

Universal blind image quality assessment metrics via natural scene statistics and multiple kernel learning.

Universal blind image quality assessment (IQA) metrics that can work for various distortions are of great importance for image processing systems, because neither ground truths are available nor the distortion types are aware all the time in practice. Existing state-of-the-art universal blind IQA algorithms are developed based on natural scene statistics (NSS). Although NSS-based metrics obtained promising performance, they have some limitations: 1) they use either the Gaussian scale mixture model or generalized Gaussian density to predict the nonGaussian marginal distribution of wavelet, Gabor, or discrete cosine transform coefficients. The prediction error makes the extracted features unable to reflect the change in nonGaussianity (NG) accurately. The existing algorithms use the joint statistical model and structural similarity to model the local dependency (LD). Although this LD essentially encodes the information redundancy in natural images, these models do not use information divergence to measure the LD. Although the exponential decay characteristic (EDC) represents the property of natural images that large/small wavelet coefficient magnitudes tend to be persistent across scales, which is highly correlated with image degradations, it has not been applied to the universal blind IQA metrics; and 2) all the universal blind IQA metrics use the same similarity measure for different features for learning the universal blind IQA metrics, though these features have different properties. To address the aforementioned problems, we propose to construct new universal blind quality indicators using all the three types of NSS, i.e., the NG, LD, and EDC, and incorporating the heterogeneous property of multiple kernel learning (MKL). By analyzing how different distortions affect these statistical properties, we present two universal blind quality assessment models, NSS global scheme and NSS two-step scheme. In the proposed metrics: 1) we exploit the NG of natural images using the original marginal distribution of wavelet coefficients; 2) we measure correlations between wavelet coefficients using mutual information defined in information theory; 3) we use features of EDC in universal blind image quality prediction directly; and 4) we introduce MKL to measure the similarity of different features using different kernels. Thorough experimental results on the Laboratory for Image and Video Engineering database II and the Tampere Image Database2008 demonstrate that both metrics are in remarkably high consistency with the human perception, and overwhelm representative universal blind algorithms as well as some standard full reference quality indexes for various types of distortions.