Blind Image Quality Assessment Methods Research Articles

Blind CT Image Quality Assessment Using DDPM-derived Content and Transformer-based Evaluator.

Lowering radiation dose per view and utilizing sparse views per scan are two common CT scan modes, albeit often leading to distorted images characterized by noise and streak artifacts. Blind image quality assessment (BIQA) strives to evaluate perceptual quality in alignment with what radiologists perceive, which plays an important role in advancing low-dose CT reconstruction techniques. An intriguing direction involves developing BIQA methods that mimic the operational characteristic of the human visual system (HVS). The internal generative mechanism (IGM) theory reveals that the HVS actively deduces primary content to enhance comprehension. In this study, we introduce an innovative BIQA metric that emulates the active inference process of IGM. Initially, an active inference module, implemented as a denoising diffusion probabilistic model (DDPM), is constructed to anticipate the primary content. Then, the dissimilarity map is derived by assessing the interrelation between the distorted image and its primary content. Subsequently, the distorted image and dissimilarity map are combined into a multi-channel image, which is inputted into a transformer-based image quality evaluator. By leveraging the DDPM-derived primary content, our approach achieves competitive performance on a low-dose CT dataset.

Progressive Bidirectional Feature Extraction and Enhancement Network for Quality Evaluation of Night-Time Images

Blind image quality assessment (BIQA) has received increasing attention in the past decades. However, it still remains inadequately researched on BIQA for night-time images suffering from the diverse authentic degradations. Since the intrinsic content degradations of night-time images are highly related to the illumination, how to use the connection between content and illumination to enhance the feature representation ability is the key issue in designing BIQA methods for night-time images. In this paper, we first construct an ultra-high-definition night-time image dataset (UHD-NID) with high image resolution and abundant parameter settings. UHD-NID contains 1600 images with a high resolution of 5616×3744, and each group of images contains ten exposure levels. Then, we conduct subjective assessment and analyze the subjective data to obtain a mean opinion score to each image in UHD-NID. To enhance the feature representation ability in content and illumination, we propose a progressive bidirectional feature extraction and enhancement network (PBFEE-Net). In addition, we use a decomposition network to decompose the input image into the reflectance and illumination, which can facilitate the ability of feature extraction to some extent. The experimental results show that our proposed method achieves superior performance in evaluating the quality of night-time images. The dataset and code will be released at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/NBUsjl/UHD-NID</uri> .

Unsupervised blind image quality assessment via joint spatial and transform features

A novel unsupervised blind image quality assessment (BIQA) method, which requires no mean opinion scores for model training is presented in this paper. The method employs joint spatial and transform features as quality degradation metrics, specifically, phase congruency, gradient magnitude (GM), and GM and Laplacian of Gaussian response and local normalized coefficient are extracted as spatial features, and Karhunen–Loéve transform coefficient and discrete cosine transform coefficient are modeled as transform features. Both spatial and transform features are well analyzed to remove the redundancy, and then fitted to the multivariate Gaussian model for no-reference image quality assessment. Extensive experiments conducted on seven IQA databases demonstrate the superiority of the proposed method over the state-of-the-art both supervised and unsupervised BIQA methods.

Data-Efficient Image Quality Assessment with Attention-Panel Decoder

Blind Image Quality Assessment (BIQA) is a fundamental task in computer vision, which however remains unresolved due to the complex distortion conditions and diversified image contents. To confront this challenge, we in this paper propose a novel BIQA pipeline based on the Transformer architecture, which achieves an efficient quality-aware feature representation with much fewer data. More specifically, we consider the traditional fine-tuning in BIQA as an interpretation of the pre-trained model. In this way, we further introduce a Transformer decoder to refine the perceptual information of the CLS token from different perspectives. This enables our model to establish the quality-aware feature manifold efficiently while attaining a strong generalization capability. Meanwhile, inspired by the subjective evaluation behaviors of human, we introduce a novel attention panel mechanism, which improves the model performance and reduces the prediction uncertainty simultaneously. The proposed BIQA method maintains a light-weight design with only one layer of the decoder, yet extensive experiments on eight standard BIQA datasets (both synthetic and authentic) demonstrate its superior performance to the state-of-the-art BIQA methods, i.e., achieving the SRCC values of 0.875 (vs. 0.859 in LIVEC) and 0.980 (vs. 0.969 in LIVE). Checkpoints, logs and code will be available at https://github.com/narthchin/DEIQT.

Progress in Blind Image Quality Assessment: A Brief Review

As a fundamental research problem, blind image quality assessment (BIQA) has attracted increasing interest in recent years. Although great progress has been made, BIQA still remains a challenge. To better understand the research progress and challenges in this field, we review BIQA methods in this paper. First, we introduce the BIQA problem definition and related methods. Second, we provide a detailed review of the existing BIQA methods in terms of representative hand-crafted features, learning-based features and quality regressors for two-stage methods, as well as one-stage DNN models with various architectures. Moreover, we also present and analyze the performance of competing BIQA methods on six public IQA datasets. Finally, we conclude our paper with possible future research directions based on a performance analysis of the BIQA methods. This review will provide valuable references for researchers interested in the BIQA problem.

FQA-Net: an efficient neural network for blind image quality assessment

Guided by the neural network (NN), NN-based blind image quality assessment (BIQA) methods have improved image quality prediction accuracy. However, the training process of NN will consume a lot of time and training data. To solve the above problems, an efficient NN structure, which is called fast quality assessment network (FQA-Net), is proposed for BIQA. FQA-Net is a very simple NN, which mainly includes a convolution layer, a standard deviation measurement layer, and a regression layer. The convolution kernels in the convolution layer are a set of visual filters with characteristics of visual neurons, which are obtained by training on natural image samples. The output of the convolutional layer is a set of feature maps. Then the standard deviation of each feature map is calculated directly. Finally, a regression model is used for the mapping between the standard deviation values and the quality scores. FQA-Net not only reduces the number of parameters and the output dimensions in the training process, but also prevents network overfitting effectively. The results on seven image databases (i.e., LIVE, CSIQ, TID2013, LIVEMD, KADID-10K, LIVEC, and KonIQ-10K) show that FQA-Net has relatively low computational complexity and high computational speed and accuracy compared with other leading BIQA methods, especially when the number of training samples is very low. Moreover, the FQA-Net can provide stable prediction results across different distortion types.

Unsupervised blind image quality assessment based on joint structure and natural scene statistics features

Compared with the widely used supervised blind image quality assessment (BIQA) models, unsupervised BIQA models require little prior knowledge for calculating the objective quality scores of distorted images. In this paper, we propose an unsupervised BIQA method that aims to achieve both good performance and generalization capability with low computational complexity. Carefully selected and extensive structure and natural scene statistics (NSS) features can better represent image quality. First, we employ phase congruency (PC) and finely selected gradient magnitude map and Laplacian of Gaussian response (GM-LOG) features to represent image structure information. Second, we calculate the local mean-subtracted and contrast-normalized (MSCN) coefficients and the Karhunen–Loéve transform (KLT) coefficients to represent the naturalness of the distorted images. Last, multivariate Gaussian (MVG) model with joint features extracted from both the pristine images and the distorted images is adopted to calculate the objective image quality. Extensive experiments conducted on nine IQA databases demonstrate that the proposed method achieves better performance than the state-of-the-art BIQA methods.

Fine-Grained Image Quality Caption With Hierarchical Semantics Degradation.

Blind image quality assessment (BIQA), which is capable of precisely and automatically estimating human perceived image quality with no pristine image for comparison, attracts extensive attention and is of wide applications. Recently, many existing BIQA methods commonly represent image quality with a quantitative value, which is inconsistent with human cognition. Generally, human beings are good at perceiving image quality in terms of semantic description rather than quantitative value. Moreover, cognition is a needs-oriented task where humans are able to extract image contents with local to global semantics as they need. The mediocre quality value represents coarse or holistic image quality and fails to reflect degradation on hierarchical semantics. In this paper, to comply with human cognition, a novel quality caption model is inventively proposed to measure fine-grained image quality with hierarchical semantics degradation. Research on human visual system indicates there are hierarchy and reverse hierarchy correlations between hierarchical semantics. Meanwhile, empirical evidence shows that there are also bi-directional degradation dependencies between them. Thus, a novel bi-directional relationship-based network (BDRNet) is proposed for semantics degradation description, through adaptively exploring those correlations and degradation dependencies in a bi-directional manner. Extensive experiments demonstrate that our method outperforms the state-of-the-arts in terms of both evaluation performance and generalization ability.

SPIQ: A Self-Supervised Pre-Trained Model for Image Quality Assessment

Blind image quality assessment (BIQA) has witnessed a flourishing progress due to the rapid advances in deep learning technique. The vast majority of prior BIQA methods try to leverage models pre-trained on ImageNet to mitigate the data shortage problem. These well-trained models, however, can be sub-optimal when applied to BIQA task that varies considerably from the image classification domain. To address this issue, we make the first attempt to leverage the plentiful unlabeled data to conduct self-supervised pre-training for BIQA task. Based on the distorted images generated from the high-quality samples using the designed distortion augmentation strategy, the proposed pre-training is implemented by a feature representation prediction task. Specifically, patch-wise feature representations corresponding to a certain grid are integrated to make prediction for the representation of the patch below it. The prediction quality is then evaluated using a contrastive loss to capture quality-aware information for BIQA task. Experimental results conducted on KADID-10 k and KonIQ-10 k databases demonstrate that the learned pre-trained model can significantly benefit the existing learning based IQA models.

Blind quality assessment of night-time image

High-quality night-time imaging is crucial to video surveillance, automatic drive and consumer electronics. However, different from day-time imaging, night-time imaging suffers from some disadvantages, such as low light, uneven illumination, difficult focusing, etc., which raises a great concern to the night-time imaging quality. Accordingly, a practical night-time image quality evaluation method is very promising to control and improve the night-time imaging system. Toward this end, in this paper, we propose a blind image quality assessment (BIQA) method to quantify the night-time image quality. Specifically, in the proposed method, we measure the night-time image quality by investigating the fundamental image properties, which are highly relevant to the image quality, such as the brightness, saturation, sharpness, noiseness, contrast and the semantics. Specific features are designed to characterize the image properties properly. Then we employ the support vector regression (SVR) method to infer the image quality with the extracted quality-aware features. The proposed BIQA method for night-time images is thoroughly evaluated on a representative night-time image database. Experimental results demonstrate that the proposed BIQA method for night-time images achieves superior prediction performance to other state-of-the-art BIQA methods.

Blind Image Quality Assessment With Active Inference.

Blind image quality assessment (BIQA) is a useful but challenging task. It is a promising idea to design BIQA methods by mimicking the working mechanism of human visual system (HVS). The internal generative mechanism (IGM) indicates that the HVS actively infers the primary content (i.e., meaningful information) of an image for better understanding. Inspired by that, this paper presents a novel BIQA metric by mimicking the active inference process of IGM. Firstly, an active inference module based on the generative adversarial network (GAN) is established to predict the primary content, in which the semantic similarity and the structural dissimilarity (i.e., semantic consistency and structural completeness) are both considered during the optimization. Then, the image quality is measured on the basis of its primary content. Generally, the image quality is highly related to three aspects, i.e., the scene information (content-dependency), the distortion type (distortion-dependency), and the content degradation (degradation-dependency). According to the correlation between the distorted image and its primary content, the three aspects are analyzed and calculated respectively with a multi-stream convolutional neural network (CNN) based quality evaluator. As a result, with the help of the primary content obtained from the active inference and the comprehensive quality degradation measurement from the multi-stream CNN, our method achieves competitive performance on five popular IQA databases. Especially in cross-database evaluations, our method achieves significant improvements.

Viewport Perception Based Blind Stereoscopic Omnidirectional Image Quality Assessment

Compared with traditional 2D images, stereoscopic omnidirectional images (SOIs) usually have more complex perceptual factors due to the particularities of imaging and display, making the objective quality assessment of SOIs challenging. In this paper, we construct a large and diverse subjective SOIs database named as NBU-SOID for further research demand. And then, we propose a viewport perception based blind SOIs quality assessment (VP-BSOIQA) method by considering the impacts of viewport, user behavior and stereoscopic perception on human visual system, which is mainly composed of binocular perception model (BPM) and omnidirectional perception model (OPM). In the BPM, a binocular combination perception map is generated by the dimension reduction of stereopair and the weighting of binocular energy to reflect the binocular masking effect. In the OPM, several viewports are first created to ensure the consistency of evaluation objects. Then, the intra-viewport and inter-viewport weighting factors are designed with the common influences of visual attention and peripheral vision sensitivity to aggregate the novel multi-orientation structural features extracted from all potential viewports. Experimental results on the NBU-SOID and SOLID databases demonstrate that BPM and OPM can be robustly combined with the existing 2D image quality assessment (IQA) methods, thus averagely achieving 10.2% and 12.2% performance gain in terms of SRCC, respectively. In addition, the proposed VP-BSOIQA method outperforms the state-of-the-art blind IQA methods in predicting the quality of SOIs.

Blind image quality prediction with hierarchical feature aggregation

Blind image quality assessment (BIQA) aims to evaluate the perceptual quality of image with no pristine image for comparison, which attracts extensive attention and is of wide applications. Research on human visual system (HVS) indicates visual perception is classically modeled as a hierarchical process. Meanwhile, empirical evidence shows that different levels of distortion generate individual degradation on hierarchical features. Although previous BIQA methods exploited different levels of feature, the degradations and correlations on hierarchical features are not detailedly analyzed. In this paper, hierarchical degradation is systematically analyzed by quantitative and qualitative experiments using interpretable hierarchical features. Additionally, as there exist strong relationships, these features cannot be simply concatenated for quality prediction. Inspired by the Bayesian brain theory, correlations among hierarchical features are thoroughly deduced with joint probability. Guided by the mathematical derivation, a new feature aggregation network (FANet) is designed to eliminate correlations for hierarchical features fusion. Finally, a novel end-to-end BIQA framework based on hierarchical feature aggregation is proposed, which eminently analyzes the degradations and correlations on hierarchical features. Experimental results on five benchmark databases show that the proposed method achieves state-of-the-art performance. Furthermore, cross-database evaluations demonstrate the generalization capability and robustness of the proposed method.

BM-IQE: An Image Quality Evaluator with Block-Matching for Both Real-Life Scenes and Remote Sensing Scenes.

Like natural images, remote sensing scene images; of which the quality represents the imaging performance of the remote sensor, also suffer from the degradation caused by imaging system. However, current methods measuring the imaging performance in engineering applications require for particular image patterns and lack generality. Therefore, a more universal approach is demanded to assess the imaging performance of remote sensor without constraints of land cover. Due to the fact that existing general-purpose blind image quality assessment (BIQA) methods cannot obtain satisfying results on remote sensing scene images; in this work, we propose a BIQA model of improved performance for natural images as well as remote sensing scene images namely BM-IQE. We employ a novel block-matching strategy called Structural Similarity Block-Matching (SSIM-BM) to match and group similar image patches. In this way, the potential local information among different patches can get expressed; thus, the validity of natural scene statistics (NSS) feature modeling is enhanced. At the same time, we introduce several features to better characterize and express remote sensing images. The NSS features are extracted from each group and the feature vectors are then fitted to a multivariate Gaussian (MVG) model. This MVG model is therefore used against a reference MVG model learned from a corpus of high-quality natural images to produce a basic quality estimation of each patch (centroid of each group). The further quality estimation of each patch is obtained by weighting averaging of its similar patches’ basic quality estimations. The overall quality score of the test image is then computed through average pooling of the patch estimations. Extensive experiments demonstrate that the proposed BM-IQE method can not only outperforms other BIQA methods on remote sensing scene image datasets but also achieve competitive performance on general-purpose natural image datasets as compared to existing state-of-the-art FR/NR-IQA methods.

Blind Image Quality Assessment Based on Multi-scale KLT

Blind image quality assessment (BIQA) plays an important role in image services as independent of the reference image. Herein, the perceptual relevant feature design is the core of BIQA methods, but their performance is still not satisfied at present. In this work, we propose an unsupervised feature extraction approach for BIQA based on Karhunen-Loéve transform (KLT). Specifically, a normalization operation is firstly applied to the test image by calculating its mean subtracted contrast normalized (MSCN) coefficient. Then, KLT is employed as a data-driven feature extraction approach to extract image structural features, wherein kernels with different sizes are utilized to perform multi-scale analysis. Finally, generalized Gaussian distribution (GGD) is employed to model the KLT coefficients distribution in different spectral components as quality relevant features. Extensive experiments conducted on four widely utilized IQA databases have demonstrated that the proposed Multi-scale KLT (MsKLT) BIQA metric compares favorably with existing BIQA methods in terms of high accordance with human subjective scores on both common and uncommon distortion types.

Magnetic Resonance Image Quality Assessment by Using Non-Maximum Suppression and Entropy Analysis.

An investigation of diseases using magnetic resonance (MR) imaging requires automatic image quality assessment methods able to exclude low-quality scans. Such methods can be also employed for an optimization of parameters of imaging systems or evaluation of image processing algorithms. Therefore, in this paper, a novel blind image quality assessment (BIQA) method for the evaluation of MR images is introduced. It is observed that the result of filtering using non-maximum suppression (NMS) strongly depends on the perceptual quality of an input image. Hence, in the method, the image is first processed by the NMS with various levels of acceptable local intensity difference. Then, the quality is efficiently expressed by the entropy of a sequence of extrema numbers obtained with the thresholded NMS. The proposed BIQA approach is compared with ten state-of-the-art techniques on a dataset containing MR images and subjective scores provided by 31 experienced radiologists. The Pearson, Spearman, Kendall correlation coefficients and root mean square error for the method assessing images in the dataset were 0.6741, 0.3540, 0.2428, and 0.5375, respectively. The extensive experimental evaluation of the BIQA methods reveals that the introduced measure outperforms related techniques by a large margin as it correlates better with human scores.

Blind image quality assessment using natural scene statistics of stationary wavelet transform

Image quality assessment (IQA) by human observers, who are the ultimate users, is of utmost importance. Since IQA performed by human observers could be time consuming therefore, computational models are required that can assess the quality of images using objective metrics. Blind image quality assessment (BIQA) methods could predicts the perceived quality of distorted images without requiring information regarding the pristine version of the image. The natural scene statistics (NSS) based BIQA methods predict the quality score by extracting features either in spatial or transform domain. We propose a new BIQA technique that extracts features both in spatial and transform domain. The NSS of stationary wavelet transform (SWT), morphological gradient, and discrete Laplacian are known to be similar to human visual system, and hence can be used for BIQA. Morphological gradient has the advantage that it is easy to generalize for any type of feature space and discrete Laplacian has the property to retain key structural information inherent in the image. SWT provides edge and high frequency information while preserving the scale of the image. We remove the redundancies between the extracted features from spatial and transform domain using an adaptive joint normalization framework. Our proposed BIQA technique is tested on five publicly available databases (i.e., TID2008, TID2013, LIVE, CSIQ, and LIVE from the wild image quality challenge). Our experimental results have shown that our proposed BIQA technique achieves higher accuracy in terms of image quality prediction in comparison to the state-of-the-art BIQA and full reference IQA techniques.

End-to-End Blind Image Quality Prediction With Cascaded Deep Neural Network

The deep convolutional neural network (CNN) has achieved great success in image recognition. Many image quality assessment (IQA) methods directly use recognition-oriented CNN for quality prediction. However, the properties of IQA task is different from image recognition task. Image recognition should be sensitive to visual content and robust to distortion, while IQA should be sensitive to both distortion and visual content. In this paper, an IQA-oriented CNN method is developed for blind IQA (BIQA), which can efficiently represent the quality degradation. CNN is large-data driven, while the sizes of existing IQA databases are too small for CNN optimization. Thus, a large IQA dataset is firstly established, which includes more than one million distorted images (each image is assigned with a quality score as its substitute of Mean Opinion Score (MOS), abbreviated as pseudo-MOS). Next, inspired by the hierarchical perception mechanism (from local structure to global semantics) in human visual system, a novel IQA-orientated CNN method is designed, in which the hierarchical degradation is considered. Finally, by jointly optimizing the multilevel feature extraction, hierarchical degradation concatenation (HDC) and quality prediction in an end-to-end framework, the Cascaded CNN with HDC (named as CaHDC) is introduced. Experiments on the benchmark IQA databases demonstrate the superiority of CaHDC compared with existing BIQA methods. Meanwhile, the CaHDC (with about 0.73M parameters) is lightweight comparing to other CNN-based BIQA models, which can be easily realized in the microprocessing system. The dataset and source code of the proposed method are available at https://web.xidian.edu.cn/wjj/paper.html .

Perceptual Quality Assessment for Screen Content Images by Spatial Continuity

In this paper, we propose an effective blind quality assessment method for screen content images (SCIs), called perceptual quality measure by spatial continuity (PQSC). With the center-surround mechanism in the human visual system (HVS), the proposed method extracts the statistical features on chromatic and textural variations in SCIs to measure the visual distortion. First, by considering the chromatic continuity between spatially adjacent pixels, photo-metric invariant chromatic descriptors are extracted as zero-order and first-order features. Second, motivated by the perceptual mechanism that the HVS is sensitive to image texture variation, we employ local ternary pattern operator to effectively depict the spatial continuity of texture. With these extracted chromatic and textural features, we further adopt histogram to compute the statistical chromatic and textural features. Support vector regression (SVR) is used to train the quality prediction model from visual features to human ratings. Experimental results on three public benchmark databases demonstrate that the performance of our method is superior to the current blind image quality assessment methods, even better than some full reference image quality assessment counterparts.

Blind Night-Time Image Quality Assessment: Subjective and Objective Approaches

Blind image quality assessment (BIQA) aims to develop quantitative measures to automatically and accurately estimate the visual quality of an image without any prior information about its reference image. This issue has been attracting a great deal of attention for a long time; however, little work has been done on night-time images, which are crucially important for consumer photography and practical applications such as automated driving systems. In this paper, to the best of our knowledge, we conduct the first exploration on subjective and objective quality assessment of night-time images. First, we build a large-scale natural night-time image database (NNID) containing 2240 images with 448 different image contents captured by different photographic equipment in real-world scenarios. Subsequently, we carry out a subjective experiment to evaluate the perceptual quality of all the images in the NNID database. Thereafter, we perform objective assessment of night-time images by proposing a blind night-time image quality assessment metric using brightness and texture features (BNBT). Finally, extensive experiments are conducted to evaluate the performance and efficiency of the proposed BNBT metric on the NNID database. The experimental results demonstrate that this metric outperforms existing state-of-the-art BIQA methods in terms of all evaluation criteria and has an acceptable computational cost at the same time. We have made the NNID database publicly available for downloading at https://sites.google.com/site/xiangtaooo/ .