No-reference Image Quality Assessment Model Research Articles

Quality-aware blind image motion deblurring

Recent mapping-based motion deblurring methods lack the regularization of prior knowledge, resulting in an over-reliance on the training data and limited generalization ability. As deblurring aims to improve image quality, we quantitatively analyze and further discover the strong correlation between image quality and sharpness. Motivated by the above facts and notable accomplishments of recent no-reference image quality assessment (NR-IQA), we present a novel framework that incorporates quality knowledge into mapping-based deblurring models. Specifically, we extract quality-aware features from NR-IQA models as prior knowledge, and subsequently propose a prediction-based strategy and an encoder-reuse strategy to integrate knowledge into the encoder and decoder, respectively. After training, the model can simultaneously deblur images and predict quality features, indicating that it has grasped the knowledge and validating the effectiveness of the proposed embedding strategies. Extensive experimental results show that embedding quality knowledge consistently improves model performance and the model achieves state-of-the-art intra/cross-dataset results. Code and pre-trained models are available at https://github.com/esnthere/QAMD.

Transformer-Based No-Reference Image Quality Assessment via Supervised Contrastive Learning

Image Quality Assessment (IQA) has long been a research hotspot in the field of image processing, especially No-Reference Image Quality Assessment (NR-IQA). Due to the powerful feature extraction ability, existing Convolution Neural Network (CNN) and Transformers based NR-IQA methods have achieved considerable progress. However, they still exhibit limited capability when facing unknown authentic distortion datasets. To further improve NR-IQA performance, in this paper, a novel supervised contrastive learning (SCL) and Transformer-based NR-IQA model SaTQA is proposed. We first train a model on a large-scale synthetic dataset by SCL (no image subjective score is required) to extract degradation features of images with various distortion types and levels. To further extract distortion information from images, we propose a backbone network incorporating the Multi-Stream Block (MSB) by combining the CNN inductive bias and Transformer long-term dependence modeling capability. Finally, we propose the Patch Attention Block (PAB) to obtain the final distorted image quality score by fusing the degradation features learned from contrastive learning with the perceptual distortion information extracted by the backbone network. Experimental results on six standard IQA datasets show that SaTQA outperforms the state-of-the-art methods for both synthetic and authentic datasets. Code is available at https://github.com/I2-Multimedia-Lab/SaTQA.

A no-reference image quality assessment model based on neighborhood component analysis and Gaussian process

In this paper, a human visual system (HVS) based no-reference image quality assessment model (NR-IQA) is proposed. This is an objective NR-IQA model that uses perceptual features such as structure and orientation to compute the loss of naturalness in the image. An adaptive feature extraction process is modeled to capture the underlying significant features of the images along with the noise. Moreover, the optimal subset of features is derived using the neighborhood component analysis and further used as inputs to the Gaussian process for regression. The various experimentations are carried out on LIVE, TID2008, and TID2013 databases to test the effectiveness of the proposed approach. It is observed that the presented model exhibits a competitive performance in comparison with the existing IQA approaches. Moreover, the computation complexity and run-time show the effectuality of the proposed approach. Further, the experiments show that the predicted score matches with human perceptions. Thus, the proposed model is more accurate, less complex, independent of distortions, and well-suited for real-time applications.

Deep Ordinal Regression Framework for No-Reference Image Quality Assessment

Due to the rapid development of deep learning techniques, no-reference image quality assessment (NR-IQA) has achieved significant improvement. NR-IQA aims to predict a real-valued variable for image quality, using the image in question as the sole input. Existing deep learning-based NR-IQA models are formulated as a regression problem and trained by minimising the mean squared error. The error measurement does not consider the relative ordering between different ratings on the quality scale, which consequently affects the efficacy of the model. To account for this problem, we reformulate NR-IQA learning as an ordinal regression problem and propose a simple yet effective framework using deep convolutional neural networks (DCNN) and Transformers. NR-IQA learning is achieved by a deep ordinal loss and using a soft ordinal inference to transform the predicted probabilities to a continuous variable for image quality. Experimental results demonstrate the superiority of our proposed NR-IQA model based on deep ordinal regression. In addition, this framework can be easily extended with various DCNN architectures to build advanced IQA models.

An efficient approach for no-reference image quality assessment based on statistical texture and structural features

This paper presents a rotation-invariant and computationally efficient no-reference image quality assessment (NR-IQA) model. It estimates the image quality based on texture and structural information associated with the images. The human visual system (HVS) uses perceptual features such as texture and structure as primary information to understand the visual scene and the image content. Moreover, the texture and structural information capture the loss of naturalness due to distortions in the image. Therefore, in this work, the important texture features are extracted using the local binary patterns (LBP).The modified LBP, also known as the hyper-smoothing LBP (H-LBP) and Laplacian of H-LBP (LH-LBP), represents the image structure. Further, the image quality prediction model computes the quality of the image based on the statistical feature measures of the texture and structural information. In the proposed approach, the image quality prediction model uses support vector regression (SVR) to measure image quality. Various experimentations are carried out on the LIVE and TID2013 database to test the effectiveness of the proposed NR-IQA model. The performance metrics such as Spearman rank-ordered correlation coefficient, Pearson linear correlation coefficient, and root mean square error is computed to show the efficiency of the presented approach. The experimental results illustrate a high correlation between the predicted quality score and the human visual perceptions. It is also found to be competitive with the best-performing full-reference and no-reference IQA models.

Generalizable No-Reference Image Quality Assessment via Deep Meta-Learning

Recently, researchers have shown great interest in using convolutional neural networks (CNNs) for no-reference image quality assessment (NR-IQA). Due to the lack of big training data, the efforts of existing metrics in optimizing CNN-based NR-IQA models remain limited. Furthermore, the diversity of distortions in images result in the generalization problem of NR-IQA models when trained with known distortions and tested on unseen distortions, which is an easy task for human. Hence, we propose a NR-IQA metric via deep meta-learning, which is highly generalizable in the face of unseen distortions. The fundamental idea is to learn the meta-knowledge shared by human when evaluating the quality of images with diversified distortions. Specifically, we define NR-IQA of different distortions as a series of tasks and propose a task selection strategy to build two task sets, which are characterized by synthetic to synthetic and synthetic to authentic distortions, respectively. Based on these two task sets, an optimization-based meta-learning is proposed to learn the generalized NR-IQA model, which can be directly used to evaluate the quality of images with unseen distortions. Extensive experiments demonstrate that our NR-IQA metric outperforms the state-of-the-arts in terms of both evaluation performance and generalization ability.

No-Reference Image Quality Assessment Based on Edge Pattern Feature in the Spatial Domain

This paper proposes a general-purpose no-reference image quality assessment (NR-IQA) method that investigates the image’s structure information from a new aspect, i.e., the characteristic of image edge profiles that depict the directional property of adjacent edge points in the spatial domain of the image. More specifically, we extracted the image’s edge map based on Laplacian of Gaussian (LoG) filtration and zero-crossing (ZC) detection and refined the edge map to be 1-pixel wide. We then explored the edge map by investigating edge profiles’ statistics in a local window with a $5\times 5$ -pixel size. Considering the consensus that natural images consist of directional structures, we found that the spatial distribution property of adjacent edge points can be represented through several edge profiles called edge patterns, which are selected from natural images with a proposed smooth criterion. With the proposed edge patterns and their statistical histogram for the image and the support vector regression technique, we proposed the NR-IQA model based on the edge patterns in the spatial domain, named EPISD. The proposed method has been extensively validated on the LIVE, CSIQ, TID2013, MDID2017, SIQAD, and SCID databases. The experimental results showed that EPISD has a competitive performance with state-of-the-art methods and works stably across different databases.

Blind S3D image quality prediction using classical and non-classical receptive field models

We develop a novel no-reference image quality assessment model for stereoscopic 3D (S3D) images that is inspired by functional receptive field models of perceptual mechanisms in primary visual cortex (V1). The approach is called the Blind S3D Integrated Quality Evaluator (BSIQE). BSIQE simulates monocular and binocular responses to stereo views using channel separation and weighted multi-channel combination models. Binocular responses are modeled as the fusion of the two channels using a weighted multi-channel combination. The responses to stereoscopic image content of both classical and non-classical anisotropic receptive fields are then modeled based on a determination of the relative importance of the receptive field responses. In the last stage of feature extraction, we deploy a simple and efficient way of decorrelating the picture data. We extract local binary pattern (LBP) statistical features from the computed receptive field responses, and use them to train a regressor to predict the perceptual quality of stereoscopic images. We carefully evaluate BSIQE on four public-domain 3D image quality databases, and find that it is statistically superior to all compared 2D and 3D IQA algorithms. BSIQE exhibits good performance across the datasets suggesting that it is general, and it has relatively low complexity.

No-Reference Image Quality Assessment Based on Dual-Domain Feature Fusion.

Image quality assessment (IQA) aims to devise computational models to evaluate image quality in a perceptually consistent manner. In this paper, a novel no-reference image quality assessment model based on dual-domain feature fusion is proposed, dubbed as DFF-IQA. Firstly, in the spatial domain, several features about weighted local binary pattern, naturalness and spatial entropy are extracted, where the naturalness features are represented by fitting parameters of the generalized Gaussian distribution. Secondly, in the frequency domain, the features of spectral entropy, oriented energy distribution, and fitting parameters of asymmetrical generalized Gaussian distribution are extracted. Thirdly, the features extracted in the dual-domain are fused to form the quality-aware feature vector. Finally, quality regression process by random forest is conducted to build the relationship between image features and quality score, yielding a measure of image quality. The resulting algorithm is tested on the LIVE database and compared with competing IQA models. Experimental results on the LIVE database indicate that the proposed DFF-IQA method is more consistent with the human visual system than other competing IQA methods.

Dual Head Network for No-Reference Quality Assessment Towards Realistic Night-Time Images

No-reference image quality assessment (NR-IQA), which devotes to predicting image quality without relying on the corresponding pristine counterpart, develops rapidly in recent years. However, little investigation has been dedicated to quality assessment of realistic night-time images. Existing NR-IQA algorithms laboriously cope with this night-time scenario since complicated authentic distortions such as low contrast, blurred details, and reduced visibility usually appear on it. In this paper, we propose an end-to-end NR-IQA model to meet this challenge based on a multi-stream deep convolutional neural network (DCNN). Two streams, brightness-aware CNN and naturalness-aware CNN are constructed respectively by a brightness-altered image identification task with a self-established dataset and a quality-prediction regression task with an existing authentically-distorted IQA dataset to improve quality-aware initializations. In this case, given the quick convergence and little transformation in the lower layers, a shallow-layer-shared architecture is explored to reduce computational cost. Finally, the features of these two pipelines are collected by an effective pooling method and then concatenated as the image representation for fine-tuning. The effectiveness and efficiency of the proposed method are verified by several different experiments on the NNID, CCRIQ and LIVE Challenge databases. Furthermore, the superiority of wide applications such as for contrast-distorted and driving scenarios is demonstrated on the CID2013, CCID2014 and BBD-100k databases.

An improved model for no-reference image quality assessment and a no-reference video quality assessment model based on frame analysis

No-reference image quality assessment (NR-IQA) uses only the test image for its quality assessment, and as video is essentially comprised of image frames with additional temporal dimension, video quality assessment (VQA) requires a thorough understanding of image quality assessment metrics and models. Therefore, in order to identify features that deteriorate video quality, a fundamental analysis of spatial and temporal artifacts with respect to individual video frames needs to be performed. Existing IQA and VQA metrics are primarily for capturing few distortions and hence may not be good for all types of images and videos. In this paper, we propose an NR-IQA model by combining existing three methods (namely NIQE, BRISQUE and BLIINDS-II) using multi-linear regression. We also present a holistic no-reference video quality assessment (NR-VQA) model by exploring quantification of certain distortions like ringing, frame difference, blocking, clipping and contrast in video frames. For the proposed NR-IQA model, the results represent improved performance as compared to the state-of-the-art methods and it requires very low fraction of samples for training to provide a consistent accuracy over different training-to-testing ratios. The performance of NR-VQA model is examined using a simple neural network model to attain high value of goodness of fit.

Image Statistic Models Characterize Well Log Image Quality

Assessing the image quality of well logs is essential to ensure the accuracy of their digitization and subsequent processing. Currently, the suitability of well logs for information retrieval is solely determined on the basis of subjective judgments of their image quality by human experts. The success of natural scene statistics (NSS)-based models that are used to conduct no-reference (NR) quality assessment of photographic images motivates us to try to exploit them to characterize the quality of nonphotographic images, such as well logs. Accordingly, we develop a scheme to characterize the quality of a well log as “acceptable” or “unacceptable” for subsequent processing based on the natural image quality evaluator (NIQE), a successful NR image quality assessment model based on the NSS. Our experimental results show that the objective quality scores thus obtained can be reliably used to eliminate well logs of inferior quality from the processing pipeline, which can serve as a beneficial step to reduce the human hours spent in examining well logs and to improve the rate of information retrieval as well as the accuracy of retrieved information. Source code for the trained well log image quality predictor is available at https://github.com/Somdyuti2/Well_log_IQA .

No-Reference Blurred Image Quality Assessment by Structural Similarity Index

No-reference (NR) image quality assessment (IQA) objectively measures the image quality consistently with subjective evaluations by using only the distorted image. In this paper, we focus on the problem of NR IQA for blurred images and propose a new no-reference structural similarity (NSSIM) metric based on re-blur theory and structural similarity index (SSIM). We extract blurriness features and define image blurriness by grayscale distribution. NSSIM scores an image quality by calculating image luminance, contrast, structure and blurriness. The proposed NSSIM metric can evaluate image quality immediately without prior training or learning. Experimental results on four popular datasets show that the proposed metric outperforms SSIM and well-matched to state-of-the-art NR IQA models. Furthermore, we apply NSSIM with known IQA approaches to blurred image restoration and demonstrate that NSSIM is statistically superior to peak signal-to-noise ratio (PSNR), SSIM and consistent with the state-of-the-art NR IQA models.

RAN4IQA: Restorative Adversarial Nets for No-Reference Image Quality Assessment

Inspired by the free-energy brain theory, which implies that human visual system (HVS) tends to reduce uncertainty and restore perceptual details upon seeing a distorted image, we propose restorative adversarial net (RAN), a GAN-based model for no-reference image quality assessment (NR-IQA). RAN, which mimics the process of HVS, consists of three components: a restorator, a discriminator and an evaluator. The restorator restores and reconstructs input distorted image patches, while the discriminator distinguishes the reconstructed patches from the pristine distortion-free patches. After restoration, we observe that the perceptual distance between the restored and the distorted patches is monotonic with respect to the distortion level. We further define Gain of Restoration (GoR) based on this phenomenon. The evaluator predicts perceptual score by extracting feature representations from the distorted and restored patches to measure GoR. Eventually, the quality score of an input image is estimated by weighted sum of the patch scores. Experimental results on Waterloo Exploration, LIVE and TID2013 show the effectiveness and generalization ability of RAN compared to the state-of-the-art NR-IQA models.

No-Reference Image Quality Assessment by Wide-Perceptual-Domain Scorer Ensemble Method.

A no-reference (NR) learning-based approach to assess image quality is presented in this paper. The devised features are extracted from wide perceptual domains, including brightness, contrast, color, distortion, and texture. These features are used to train a model (scorer) which can predict scores. The scorer selection algorithms are utilized to help simplify the proposed system. In the final stage, the ensemble method is used to combine the prediction results from selected scorers. Two multiple-scale versions of the proposed approach are also presented along with the single-scale one. They turn out to have better performances than the original single-scale method. Because of having features from five different domains at multiple image scales and using the outputs (scores) from selected score prediction models as features for multi-scale or cross-scale fusion (i.e., ensemble), the proposed NR image quality assessment models are robust with respect to more than 24 image distortion types. They also can be used on the evaluation of images with authentic distortions. The extensive experiments on three well-known and representative databases confirm the performance robustness of our proposed model.

Super resolution and recognition of long range captured multi‐frame iris images

In this study, a framework to super resolve and recognise the long range captured iris polar images is proposed. The proposed framework consists of best frame selection algorithm, modified diamond search algorithm, Gaussian process regression (GPR) based and enhanced iterated back projection (EIBP)-based super-resolution approach, fuzzy entropy-based feature selector and neural network (NN) classifier. The framework uses linear kernel co-variance function in local patch-based GPR and EIBP algorithms and it super resolves the iris images depending on the contents of the patches, without an external dataset. NN classifier classifies the iris images by using features extracted using discrete cosine transform domain based no-reference image quality assessment model, Gray level co-occurrence matrix, Hu seven moments and statistical features. The framework is tested using CASIA long range iris database by comparing and analysing the peak signal-to-noise ratio, structural similarity index matrix and visual information fidelity in pixel domain of proposed algorithms with Yang and Nguyen framework. The results demonstrate that the proposed framework is well suited for recognition of iris images captured at a long distance.

No-reference image quality assessment in curvelet domain

We study the efficacy of utilizing a powerful image descriptor, the curvelet transform, to learn a no-reference (NR) image quality assessment (IQA) model. A set of statistical features are extracted from a computed image curvelet representation, including the coordinates of the maxima of the log-histograms of the curvelet coefficients values, and the energy distributions of both orientation and scale in the curvelet domain. Our results indicate that these features are sensitive to the presence and severity of image distortion. Operating within a 2-stage framework of distortion classification followed by quality assessment, we train an image distortion and quality prediction engine using a support vector machine (SVM). The resulting algorithm, dubbed CurveletQA for short, was tested on the LIVE IQA database and compared to state-of-the-art NR/FR IQA algorithms. We found that CurveletQA correlates well with human subjective opinions of image quality, delivering performance that is competitive with popular full-reference (FR) IQA algorithms such as SSIM, and with top-performing NR IQA models. At the same time, CurveletQA has a relatively low complexity.

Making a “Completely Blind” Image Quality Analyzer

An important aim of research on the blind image quality assessment (IQA) problem is to devise perceptual models that can predict the quality of distorted images with as little prior knowledge of the images or their distortions as possible. Current state-of-the-art “general purpose” no reference (NR) IQA algorithms require knowledge about anticipated distortions in the form of training examples and corresponding human opinion scores. However we have recently derived a blind IQA model that only makes use of measurable deviations from statistical regularities observed in natural images, without training on human-rated distorted images, and, indeed without any exposure to distorted images. Thus, it is “completely blind.” The new IQA model, which we call the Natural Image Quality Evaluator (NIQE) is based on the construction of a “quality aware” collection of statistical features based on a simple and successful space domain natural scene statistic (NSS) model. These features are derived from a corpus of natural, undistorted images. Experimental results show that the new index delivers performance comparable to top performing NR IQA models that require training on large databases of human opinions of distorted images. A software release is available at http://live.ece.utexas.edu/research/quality/niqe_release.zip.

Local region-based image quality assessment independent of JPEG and JPEG2000 coded color images

The importance of the perceived quality measurement is fundamental for many image processing applications, such as compression, acquisition, restoration, enhancement, and reproduction. Color information is also of great importance for the perceived image quality, although perceived information is mainly represented by luminance. We present a computational and memory-efficient no-reference image quality assessment model independent of JPEG and JPEG2000 coded color images based on local regions. We also present the discrimination algorithm for these two types of coded images. The features of local regions are blockiness around the block boundary, average absolute difference between adjacent pixels within the block, and zero crossing rate within the block of the image. We validate the performance of our model on our subjective database, which shows good quality prediction performance, and the model's generalization ability is also verified on the other database.