Natural Image Quality Assessment Research Articles

Learning sparse feature representation for blind quality assessment of night-time images

Capturing Night-Time Images (NTIs) with high-quality is quite challenging for consumer photography and several practical applications. Thus, addressing the quality assessment of night-time images is urgently needed. Since there is no available reference image for such images, Night-Time image Quality Assessment (NTQA) should be done blindly. Although Blind natural Image Quality Assessment (BIQA) has attracted a great deal of attention for a long time, very little work has been done in the field of NTQA. Due to the capturing conditions, NTIs suffer from various complex authentic distortions that make it a challenging field of research. Therefore, previous BIQA methods, do not provide sufficient correlation with subjective scores in the case of NTIs and special methods of NTQA should be developed. In this paper we conduct an unsupervised feature learning method for blind quality assessment of night-time images. The features are the sparse representation over the data-adaptive dictionaries learned on the image exposure and gradient magnitude maps. Having these features, an ensemble regression model trained using least squares gradient boosting scheme predicts high correlated objective scores on the standard datasets.

Spatial resolution and natural image quality assessment evaluation of gamma camera image using pinhole collimator in lutetium-yttrium oxyorthosilicate scintillation detector

Scintillator materials are widely used in the medical and industrial fields for imaging systems using gamma cameras. In this study, image evaluation is performed by modeling a gamma camera system based on a lutetium-yttrium oxyorthosilicate (LYSO) scintillation detector using a pinhole collimator that can improve the spatial resolution. A LYSO detector-based gamma camera system is modeled using a Monte Carlo simulation tool. The geometric concept of the pinhole collimator is designed using various magnification factors, and the spatial resolution is measured using the acquired source image. To evaluate the resolution, the full width at half maximum (FWHM) and natural image quality assessment (NIQE), a no-reference-based parameter, are used. We confirm that the FWHM and NIQE values decrease simultaneously when the diameter of the pinhole collimator increases. Additionally, we confirm that the spatial resolution improves as the magnification factor increases under the same pinhole diameter condition. Particularly, a 0.57 mm FWHM value is obtained using the modeled gamma camera system with a LYSO scintillation detector. In conclusion, our results demonstrate that a pinhole collimator with a LYSO scintillation detector is a promising gamma camera imaging system.

A Human Visual System Inspired No-Reference Image Quality Assessment Method Based on Local Feature Descriptors.

Objective quality assessment of natural images plays a key role in many fields related to imaging and sensor technology. Thus, this paper intends to introduce an innovative quality-aware feature extraction method for no-reference image quality assessment (NR-IQA). To be more specific, a various sequence of HVS inspired filters were applied to the color channels of an input image to enhance those statistical regularities in the image to which the human visual system is sensitive. From the obtained feature maps, the statistics of a wide range of local feature descriptors were extracted to compile quality-aware features since they treat images from the human visual system’s point of view. To prove the efficiency of the proposed method, it was compared to 16 state-of-the-art NR-IQA techniques on five large benchmark databases, i.e., CLIVE, KonIQ-10k, SPAQ, TID2013, and KADID-10k. It was demonstrated that the proposed method is superior to the state-of-the-art in terms of three different performance indices.

Dual-Channel Multi-Task CNN for No-Reference Screen Content Image Quality Assessment

Nowadays the problem of image quality assessment (IQA) for screen content images (SCIs) has become a research hotspot as they are ubiquitous in multimedia applications. Although the quality assessment of natural images (NIs) has been continuously developed in the past few decades, few NI-oriented IQA methods can be directly applied on SCIs due to different visual characteristics between them. In this paper, we present a no-reference quality prediction approach considering the content information of SCIs, which is based on dual-channel multi-task convolutional neural network. First, we segment a SCI into small patches and classify them as the textual patches and the pictorial patches. Then, we devise a novel dual-channel convolutional neural network (CNN) to predict the quality of textual patches and pictorial patches. Finally, we propose an effective adaptive weighting strategy for quality score aggregation. The proposed CNN is built on an end-to-end multi-task learning framework, which assists the SCI quality prediction task through the histogram of oriented gradient (HOG) feature prediction task to learn a better mapping between the input patch and its quality score. The adaptive weighting strategy further improves the representation ability of each SCI patch. Experimental results on two largest SCI-oriented databases demonstrate that the proposed method outperforms most of the state-of-the-art no-reference IQA methods and the full-reference IQA methods.

Attention integrated hierarchical networks for no-reference image quality assessment

Quality assessment of natural images is influenced by perceptual mechanisms, e.g., attention and contrast sensitivity, and quality perception can be generated in a hierarchical process. This paper proposes an architecture of Attention Integrated Hierarchical Image Quality networks (AIHIQnet) for no-reference quality assessment. AIHIQnet consists of three components: general backbone network, perceptually guided neck network, and head network. Multi-scale features extracted from the backbone network are fused to simulate image quality perception in a hierarchical manner. The attention and contrast sensitivity mechanisms modelled by an attention module capture essential information for quality perception. Considering that image rescaling potentially affects perceived quality, appropriate pooling methods in the non-convolution layers in AIHIQnet are employed to accept images with arbitrary resolutions. Comprehensive experiments on publicly available databases demonstrate outstanding performance of AIHIQnet compared to state-of-the-art models. Ablation experiments were performed to investigate the variants of the proposed architecture and reveal importance of individual components.

An optimized CNN-based quality assessment model for screen content image

Most existing convolutional neural network (CNN) based models designed for natural image quality assessment (IQA) employ image patches as training samples for data augmentation, and obtain final quality score by averaging all predicted scores of image patches. This brings two problems when applying these methods for screen content image (SCI) quality assessment. Firstly, SCI contains more complex content compared to natural image. As a result, qualities of SCI patches are different, and the subjective differential mean opinion score (DMOS) is not appropriate as qualities of all image patches. Secondly, the average score of image patches does not represent the quality of entire SCI since the human visual system (HVS) is sensitive to image patches containing texture and edge information. In this paper, we propose a novel quadratic optimized model based on the deep convolutional neural network (QODCNN) for full-reference (FR) and no-reference (NR) SCI quality assessment to overcome these two problems. The contribution of our algorithm can be concluded as follows: 1) Considering the characteristics of SCIs, a valid network architecture is designed for both NR and FR visual quality evaluation of SCIs, which makes the networks learn the feature differences for FR-IQA; 2) with the consideration of correlation between local quality and DMOS, a training data selection method is proposed to fine-tune the pre-trained model with valid SCI patches; 3) an adaptive pooling approach is employed to fuse patch quality to obtain image quality, owns strong noise robust and effects on both FR and NR IQA. Experimental results verify that our model outperforms both current no-reference and full-reference image quality assessment methods on the benchmark screen content image quality assessment database (SIQAD). Cross-database evaluation shows high generalization ability and high effectiveness of our model.

No-reference image quality assessment using fusion metric

This paper presents a fusion featured metric for no-reference image quality assessment of natural images. Natural images exhibit strong statistical properties across the visual contents such as leading edge, high dimensional singularity, scale invariance, etc. The leading edge represents the strong presence of continuous points, whereas high singularity conveys about non-continuous points along the curves. Both edges and curves are equally important in perceiving the natural images. Distortions to the image affect the intensities of these points. The change in the intensities of these key points can be measured using SIFT. However, SIFT tends to ignore certain points such as the points in the low contrast region which can be identified by curvelet transform. Therefore, we propose a fusion of SIFT key points and the points identified by curvelet transform to model these changes. The proposed fused feature metric is computationally efficient and light on resources. The neruofuzzy classifier is employed to evaluate the proposed feature metric. Experimental results show a good correlation between subjective and objective scores for public datasets LIVE, TID2008, and TID2013.

Natural Image Quality Assessment Based on Visual Biological Cognitive Mechanism

With the focus of the main problems in no-reference natural image quality assessment (NR-IQA), the researchers propose a more universal, efficient and integrated resolution based on visual biological cognitive mechanism. First, the authors bring up an inspiring visual cognitive computing model (IVCCM) on the basis of visual heuristic principles. Second, the authors put forward an asymmetric generalized gaussian mixture distribution model (AGGMD), and the model can describe the probability distribution density of the images more precisely. Third, the authors extract the quality-aware multiscale local invariant features (QAMLIF) statistic and perceptive from natural images and form quality-aware uniform features descriptors (QAUFD) based on clustering and encoding the visual quality features. Fourth, the authors build topic semantic model and realize the resolution with Bayesian inference with IVCCM, AGGDM and QAUFD to implement NR-IQA. Theoretical research and experimental results show that the proposed resolution perform better with biological cognitive mechanism.

Sparse Representation for No-Reference Quality Assessment of Satellite Stereo Images

Different from natural image quality assessment methods, satellite stereo images have different requirements on quality in different application scenarios, which poses a huge challenge to establish a suitable objective evaluation model. In this paper, we focus on the quality evaluation of high resolution panchromatic (satellite stereo) images in specific application scenarios of building detection. First, we build a new satellite stereo image database (SSID), which consists of 400 distorted source satellite stereo images (SSIs) generated from the 20-source SSIs with two distortion types and 10-distortion strengths. We use detection accuracy scores to represent the quality of the SSIs, which is obtained through building detection, not subjective testing. We then propose an objective evaluation model based on joint dictionary learning. In the training phase, we bridge the features of the SSIs and the corresponding detection accuracy scores through joint dictionary learning. In the testing phase, we used sparse coding to get the quality of the testing image. The experimental results demonstrate the effectiveness of the proposed method.

Quality Assessment of Screen Content Images via Convolutional-Neural-Network-Based Synthetic/Natural Segmentation.

The recent popularity of remote desktop software and live streaming of composited video has given rise to a growing number of applications which make use of so-called screen content images that contain a mixture of text, graphics, and photographic imagery. Automatic quality assessment (QA) of screen-content images is necessary to enable tasks such as quality monitoring, parameter adaptation, and other optimizations. Although QA of natural images has been heavily researched over the last several decades, QA of screen content images is a relatively new topic. In this paper, we present a QA algorithm, called convolutional neural network (CNN) based screen content image quality estimator (CNN-SQE), which operates via a fuzzy classification of screen content images into plain-text, computergraphics/ cartoons, and natural-image regions. The first two classes are considered to contain synthetic content (text/graphics), and the latter two classes are considered to contain naturalistic content (graphics/photographs), where the overlap of the classes allows the computer graphics/cartoons segments to be analyzed by both text-based and natural-image-based features. We present a CNN-based approach for the classification, an edge-structurebased quality degradation model, and a region-size-adaptive quality-fusion strategy. As we will demonstrate, the proposed CNN-SQE algorithm can achieve better/competitive performance as compared with other state-of-the-art QA algorithms.

A consistency evaluation of signal-to-noise ratio in the quality assessment of human brain magnetic resonance images

BackgroundQuality assessment of medical images is highly related to the quality assurance, image interpretation and decision making. As to magnetic resonance (MR) images, signal-to-noise ratio (SNR) is routinely used as a quality indicator, while little knowledge is known of its consistency regarding different observers.MethodsIn total, 192, 88, 76 and 55 brain images are acquired using T2*, T1, T2 and contrast-enhanced T1 (T1C) weighted MR imaging sequences, respectively. To each imaging protocol, the consistency of SNR measurement is verified between and within two observers, and white matter (WM) and cerebral spinal fluid (CSF) are alternately used as the tissue region of interest (TOI) for SNR measurement. The procedure is repeated on another day within 30 days. At first, overlapped voxels in TOIs are quantified with Dice index. Then, test-retest reliability is assessed in terms of intra-class correlation coefficient (ICC). After that, four models (BIQI, BLIINDS-II, BRISQUE and NIQE) primarily used for the quality assessment of natural images are borrowed to predict the quality of MR images. And in the end, the correlation between SNR values and predicted results is analyzed.ResultsTo the same TOI in each MR imaging sequence, less than 6% voxels are overlapped between manual delineations. In the quality estimation of MR images, statistical analysis indicates no significant difference between observers (Wilcoxon rank sum test, pw ≥ 0.11; paired-sample t test, pp ≥ 0.26), and good to very good intra- and inter-observer reliability are found (ICC, picc ≥ 0.74). Furthermore, Pearson correlation coefficient (rp) suggests that SNRwm correlates strongly with BIQI, BLIINDS-II and BRISQUE in T2* (rp ≥ 0.78), BRISQUE and NIQE in T1 (rp ≥ 0.77), BLIINDS-II in T2 (rp ≥ 0.68) and BRISQUE and NIQE in T1C (rp ≥ 0.62) weighted MR images, while SNRcsf correlates strongly with BLIINDS-II in T2* (rp ≥ 0.63) and in T2 (rp ≥ 0.64) weighted MR images.ConclusionsThe consistency of SNR measurement is validated regarding various observers and MR imaging protocols. When SNR measurement performs as the quality indicator of MR images, BRISQUE and BLIINDS-II can be conditionally used for the automated quality estimation of human brain MR images.

Can Signal-to-Noise Ratio Perform as a Baseline Indicator for Medical Image Quality Assessment

Natural image quality assessment (NIQA) wins increasing attention, while NIQA models are rarely used in the medical community. A couple of studies employ the NIQA methodologies for medical image quality assessment (MIQA), but building the benchmark data sets necessitates considerable time and professional skills. In particular, the characteristics of synthesized distortions are different from those of clinical distortions, which make the results not so convincing. In clinic, signal-to-noise ratio (SNR) is widely used, which is defined as the quotient of the mean signal intensity measured in a tissue region of interest (ROI) and the standard deviation of the signal intensity in an air region outside the imaged object, and both regions are outlined by specialists. We take advantage of the knowledge that SNR is routinely used and concern whether SNR measure can perform as a baseline metric for the development of MIQA algorithms. To address the issue, the inter-observer reliability of SNR measure is investigated regarding to different tissue ROIs [white matter (WM); cerebral spinal fluid (CSF)] in magnetic resonance (MR) images. A total of 192 T <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sub> , 88 T <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> , 76 T <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sub> and 55 contrast-enhanced T <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> (T <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> C) weighted images are analyzed. Statistical analysis indicates that SNR values show consistency between different observers to the same ROI in each modality (Wilcoxon rank sum test, p <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">w</sub> ≥ 0.11; and paired sample t-test, pp 0.28). Moreover, whether off-the-shelf NIQA models can predict MR image quality is considered by using SNR values as the reference scores. Four NIQA models (BIQI, BLIINDS-II, BRISQUE, and NIQE) are evaluated, and the correlation between SNR values and NIQA results is evaluated. Pearson correlation coefficient (r <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">p</sub> ) shows that WM-based SNR values correlates well with BIQI, BLIINDS-II and BRISQUE in T <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sub> <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">*</sup> images (rp 0.77), BRISQUE and NIQE in T <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> images (r <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">p</sub> ≥ 0.75), BLIINDS-II in T <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sub> images (rp 0.67), and BRISQUE and NIQE in T <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> C images (r <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">p</sub> ≥ 0.58), while CSF-based SNR values correlates well with BLIINDS-II in T <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sub> <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">*</sup> images (r <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">p</sub> ≥ 0.64) and T <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sub> images (r <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">p</sub> ≥ 0.60), and all p <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">p</sub> <; 10 <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">-4</sup> . The prediction performance analysis further proves the result from the correlation analysis. Conclusively, SNR measure is reliable to different observations and can perform as a baseline indicator for the development of MIQA algorithms. In general, BRISQUE and BLIINDS-II are full of potential to be conditionally used as objective MIQA models toward human brain MR images. This paper presents the first attempt of using SNR measure to bridge the gap between NIQA and MIQA, and large-scale experiments should be further conducted to confirm the conclusion in this paper.

Nonparametric Quality Assessment of Natural Images

In this article, the authors explore an alternative way to perform no-reference image quality assessment (NR-IQA). Following a feature extraction stage in which spatial domain statistics are utilized as features, a two-stage nonparametric NR-IQA framework is proposed. This approach requires no training phase, and it enables prediction of the image distortion type as well as local regions' quality, which is not available in most current algorithms. Experimental results on IQA databases show that the proposed framework achieves high correlation to human perception of image quality and delivers competitive performance to state-of-the-art NR-IQA algorithms.