Image Quality Assessment Model Research Articles

No-Reference Image Quality Assessment Based on Dual-Domain Feature Fusion.

Image quality assessment (IQA) aims to devise computational models to evaluate image quality in a perceptually consistent manner. In this paper, a novel no-reference image quality assessment model based on dual-domain feature fusion is proposed, dubbed as DFF-IQA. Firstly, in the spatial domain, several features about weighted local binary pattern, naturalness and spatial entropy are extracted, where the naturalness features are represented by fitting parameters of the generalized Gaussian distribution. Secondly, in the frequency domain, the features of spectral entropy, oriented energy distribution, and fitting parameters of asymmetrical generalized Gaussian distribution are extracted. Thirdly, the features extracted in the dual-domain are fused to form the quality-aware feature vector. Finally, quality regression process by random forest is conducted to build the relationship between image features and quality score, yielding a measure of image quality. The resulting algorithm is tested on the LIVE database and compared with competing IQA models. Experimental results on the LIVE database indicate that the proposed DFF-IQA method is more consistent with the human visual system than other competing IQA methods.

Unified non-uniform scale adaptive sampling model for quality assessment of natural scene and screen content images

Among many multimedia scenarios, digital images are mainly composed of natural scene images (NSIs) and screen content images (SCIs). NSIs are always captured by cameras from the real world while SCIs are typically the mixture of pictures and text. Traditional full-reference image quality assessment (IQA) metrics are designed for natural scene images, and they cannot work well on SCIs, because they do not take account of characteristics of different visual content/regions between NSIs and SCIs. Inspired by early eye movement study that human visual system samples the world non-completely or non-uniformly, we observe that human brain perceive different image content at different scales. Given these discoveries, we propose a non-uniform scale adaptive sampling model (NUSM) which dedicates to preprocess the input images for IQA methods, enabling traditional NSI IQA models to predict the quality of SCIs effectively. The proposed sampling model applies different sample scales into different regions to generate pre-sampled image pairs, then send to existing IQAs to evaluate the quality of images. The experimental results demonstrate that our model has achieved a promising performance on both NSIs and SCIs.

Enhanced image no‐reference quality assessment based on colour space distribution

In this study, the authors investigate the problem of enhanced image no-reference (NR) quality assessment. For resolving the problem of the enhanced images, it is difficult to obtain reference images, this study proposes an NR image quality assessment (IQA) model based on colour space distribution. Given an enhanced image, our method first uses a gist to select a clear target image in which the scene, colour and quality are similar to the hypothetical reference images. And then, the colour transfer is used between the input images and target images to construct the reference image. Next, the appropriate IQA method is used to assess enhanced image quality. The absolute colour difference and feature similarity (FSIM) are used to measure the colour and grey-scale image quality, respectively. Extensive experiments demonstrate that the proposed method is good at evaluating enhanced image quality for X-ray, dust, underwater and low-light images. The experimental results are consistent with human subjective evaluation and achieve good assessment effects.

A Light Field Image Quality Assessment Model Based on Symmetry and Depth Features

This paper presents a new full-reference image quality assessment (IQA) method for conducting the perceptual quality evaluation of the light field (LF) images, called the symmetry and depth feature-based model (SDFM). Specifically, the radial symmetry transform is first employed on the luminance components of the reference and distorted LF images to extract their symmetry features for capturing the spatial quality of each view of an LF image. Second, the depth feature extraction scheme is designed to explore the geometry information inherited in an LF image for modeling its LF structural consistency across views. The similarity measurements are subsequently conducted on the comparison of their symmetry and depth features separately, which are further combined to achieve the quality score for the distorted LF image. Note that the proposed SDFM that explores the symmetry and depth features is conformable to the human vision system, which identifies the objects by sensing their structures and geometries. Extensive simulation results on the dense light fields dataset have clearly shown that the proposed SDFM outperforms multiple classical and recently developed IQA algorithms on quality evaluation of the LF images.

Automated fundus image quality assessment and segmentation of optic disc using convolutional neural networks

An automated fundus image analysis is used as a tool for the diagnosis of common retinal diseases. A good quality fundus image results in better diagnosis and hence discarding the degraded fundus images at the time of screening itself provides an opportunity to retake the adequate fundus photographs, which save both time and resources. In this paper, we propose a novel fundus image quality assessment (IQA) model using the convolutional neural network (CNN) based on the quality of optic disc (OD) visibility. We localize the OD by transfer learning with Inception v-3 model. Precise segmentation of OD is done using the GrabCut algorithm. Contour operations are applied to the segmented OD to approximate it to the nearest circle for finding its center and diameter. For training the model, we are using the publicly available fundus databases and a private hospital database. We have attained excellent classification accuracy for fundus IQA on DRIVE, CHASE-DB, and HRF databases. For the OD segmentation, we have experimented our method on DRINS-DB, DRISHTI-GS, and RIM-ONE v.3 databases and compared the results with existing state-of-the-art methods. Our proposed method outperforms existing methods for OD segmentation on Jaccard index and F-score metrics.

AdaBoost neural network and cyclopean view for no-reference stereoscopic image quality assessment

Stereoscopic imaging is widely used in many fields. In many scenarios, stereo images quality could be affected by various degradations, such as asymmetric distortion. Accordingly, to guarantee the best quality of experience, robust and accurate reference-less metrics are required for quality assessment of stereoscopic content. Most existing stereo no-reference Image Quality Assessment (IQA) models are not consistent with asymmetrical distortions. This paper presents a new no-reference stereoscopic image quality assessment metric using a human visual system (HVS) modeling and an advanced machine-learning algorithm. The proposed approach consists of two stages. In the first stage, cyclopean image is constructed considering the presence of binocular rivalry in order to cover the asymmetrically distorted part. In the second stage, gradient magnitude, relative gradient magnitude, and gradient orientation are extracted. These are used as a predictive source of information for the quality. In order to obtain the best overall performance against different databases, Adaptive Boosting (AdaBoost) idea of machine learning combined with artificial neural network model has been adopted. The benchmark LIVE 3D phase-I, phase-II, and IRCCyN/IVC 3D databases have been used to evaluate the performance of the proposed approach. Experimental results have demonstrated that the proposed metric performance achieves high consistency with subjective assessment and outperforms the blind stereo IQA over various types of distortion.

No‐reference image quality assessment via structural information fluctuation

Image quality assessment (IQA) is a meaningful research topic to meet the increasing demand of high-quality image. The degradation of image quality will cause changes in image structural information. Meanwhile, human visual system is sensitive to changes in structural information. This finding motivates us to utilise structural information for proposing IQA method which is consistent with human visual perception. Recently, IQA methods are mainly focused on individual image type, e.g. natural image or screen content image (SCI), thus, the authors proposed a novel no-reference IQA method which can be suitable for both natural image and SCI. The proposed method is based on structural information analysis. For each image, they first obtain the grey-scale fluctuation maps (GFMs) in four detection directions. After that, the grey-scale fluctuation direction map (GFD) of certain image can be acquired via its GFMs. Based on the GFMs and GFD, the structural features of each image are extracted, and then collected and transformed to feature vectors. Subsequently, the IQA model is trained by support vector regression. The experimental results on the public databases demonstrate the proposed method can predict image quality accurately for both natural image and SCI, and the performance is competitive with prevalent methods.

Objective image quality assessment based on image color appearance and gradient features

With the rapid development of color image contents and imaging devices in various kinds of multimedia communication systems, conventional grayscale counterparts are replaced by chromatic ones. Under such a transition, the image quality assessment (IQA) model needs to be built by subjective visual measurement, designed in accordance with the results, and applied to the related practical problems. Based on the visual perception characteristics, chromaticity and the structure feature information are quantified, and an objective IQA model combining the color appearance and the gradient image features is proposed in this paper, namely color appearance and gradient similarity(CAGS) model. Two new color appearance indices, vividness and depth, are selected to build the chromatic similarity map. The structure information is characterized by gradient similarity map. Vividness map plays two roles in the proposed model. One is utilized as feature extractor to compute the local quality of distorted image, and the other is as a weight part to reflect the importance of local domain. To quantify the specific parameters of CAGS, Taguchi method is used and four main parameters, i.e., <i>K</i><sub><i>V</i></sub>, <i>K</i><sub><i>D</i></sub>, <i>K</i><sub><i>G</i></sub> and <i>α</i>, of this model are determined based on the statistical correlation indices. The optimal parameters of CAGS are <i>K</i><sub><i>V</i></sub> = <i>K</i><sub><i>D</i></sub> = 0.02, <i>K</i><sub><i>G</i></sub> = 50, and <i>α</i> = 0.1. Furthermore, the CAGS is tested by utilizing 94 reference images and 4830 distorted images from the four open image databases (LIVE, CSIQ, TID2013 and IVC). Additionally, the influences of the 35 distortion types on IQA are analyzed. Massive experiments are performed on four publicly available benchmark databases between CAGS and other 10 state-of-the-art and recently published IQA models, for the accuracy, complexity and generalization performance of IQA. The experimental results show that the accuracy PLCC of the CAGS model can achieve 0.8455 at lowest and 0.9640 at most in the four databases, and the results about commonly evaluation criteria prove that the CAGS performs higher consistency with the subjective evaluations. Among the 35 distortion types, the two distortion types, namely contrast change and change of color saturation, CAGS and mostly IQA models have the worst influence on IQA, and the CAGS yields the highest top three rank number. Moreover, the SROCC values of CAGS for other distortion types are all larger than 0.6 and the number of SROCC value larger than 0.95 is 14 times. Besides, the CAGS maintains a moderate computational complexity. These results of test and comparison above show that the CAGS model is effective and feasible, and the corresponding model has an excellent performance.

Full Reference Image Quality Assessment Based on Visual Salience With Color Appearance and Gradient Similarity

Image quality assessment (IQA) model is designed to measure the image quality in consistent with subjective ratings by computational models. In this research, a reliable full reference color IQA model is proposed by combining the Visual saliency with Color appearance (VC) similarity, gradient similarity and chrominance similarity. Two new color appearance indices, vividness and depth, are selected to build the visual saliency similarity map. The structure and chrominance features are characterized by different channels of chosen color space. VC map plays two roles in the proposed model. One is utilized as feature to compute the local quality of distorted image, and the other is as a weight part to reflect the importance of local domain. The novel model is called visual saliency with color appearance and gradient similarity (VCGS). To quantify the specific parameters of VCGS, some experiments are conducted based on the statistical correlation indices. Massive experiments are performed on the publicly available benchmark single and multiple distortion databases, and the commonly evaluation criteria results prove that VCGS works with higher consistency with the subjective evaluations than the other state-of-the-art IQA models for the prediction accuracy. Besides, VCGS maintain a moderate computational complexity. The MATLAB source code of VCGS is publicly available online at https://github.com/AlAlien/VCGS.

Objective assessment of image quality based on image content contrast perception

Image quality assessment (IQA) plays a very important role in acquiring, storing, transmitting and processing image and video. Using the characteristics of human visual perception and the features of the gray, gradient, local contrast, and blurring of image, an IQA method based on the image content contrast perception is proposed in the paper, which is called MPCC. In the proposed method, firstly, combining with the characteristics of human visual perception, based on the definition of the contrast in physics, a novel definition for image quality and its calculation method are proposed. Then, based on the gray gradient co-occurrence matrix, a novel concept, namely the gray-gradient entropy of image, and its calculation method, are proposed. And based on the gray-gradient entropy, local contrast and blurring of image, a method of describing the image content and their visual perception are proposed. Finally, based on the image content features and the image quality definition, an IQA method and its mathematical model are proposed by comprehensive analysis. Further, the proposed IQA model MPCC is tested by using 119 reference images and 6395 distorted images from the five open image databases (LIVE, CSIQ, TID2008, TID2013 and IVC). Moreover, the influences of the 52 distortion types on IQA are analyzed. In addition, in order to illustrate the advantages of the MPCC model, it is compared with the seven existing typical IQA models in terms of the accuracy, complexity and generalization performance of model. The experimental results show that the accuracy PLCC of the MPCC model can achieve 0.8616 at lowest and 0.9622 at most in the five databases; among the 52 distortion types, the two distortion types, namely the change of color saturation and the local block-wise distortions of different intensity, have the greatest influence on IQA, and the accuracy PLCC values of the seven existing IQA models are almost all below 0.6, but the PLCC of the MPCC model can reach more than 0.68; and the comprehensive benefit of the performance of the MPCC model is better than those of the seven existing IQA models. These results of test and comparison above show that the proposed IQA method is effective and feasible, and the corresponding model has an excellent performance.

Adaptively Truncating Gradient for Image Quality Assessment

Abstract Objective image quality assessment (IQA) aims to develop computational models to predict the perceptual image quality consistent with subjective evaluations. As image information is presented by the change in intensity values in the spatial domain, the gradient, as a basic tool for measuring the change, is widely used in IQA models. However, does the change measured by the gradient actually correspond to the change perceived by the human visual system (HVS)? To explore this issue, in this paper, we analyze how the ability of the HVS to perceive changes is affected by the upper threshold, and we propose an IQA index based on an adaptively truncating gradient. Specifically, the upper threshold at each pixel in an image is adaptively determined according to the image content, and the adaptively truncating gradient is obtained by retaining the part of the gradient magnitude that is less than the upper threshold and truncating the part that is greater than the upper threshold. Then, the distorted image quality is calculated by comparing the similarity of the adaptively truncating gradient between a reference image and the distorted image. Experimental results on six benchmark databases demonstrate that the proposed index correlates well with human evaluations.

Blind Image Quality Assessment for Super Resolution via Optimal Feature Selection

Methods for image Super Resolution (SR) have started to benefit from the development of perceptual quality predictors that are designed for super resolved images. However, extensive cross dataset validation studies have not yet been performed on Image Quality Assessment (IQA) for super resolved images. Moreover, powerful natural scene statistics-based approaches for IQA have not yet been studied for SR. To address these issues, we introduced a new dataset of super-resolved images with associated human quality scores. The dataset is based on the existing SupER dataset, which contains real low-resolution images. This new dataset also has 7 SR algorithms at three magnification scales. We selected optimal quality aware features to create two no-reference, (NR) opinion-distortion unaware (ODU) IQA models. Using the same set of selected features, we also implemented two NR-IQA opinion/distortion aware (ODA) models. The selection process identified paired-product (PP) features and those derived from discrete cosine transform coefficients (DCT) as the most relevant for the quality prediction of SR images. We conducted cross dataset validation for several state-of-the-art quality algorithms in four datasets, including our new dataset. The conducted experiments indicate that our models achieved better than state-of-the-art performance among the NR-IQA metrics. Our NR-IQA source code and the dataset are available at https://github.com/juanpaberon/IQA_SR.

Blind Stereo Image Quality Assessment Based on Binocular Visual Characteristics and Depth Perception

The quality prediction of stereo images has great challenges without reference images. In this paper, we propose a novel no-reference stereo image quality assessment (NR-SIQA) model based on binocular visual characteristics and depth perception, which can effectively evaluate the quality of symmetric distortion and asymmetric distortion images. To be specific, we discriminate the different binocular behaviors by analyzing binocular visual characteristics, and construct the corresponding cyclopean view instead of single cyclopean view to simulate different binocular behaviors. Then, we extract monocular and binocular visual features from the left view, the right view and the synthetic cyclopean view. Furthermore, in order to evaluate the depth quality of the stereo image accurately, we extract the depth perception features from the weighted disparity map and the longitudinal correlation coefficient map. Finally, we construct the mapping relationship model from quality perception feature domain to quality score domain by training an adaptive enhancement algorithm based on support vector regression (SVR). We evaluate the performance of the proposed algorithm on four stereo image databases. The experimental results show that compared with the state-of-the-art full reference(FR), reduced reference(RR) and NR-SIQA algorithms, the proposed algorithm achieves highly competitive performance for both symmetric and asymmetric distortions.

KonIQ-10k: An ecologically valid database for deep learning of blind image quality assessment.

Deep learning methods for image quality assessment (IQA) are limited due to the small size of existing datasets. Extensive datasets require substantial resources both for generating publishable content and annotating it accurately. We present a systematic and scalable approach to creating KonIQ-10k, the largest IQA dataset to date, consisting of 10,073 quality scored images. It is the first in-the-wild database aiming for ecological validity, concerning the authenticity of distortions, the diversity of content, and quality-related indicators. Through the use of crowdsourcing, we obtained 1.2 million reliable quality ratings from 1,459 crowd workers, paving the way for more general IQA models. We propose a novel, deep learning model (KonCept512), to show an excellent generalization beyond the test set (0.921 SROCC), to the current state-of-the-art database LIVE-in-the-Wild (0.825 SROCC). The model derives its core performance from the InceptionResNet architecture, being trained at a higher resolution than previous models (512 × 384). Correlation analysis shows that KonCept512 performs similar to having 9 subjective scores for each test image.

End-to-End Image Patch Quality Assessment for Image/Video With Compression Artifacts

In this paper, we present an experimental image quality assessment (IQA) method for image/video patches with compression artifacts. Using the High Efficiency Video Coding (HEVC) standard, we create a new database of image patches with compression artifacts. Then, we conduct a completed subjective testing process to obtain the ‘ground truth’ quality scores for the mentioned database. Finally, we employ an end-to-end learning method to estimate the IQA model for the patches with HEVC compression artifacts. In such proposed method, a modified convolutional neural network (CNN) architecture is exploited for feature extraction while an adaptive moment estimation optimizer solution is used to perform the training process. Experimental results show that the proposed end-to-end IQA method significantly outperforms the relevant IQA benchmarks, especially when the compression artifacts are strongly realized in image/video patches. The proposed IQA method is expected to drive a new set of image/video compression solutions in future image/video coding and transmissions.

Blind Stereoscopic Image Quality Assessment Accounting for Human Monocular Visual Properties and Binocular Interactions

Human visual perceptual model is a key factor for evaluating stereoscopic image quality. This paper focuses on the contributions of monocular and binocular properties on quality perception and proposes a novel blind stereoscopic image quality assessment model by comprehensively digging the relationship between visual features and quality perception. The statistical quality-aware monocular features are extracted from both left view and right view to reveal monocular quality perception, including the color statistical features which are missed in most previous models, while the multiple features of the summation signal and the entropy features of the difference signal are extracted to quantify the binocular quality perception. Finally, support vector regression (SVR) is utilized to train a regression model based on the extracted features and the subjective scores. Three public databases, LIVE 3D Phase I, LIVE 3D Phase II, and MCL 3D Database, are adopted to prove the effectiveness of the proposed model. Experimental results demonstrate that the proposed model is superior to other existing state-of-the-art quality metrics.

Screen content image quality assessment based on convolutional neural networks

Screen content image (SCI) is a composite image including textual and pictorial regions resulting in many difficulties in image quality assessment (IQA). Large SCIs are divided into image patches to increase training samples for CNN training of IQA model, and this brings two problems: (1) local quality of each image patch is not equal to subjective differential mean opinion score (DMOS) of an entire image; (2) importance of different image patches is not same for quality assessment. In this paper, we propose a novel no-reference (NR) IQA model based on the convolutional neural network (CNN) for assessing the perceptual quality of SCIs. Our model conducts two designs solving problems which benefits from two strategies. For the first strategy, to imitate full-reference (FR) CNN-based model behavior, a CNN-based model is designed for both FR and NR IQA, and performance of NR-IQA part improves when the image patch scores predicted by FR-IQA part are adopted as the ground-truth to train NR-IQA part. For the second strategy, image patch qualities of one entire SCI are fused to obtain the SCI quality with an adaptive weighting method taking account the effect of the different image patch contents. Experimental results verify that our model outperforms all test NR IQA methods and most FR IQA methods on the screen content image quality assessment database (SIQAD). On the cross-database evaluation, the proposed method outperforms the existing NR IQA method in terms of at least 2.4 percent in PLCC and 2.8 percent in SRCC, which shows high generalization ability and high effectiveness of our model.

No-reference screen content image quality assessment based on multi-region features

Unlike natural image captured with cameras, screen content image (SCI) is a composite image including textual and pictorial regions. The different characteristics lead to many difficulties in image quality assessment (IQA). Most existing models based on the convolutional neural network (CNN) divide large SCIs into image patches to increase training samples for CNN training. This brings two problems: (1) a single image patch can not represent the quality of the entire image, especially in IQA of SCI; (2) SCI patches of an entire image degraded by the same distortion type and strength may have drastically different quality. In addition, these models adopt the mean square error (MSE) between the predicted quality and the subjective differential mean opinion score (DMOS) to train the CNN, without considering quality ranking between different SCIs. In this paper, we propose a novel no-reference (NR) IQA model based on the convolutional neural network (CNN). The contributions of our algorithm can be concluded as follows: (1) considering a large difference exists in different regions in a SCI, the pseudo global features generated with multi-region local features are utilized for quality evaluation, which better reflect image quality than local features of each image patch; (2) the noise classification task is used as an auxiliary task which aids the quality score prediction task to improve the representation ability; (3) the Siamese networks are used to predict the quality scores of two different SCIs, and a new ranking loss is proposed to rank the predicted scores, aiming to enhance the ability of the model to rank image in terms of quality. Experimental results verify that our model outperforms all test NR IQA methods and full-reference (FR) IQA methods on the screen content image quality assessment database (SIQAD).

No-Reference Image Quality Assessment Method Based on Visual Parameters

Recent studies on no-reference image quality assessment (NR-IQA) methods usually learn to evaluate the image quality by regressing from human subjective scores of the training samples. This study presented an NR-IQA method based on the basic image visual parameters without using human scored image databases in learning. We demonstrated that these features comprised the most basic characteristics for constructing an image and influencing the visual quality of an image. In this paper, the definitions, computational method, and relationships among these visual metrics were described. We subsequently proposed a no-reference assessment function, which was referred to as a visual parameter measurement index (VPMI), based on the integration of these visual metrics to assess image quality. It is established that the maximum of VPMI corresponds to the best quality of the color image. We verified this method using the popular assessment database—image quality assessment database (LIVE), and the results indicated that the proposed method matched better with the subjective assessment of human vision. Compared with other image quality assessment models, it is highly competitive. VPMI has low computational complexity, which makes it promising to implement in real-time image assessment systems.

Human Visual Perception-Based Multi-Exposure Fusion Image Quality Assessment

Compared with ordinary single exposure images, multi-exposure fusion (MEF) images are prone to color imbalance, detail information loss and abnormal exposure in the process of combining multiple images with different exposure levels. In this paper, we proposed a human visual perception-based multi-exposure fusion image quality assessment method by considering the related perceptual features (i.e., color, dense scale invariant feature transform (DSIFT) and exposure) to measure the quality degradation accurately, which is closely related to the symmetry principle in human eyes. Firstly, the L1 norm of chrominance components between fused images and the designed pseudo images with the most severe color attenuation is calculated to measure the global color degradation, and the color saturation similarity is added to eliminate the influence of color over-saturation. Secondly, a set of distorted images under different exposure levels with strong edge information of fused image is constructed through the structural transfer, thus DSIFT similarity and DSIFT saturation are computed to measure the local detail loss and enhancement, respectively. Thirdly, Gauss exposure function is used to detect the over-exposure or under-exposure areas, and the above perceptual features are aggregated with random forest to predict the final quality of fused image. Experimental results on a public MEF subjective assessment database show the superiority of the proposed method with the state-of-the-art image quality assessment models.