No-reference Image Quality Assessment Research Articles

QL-IQA: Learning distance distribution from quality levels for blind image quality assessment

Recently, blind image quality assessment (BIQA) has been intensively studied with deep learning. However, the limited quality-annotated datasets restrict its further development. Although patch-based methods have been leveraged to generate more training data, they usually assign the image quality score to all patches in an image. Consequently, much noise would be introduced. To avoid this, we propose a method learning distance distribution from quality levels to expand the training data, which is pseudo Siamese network-based no-reference image quality assessment (NR-IQA). Specifically, we firstly use the K-means clustering algorithm to classify the quality scores of distorted images into five levels. Subsequently, the pseudo Siamese network is adopted to learn the distance distribution from these quality levels. We mainly decrease distances between the quality scores of images from the same quality level and increase distances between the quality scores of images from different quality levels, in which the distorted images can be compared across different distortion types. Finally, we introduce a fusion layer to average the quality scores learnt by the two branches of the trained pseudo Siamese network, and take fine-tuning on this basis to learn image quality score from one image. To demonstrate the effectiveness of the proposed approach, we evaluate it on benchmark databases (LIVE, TID2013 and LIVE MD). The result shows superior performance to some widely used NR-IQA methods and full-reference IQA (FR-IQA) methods. Cross-database evaluation verifies high generalization ability and high effectiveness of our model.

A technical review of no-reference image quality assessment algorithms for contrast distorted images

Automatic image quality assessment similar to human vision perception is an essential process for real-time image processing applications to perform perceptual image assessments for effectively achieving their goals. As no-reference image quality assessment (NR-IQA) schemes perform perceptual assessments of images without any information about their original version, these algorithms suit real-time computer vision techniques because of the non-availability of reference images. Contrast and colorfulness play important roles in determining the quality of color images. By combining many IQA metrics, a number of combined metrics had been devised. This study provides an insight into major NR-IQA methods and their effectiveness in assessing contrast, colorfulness, and overall quality of contrast-degraded images with technical analysis. The effectiveness of top-ranking NR-IQA methods is experimentally assessed with benchmark assessment methods on images from benchmarked databases. The study provides insight into open research challenges in the area of NR-IQA for developing new promising methods by clearly demarcating the difficulties of top-ranking NR-IQA methods.

CNN-based no-reference video quality assessment method using a spatiotemporal saliency patch selection procedure

We propose a yet lightweight no-reference (NR) video quality assessment (VQA) method, which uses a convolution neural network (CNN) architecture. The proposed method implements a spatiotemporal saliency patch selection procedure that crops the frame into small nonoverlapping blocks of images (patches) and selects the most perceptually relevant ones. The selected patches are then forwarded to the CNN. To determine which patches are the most relevant, spatial and temporal saliency features are computed for each frame. The proposed method does not require subjective scores to train the CNN. It uses objective quality scores as target quality scores for each video frame, which are computed using an NR image quality assessment method. Given the lack of large annotated video quality databases, this is an advantage of the proposed method. Finally, although it has much smaller cost of data-processing, compared with other state-of-the-art methods, the proposed NR-VQA obtains robust and competitive results.

Precise No-Reference Image Quality Evaluation Based on Distortion Identification

The difficulty of no-reference image quality assessment (NR IQA) often lies in the lack of knowledge about the distortion in the image, which makes quality assessment blind and thus inefficient. To tackle such issue, in this article, we propose a novel scheme for precise NR IQA, which includes two successive steps, i.e., distortion identification and targeted quality evaluation. In the first step, we employ the well-known Inception-ResNet-v2 neural network to train a classifier that classifies the possible distortion in the image into the four most common distortion types, i.e., Gaussian white noise (WN), Gaussian blur (GB), jpeg compression (JPEG), and jpeg2000 compression (JP2K). Specifically, the deep neural network is trained on the large-scale Waterloo Exploration database, which ensures the robustness and high performance of distortion classification. In the second step, after determining the distortion type of the image, we then design a specific approach to quantify the image distortion level, which can estimate the image quality specially and more precisely. Extensive experiments performed on LIVE, TID2013, CSIQ, and Waterloo Exploration databases demonstrate that (1) the accuracy of our distortion classification is higher than that of the state-of-the-art distortion classification methods, and (2) the proposed NR IQA method outperforms the state-of-the-art NR IQA methods in quantifying the image quality.

RM-IQA: A new no-reference image quality assessment framework based on range mapping method

Significant progress has been made in recent years in image quality assessment (IQA). In particular, the development of deep learning has provided no-reference (NR)-IQA with more impressive solutions. However, improving the generalization of NR-IQA models is still an urgent necessity. In this study, we propose a new framework that uses the range mapping method to map an existing full-reference (FR)-IQA dataset to an NR-IQA dataset, thereby further enhancing the accuracy and generalization of the NR-IQA model. First, an NR-IQA model is employed to score an FR-IQA dataset to obtain the corresponding mean opinion score (MOS) values. Then, the correlation coefficients between these MOS values and the original differential mean opinion score (DMOS) values marked by the FR-IQA dataset itself is calculated. Subsequently, the matching sequence pair is obtained according to these correlation coefficients. Then, a range mapping function is selected based on this sequence pair, and this function is used to map the entire FR-IQA dataset to the existing NR-IQA dataset, and a new NR-IQA dataset is generated. Finally, the new and the existing NR-IQA datasets are merged into a new dataset, which can train an end-to-end multi-task network to obtain the final model RM-IQA. This model exhibits better performance as it exploits more prior information. Based on the largest available NR-IQA dataset KonIQ-10k and FR-IQA dataset KADID-10K, the experimental results proved the effectiveness of the proposed framework.

Analysis of Benford’s Law for No-Reference Quality Assessment of Natural, Screen-Content, and Synthetic Images

With the tremendous growth and usage of digital images, no-reference image quality assessment is becoming increasingly important. This paper presents in-depth analysis of Benford’s law inspired first digit distribution feature vectors for no-reference quality assessment of natural, screen-content, and synthetic images in various viewpoints. Benford’s law makes a prediction for the probability distribution of first digits in natural datasets. It has been applied among others for detecting fraudulent income tax returns, detecting scientific fraud, election forensics, and image forensics. In particular, our analysis is based on first digit distributions in multiple domains (wavelet coefficients, DCT coefficients, singular values, etc.) as feature vectors and the extracted features are mapped onto image quality scores. Extensive experiments have been carried out on seven large image quality benchmark databases. It has been demonstrated that first digit distributions are quality-aware features, and it is possible to reach or outperform the state-of-the-art with them.

A comprehensive survey: Image deraining and stereo‐matching task‐driven performance analysis

Deraining has been attracting a lot of attention from researchers, and various methods have been proposed, especially deep-networks are widely adopted in recent years. Their structures and learning become more and more complicated and diverse, making it difficult to analyze the contributions and improvements. In this paper, a comprehensive review for current rain removal methods is first provided to show their contributions. Specifically, they are reviewed in terms of handing rain streaks and rain mist. Second, besides evaluating their rain removal ability, they are also evaluated in terms of their impact on subsequent stereo-matching task. To this end, a new deraining dataset is first prepared, called Rain-Kitti2012 and Rain-Kitti2015. They are created by adding rain part to clean image-pairs in Kitti2012 and Kitti2015. By then, nine state-of-the-art deraining methods are evaluated with full-reference and no-reference image quality assessment metrics. Furthermore, the blurriness and distortion types introduced during deraining are measured. Finally, three learning-based stereo matching methods are compared, and they take the outputs of deraining methods as inputs. It is further discussed how derained images influence the accuracy of stereo matching, which can provide some insight for jointly handling rain removal and stereo matching. 1: A comprehensive review for the current rain removal methods is provided. They are categorized into rain-streak-oriented and rain-mist-oriented approaches in terms of degradation type, and are categorized into model-driven and data-driven approaches in terms of methodology. 2: A new image deraining dataset is introduced, which is the first dataset that can be used to perform stereo-matching-driven evaluation for deraining methods. The dataset is created by adding rain part to clean images in KITTI2012 and KITTI2015. 3: We evaluate 9 deep learning based deraining methods with full-reference and no- reference metrics. In addition, the types of distortions produced by these methods are discussed and measured quantitatively. And, the impact of 9 deraining methods on the subsequent stereo matching task is evaluated, which can provide some insight on how to design stereo matching task-driven deraining methods.

No‐reference image quality assessment based on multiscale feature representation

The no-reference image quality assessment (NR-IQA) method can evaluate the distortions in an image without the reference image. However, due to the diversity of the image contents and distortion types, it is hard for the existing NR-IQA algorithms to obtain competitive performance on both synthetically and authentically distorted images. To address the problem, a multiscale feature representation-based NR-IQA method that performs well for both synthetic and authentic distortions is proposed. This model consists of two parts: The feature extraction part and the feature fusion part. First, part of the Res2Net-50 network is chosen as the feature extraction part due to its high ability in increasing the range of receptive fields. Then, the feature fusion part consisting of a novel residual block and two fully connected layers is designed to fuse the extracted features and realize the quality score mapping. After a series of stepwise optimization experiments, the most competitive network architecture consisting of the feature extraction part and the feature fusion part is obtained. Comprehensive experiments on the LIVE, TID2013, CSIQ, KADID-10k, KonIQ-10K, and LIVE challenge databases demonstrate that the proposed method can work powerfully on both the synthetic and authentic distortions and also has a strong generalization ability.

No-reference stereo image quality assessment based on discriminative sparse representation

No-reference stereo image quality assessment based on discriminative sparse representation

Gauss-guided patch-based deep convolutional neural networks for no-reference image quality assessment

Patch-based deep convolutional neural network (DCNN) has been proved to have advanced performance in no-reference image quality assessment (NR-IQA). However, these methods generally take global quality score as the quality score of each patch mainly since local quality score is not provided. Unfortunately, the perceived quality of image patch is difficult to maintain a high degree of consistency. Thus, the use of the same global quality score in different patches of the same image may hinder training of DCNNs. In this paper, we propose a universal and nearly cost-free model called Gaussian Random Jitter (GRJ). According to the uncertainty of the perceived quality, GRJ divided the training images into high-confidence distorted images and low-confidence distorted images, and reasonably assigned different local quality scores to each patch through specific gaussian functions with the global quality score as the mean value and the undetermined hyperparameter as the standard deviation. We took one of the most advanced patch-based DCNNs models as backbone and tested the improved performance over three widely used image quality databases. We show that our model can further improve the performance of patch-based models and even help them comparable with those of state-of-the-art NR-IQA algorithms.

An efficient approach for no-reference image quality assessment based on statistical texture and structural features

This paper presents a rotation-invariant and computationally efficient no-reference image quality assessment (NR-IQA) model. It estimates the image quality based on texture and structural information associated with the images. The human visual system (HVS) uses perceptual features such as texture and structure as primary information to understand the visual scene and the image content. Moreover, the texture and structural information capture the loss of naturalness due to distortions in the image. Therefore, in this work, the important texture features are extracted using the local binary patterns (LBP).The modified LBP, also known as the hyper-smoothing LBP (H-LBP) and Laplacian of H-LBP (LH-LBP), represents the image structure. Further, the image quality prediction model computes the quality of the image based on the statistical feature measures of the texture and structural information. In the proposed approach, the image quality prediction model uses support vector regression (SVR) to measure image quality. Various experimentations are carried out on the LIVE and TID2013 database to test the effectiveness of the proposed NR-IQA model. The performance metrics such as Spearman rank-ordered correlation coefficient, Pearson linear correlation coefficient, and root mean square error is computed to show the efficiency of the presented approach. The experimental results illustrate a high correlation between the predicted quality score and the human visual perceptions. It is also found to be competitive with the best-performing full-reference and no-reference IQA models.

A No-Reference Image Quality Assessment Metric for Wood Images

Image Quality Assessment (IQA) is a vital element in improving the efficiency of an automatic recognition system of various wood species. There is a need to develop a No-Reference IQA (NR-IQA) system as a perfect and distortion free wood images may be impossible to be acquired in the dusty environment in timber factories. To the best of our knowledge, there is no NR-IQA developed for wood images specifically. Therefore, a Gray Level Co-Occurrence Matrix (GLCM) and Gabor features-based NR-IQA (GGNR-IQA) metric is proposed to assess the quality of wood images. The proposed metric is developed by training the support vector machine regression with GLCM and Gabor features calculated for wood images together with scores obtained from subjective evaluation. The proposed IQA metric is compared with a widely used NR-IQA metric, Blind/Referenceless Image Spatial Quality Evaluator (BRISQUE) and Full Reference-IQA (FR-IQA) metrics. Results shows that the proposed NR-IQA metric outperforms the BRISQUE and the FR-IQA metrics. Moreover, the proposed NR-IQA metric is beneficial in wood industry as a distortion free reference image is not needed to evaluate the wood images.

No-reference image quality assessment for contrast-changed images via a semi-supervised robust PCA model

Contrast plays an important role in human perception of image quality. In this paper, we propose a metric for no-reference quality assessment of contrast-changed images by using a novel semi-supervised robust PCA, which can realize feature selection and denoising simultaneously, guided by the available supervisory information. To select features adaptively, the information-oriented features (e.g. entropy and natural scene statistics) and appearance-oriented features (e.g. colorfulness) are adopted. The proposed model is formulated as a constraint optimization problem, which is further casted to a convex problem and solved via augmented Lagrangian multiplier method. Extensive experimental results on CCID2014, CSIQ, SIQAD and TID2013 databases show that the proposed semi-supervised image quality metric based on robust PCA (SIQMR) provides a more accurate prediction than other metrics on the human perception of contrast variations.

MoNET: no-reference image quality assessment based on a multi-depth output network

When deep convolutional neural networks perform feature extraction, the features computed at each layer express different abstractions of visual information. The earlier layers extract highly compact low-level features such as bandpass and directional primitives, whereas deeper layers extract structural features of increasing abstraction, similar to contours, shapes, and edges, becoming less effable as the depth increases. We propose a different kind of end-to-end no-reference (NR) image quality assessment (IQA) model, which is defined as a multi-depth output convolutional neural network (MoNET). It accomplishes this by mapping both shallow and deep features to perceived quality. MoNET delivers three outputs that express shallow (lower-level) and deep (high-level) features, and maps them to subjective quality scores. The multiple outputs are combined into a single, final quality score. MoNET does this by combining the responses of three learning machines, so it may be viewed as a form of ensemble learning. The experimental results on three public image quality databases show that our proposed model achieves better performance than other state-of-the-art NR IQA algorithms.

No-Reference Image Quality Assessment with Multi-Scale Orderless Pooling of Deep Features.

The goal of no-reference image quality assessment (NR-IQA) is to evaluate their perceptual quality of digital images without using the distortion-free, pristine counterparts. NR-IQA is an important part of multimedia signal processing since digital images can undergo a wide variety of distortions during storage, compression, and transmission. In this paper, we propose a novel architecture that extracts deep features from the input image at multiple scales to improve the effectiveness of feature extraction for NR-IQA using convolutional neural networks. Specifically, the proposed method extracts deep activations for local patches at multiple scales and maps them onto perceptual quality scores with the help of trained Gaussian process regressors. Extensive experiments demonstrate that the introduced algorithm performs favorably against the state-of-the-art methods on three large benchmark datasets with authentic distortions (LIVE In the Wild, KonIQ-10k, and SPAQ).

Subjective and Objective Quality Assessments of Display Products.

In recent years, people’s daily lives have become inseparable from a variety of electronic devices, especially mobile phones, which have undoubtedly become necessity in people’s daily lives. In this paper, we are looking for a reliable way to acquire visual quality of the display product so that we can improve the user’s experience with the display product. This paper proposes two major contributions: the first one is the establishment of a new subjective assessment database (DPQAD) of display products’ screen images. Specifically, we invited 57 inexperienced observers to rate 150 screen images showing the display product. At the same time, in order to improve the reliability of screen display quality score, we combined the single stimulation method with the stimulation comparison method to evaluate the newly created display products’ screen images database effectively. The second one is the development of a new no-reference image quality assessment (IQA) metric. For a given image of the display product, first our method extracts 27 features by analyzing the contrast, sharpness, brightness, etc., and then uses the regression module to obtain the visual quality score. Comprehensive experiments show that our method can evaluate natural scene images and screen content images at the same time. Moreover, compared with ten state-of-the-art IQA methods, our method shows obvious superiority on DPQAD.

A no‐reference blurred colourful image quality assessment method based on dual maximum local information

Images can be blurred due to the imperfection of the imaging system and blurriness is one of the challenging problems for image quality assessment (IQA). No-reference blurred IQA methods have been proposed in the literature to calculate image blurriness. Inspired by image processing-based auto-focussing and maximum local information theories, a no-reference blurred colourful IQA method based on Dual Maximum Local Information is proposed here. First, a window extraction method that combines the maximum gradient with local entropy is proposed to obtain the region of interest (ROI) for subsequent processing. Second, an improved maximum gradient method that leverages information from different channel images is presented to calculate the maximum gradient variation within the ROI for final sharpness score. Experimental results illustrated that the proposed method has better performance under various measurements compared with the state-of-the-art methods on LIVE, CSIQ, TID2008, TID2013, VCL@FER, IVC image databases.

No-Reference Quality Assessment for 3D Synthesized Images Based on Visual-Entropy-Guided Multi-Layer Features Analysis.

Multiview video plus depth is one of the mainstream representations of 3D scenes in emerging free viewpoint video, which generates virtual 3D synthesized images through a depth-image-based-rendering (DIBR) technique. However, the inaccuracy of depth maps and imperfect DIBR techniques result in different geometric distortions that seriously deteriorate the users’ visual perception. An effective 3D synthesized image quality assessment (IQA) metric can simulate human visual perception and determine the application feasibility of the synthesized content. In this paper, a no-reference IQA metric based on visual-entropy-guided multi-layer features analysis for 3D synthesized images is proposed. According to the energy entropy, the geometric distortions are divided into two visual attention layers, namely, bottom-up layer and top-down layer. The feature of salient distortion is measured by regional proportion plus transition threshold on a bottom-up layer. In parallel, the key distribution regions of insignificant geometric distortion are extracted by a relative total variation model, and the features of these distortions are measured by the interaction of decentralized attention and concentrated attention on top-down layers. By integrating the features of both bottom-up and top-down layers, a more visually perceptive quality evaluation model is built. Experimental results show that the proposed method is superior to the state-of-the-art in assessing the quality of 3D synthesized images.

Performance evaluation of median-modified Wiener filter algorithm in high-resolution complementary metal–oxide–semiconductor radio-magnetic X-ray imaging system: An experimental study

Noise reduction is a fundamental early stage in X-ray image performance evaluation. In this study, the median-modified Wiener filter (MMWF) algorithm based on a nonlinear adaptive spatial filter is compared to two general noise-reduction techniques using median and Wiener filters to confirm an excellent denoising approach in X-ray images. To acquire images, a high-resolution complementary metal–oxide–semiconductor (CMOS) radio-magnetic X-ray imaging system (exposure conditions: 100, 400, and 700μA) and rat phantom were used. The performances of the denoising methods were evaluated in terms of contrast-to-noise ratio (CNR), coefficient of variation (COV), and no-reference image quality assessment using the Blind/Referenceless Image Spatial Quality Evaluator (BRISQUE). The average results for CNR, COV, and BRISQUE in the acquired X-ray image using the MMWF algorithm were 1.08, 1.10, and 1.03 times higher than those of the noisy image, respectively. On average, the MMWF algorithm provided better image restoration than general noise-reduction techniques and was found to be most effective in relatively lower exposure conditions.

Contrast-distorted image quality assessment based on curvelet domain features

Contrast is one of the most popular forms of distortion. Recently, the existing image quality assessment algorithms (IQAs) works focusing on distorted images by compression, noise and blurring. Reduced-reference image quality metric for contrast-changed images (RIQMC) and no reference-image quality assessment (NR-IQA) for contrast-distorted images (NR-IQA-CDI) have been created for CDI. NR-IQA-CDI showed poor performance in two out of three image databases, where the pearson correlation coefficient (PLCC) were only 0.5739 and 0.7623 in TID2013 and CSIQ database, respectively. Spatial domain features are the basis of NR-IQA-CDI architecture. Therefore, in this paper, the spatial domain features are complementary with curvelet domain features, in order to take advantage of the potent properties of the curvelet in extracting information from images such as multiscale and multidirectional. The experimental outcome rely on K-fold cross validation (K ranged 2-10) and statistical test showed that the performance of NR-IQA-CDI rely on curvelet domain features (NR-IQA-CDI-CvT) significantly surpasses those which are rely on five spatial domain features.