No-reference Image Quality Assessment Metrics Research Articles

QMRNet: Quality Metric Regression for EO Image Quality Assessment and Super-Resolution

The latest advances in super-resolution have been tested with general-purpose images such as faces, landscapes and objects, but mainly unused for the task of super-resolving earth observation images. In this research paper, we benchmark state-of-the-art SR algorithms for distinct EO datasets using both full-reference and no-reference image quality assessment metrics. We also propose a novel Quality Metric Regression Network (QMRNet) that is able to predict the quality (as a no-reference metric) by training on any property of the image (e.g., its resolution, its distortions, etc.) and also able to optimize SR algorithms for a specific metric objective. This work is part of the implementation of the framework IQUAFLOW, which has been developed for the evaluation of image quality and the detection and classification of objects as well as image compression in EO use cases. We integrated our experimentation and tested our QMRNet algorithm on predicting features such as blur, sharpness, snr, rer and ground sampling distance and obtained validation medRs below 1.0 (out of N = 50) and recall rates above 95%. The overall benchmark shows promising results for LIIF, CAR and MSRN and also the potential use of QMRNet as a loss for optimizing SR predictions. Due to its simplicity, QMRNet could also be used for other use cases and image domains, as its architecture and data processing is fully scalable.

No-reference perceptual CT image quality assessment based on a self-supervised learning framework

Accurate image quality assessment (IQA) is crucial to optimize computed tomography (CT) image protocols while keeping the radiation dose as low as reasonably achievable. In the medical domain, IQA is based on how well an image provides a useful and efficient presentation necessary for physicians to make a diagnosis. Moreover, IQA results should be consistent with radiologists’ opinions on image quality, which is accepted as the gold standard for medical IQA. As such, the goals of medical IQA are greatly different from those of natural IQA. In addition, the lack of pristine reference images or radiologists’ opinions in a real-time clinical environment makes IQA challenging. Thus, no-reference IQA (NR-IQA) is more desirable in clinical settings than full-reference IQA (FR-IQA). Leveraging an innovative self-supervised training strategy for object detection models by detecting virtually inserted objects with geometrically simple forms, we propose a novel NR-IQA method, named deep detector IQA (D2IQA), that can automatically calculate the quantitative quality of CT images. Extensive experimental evaluations on clinical and anthropomorphic phantom CT images demonstrate that our D2IQA is capable of robustly computing perceptual image quality as it varies according to relative dose levels. Moreover, when considering the correlation between the evaluation results of IQA metrics and radiologists’ quality scores, our D2IQA is marginally superior to other NR-IQA metrics and even shows performance competitive with FR-IQA metrics.

A Novel Image Focus Metric based on Power of Companion Matrix and Gerschgorin Circle Bound

ABSTRACT In this paper, a simple and efficient no-reference image quality assessment metric is proposed. It is based on the concept of polynomial coefficients, power of companion matrix, and Gerschgorin circles bound. The polynomial coefficient-based companion matrix captures the main features and dynamics of an image. Moreover, the Gerschgorin circles bound is used to define the focus metric. The proposed focus metric is tested on various real as well as synthetic image data sets. It is observed that the presented metric is unimodal to noise and invariant to the contrast changes that occur due to the variation in illumination effect. Moreover, it is robust under the varying salt-and-pepper and Gaussian noise. The performance of the proposed focus metric is also compared with the various existing focus metrics.

Dual-quality map based no reference image quality assessment using deformable convolution

Traditional feature-based no-reference image quality assessment (NR-IQA) technologies rely heavily on the correlation between the handcrafted features and distortions, and often fail to predict perceptual quality because of the diversity of image distortion types. End-to-end deep learning-based metrics are driven mainly by human-labeled data, which are prohibitively labor-expensive to collect. An NR-IQA method that imitates the hierarchical and inferential perceptual behavior of the human vision system (HVS) is proposed to address the challenges of distortion diversity and lack of labeled data. In the proposed method, first, a dual-quality map is generated for use as an intermediate target, which contains multiscale features that are highly related to image quality. To capture human eye-sensitive distortion information in distorted images, a dual attention mechanism is utilized in the quality map generation network. Then, a regression network is designed for mapping the intermediate target to a quality score. The deformable convolution which has adaptive receptive field is introduced to extract multiple scale features from the dual-quality map. Experiments on eight benchmark databases show the superiority of the proposed method over other state-of-the-art NR-IQA metrics.

A comprehensive survey: Image deraining and stereo‐matching task‐driven performance analysis

Deraining has been attracting a lot of attention from researchers, and various methods have been proposed, especially deep-networks are widely adopted in recent years. Their structures and learning become more and more complicated and diverse, making it difficult to analyze the contributions and improvements. In this paper, a comprehensive review for current rain removal methods is first provided to show their contributions. Specifically, they are reviewed in terms of handing rain streaks and rain mist. Second, besides evaluating their rain removal ability, they are also evaluated in terms of their impact on subsequent stereo-matching task. To this end, a new deraining dataset is first prepared, called Rain-Kitti2012 and Rain-Kitti2015. They are created by adding rain part to clean image-pairs in Kitti2012 and Kitti2015. By then, nine state-of-the-art deraining methods are evaluated with full-reference and no-reference image quality assessment metrics. Furthermore, the blurriness and distortion types introduced during deraining are measured. Finally, three learning-based stereo matching methods are compared, and they take the outputs of deraining methods as inputs. It is further discussed how derained images influence the accuracy of stereo matching, which can provide some insight for jointly handling rain removal and stereo matching. 1: A comprehensive review for the current rain removal methods is provided. They are categorized into rain-streak-oriented and rain-mist-oriented approaches in terms of degradation type, and are categorized into model-driven and data-driven approaches in terms of methodology. 2: A new image deraining dataset is introduced, which is the first dataset that can be used to perform stereo-matching-driven evaluation for deraining methods. The dataset is created by adding rain part to clean images in KITTI2012 and KITTI2015. 3: We evaluate 9 deep learning based deraining methods with full-reference and no- reference metrics. In addition, the types of distortions produced by these methods are discussed and measured quantitatively. And, the impact of 9 deraining methods on the subsequent stereo matching task is evaluated, which can provide some insight on how to design stereo matching task-driven deraining methods.

A No-Reference Image Quality Assessment Metric for Wood Images

Image Quality Assessment (IQA) is a vital element in improving the efficiency of an automatic recognition system of various wood species. There is a need to develop a No-Reference IQA (NR-IQA) system as a perfect and distortion free wood images may be impossible to be acquired in the dusty environment in timber factories. To the best of our knowledge, there is no NR-IQA developed for wood images specifically. Therefore, a Gray Level Co-Occurrence Matrix (GLCM) and Gabor features-based NR-IQA (GGNR-IQA) metric is proposed to assess the quality of wood images. The proposed metric is developed by training the support vector machine regression with GLCM and Gabor features calculated for wood images together with scores obtained from subjective evaluation. The proposed IQA metric is compared with a widely used NR-IQA metric, Blind/Referenceless Image Spatial Quality Evaluator (BRISQUE) and Full Reference-IQA (FR-IQA) metrics. Results shows that the proposed NR-IQA metric outperforms the BRISQUE and the FR-IQA metrics. Moreover, the proposed NR-IQA metric is beneficial in wood industry as a distortion free reference image is not needed to evaluate the wood images.

Evaluation of Rounding Functions in Nearest Neighbor Interpolation

A novel evaluation study of the most appropriate round function for nearest neighbor image interpolation is presented. Evaluated rounding functions are selected among the five rounding rules defined by the Institute of Electrical and Electronics Engineers IEEE 754-2008 standard. Both full- and no-reference image quality assessment metrics are used to evaluate the influence of rounding functions on nearest neighbor interpolation image quality. The concept of the number of occurrences is used to determine the percentage of achieved occurrences, which is also equivalent to the sample proportion percentage used to calculate the margin of error. Inferential statistical analysis is used to deduce from a small number of images and draw a conclusion of the performance of each rounding function on a big number of images. Experimental results are provided to demonstrate with 95% confidence the minimum performance percentage of the most appropriate rounding function.

Multi-focus Image Fusion for Confocal Microscopy Using U-Net Regression Map.

Characterizing the spatial relationship between blood vessel and lymphatic vascular structures, in the mice dura mater tissue, is useful for modeling fluid flows and changes in dynamics in various disease processes. We propose a new deep learning-based approach to fuse a set of multi-channel single-focus microscopy images within each volumetric z-stack into a single fused image that accurately captures as much of the vascular structures as possible. The red spectral channel captures small blood vessels and the green fluorescence channel images lymphatics structures in the intact dura mater attached to bone. The deep architecture Multi-Channel Fusion U-Net (MCFU-Net) combines multi-slice regression likelihood maps of thin linear structures using max pooling for each channel independently to estimate a slice-based focus selection map. We compare MCFU-Net with a widely used derivative-based multi-scale Hessian fusion method [8]. The multi-scale Hessian-based fusion produces dark-halos, non-homogeneous backgrounds and less detailed anatomical structures. Perception based no-reference image quality assessment metrics PIQUE, NIQE, and BRISQUE confirm the effectiveness of the proposed method.

Objective quality assessment of synthesized images by local variation measurement

Due to the rapid development of free-viewpoint television (FVT), Depth-Image-Based Rendering (DIBR) technology has been widely used to synthesize images of virtual view-points. However, the types of distortions in the synthesized images are different from those of natural images, such as discontinuity, flickering, stretching, etc. To measure the distortion occurred in the synthesized images, we propose a full-reference (FR) quality assessment method by local variation measurement consisting of three-modules. Firstly, since the distortion in the synthesized image mainly occurs in the region with high-frequency structure information, the Neutrosophic domain is employed to evaluate the degradation of local image structure. Secondly, by considering that the texture of the synthesized image might be damaged due to the warping of 2D image or the loss of information in the occlusion region, we evaluate the visual quality of local texture by using the features obtained from frequency domain. Thirdly, to measure the stretching distortion which is unique in the synthesized image, the visual quality of extracted stretching area is measured by entropy. Finally, a pooling operation is used to combine the quality scores of the three modules to obtain the final predicted quality score. Experimental results show that the performance of the proposed algorithm is competitive with state-of-the-art FR and no-reference image quality assessment metrics.

Towards Automatic Image Exposure Level Assessment

The quality of acquired images can be surely reduced by improper exposures. Thus, in many vision-related industries, such as imaging sensor manufacturing and video surveillance, an approach that can routinely and accurately evaluate exposure levels of images is in urgent need. Taking an image as input, such a method is expected to output a scalar value, which can represent the overall perceptual exposure level of the examined image, ranging from extremely underexposed to extremely overexposed. However, studies focusing on image exposure level assessment (IELA) are quite sporadic. It should be noted that blind NR-IQA (no-reference image quality assessment) algorithms or metrics used to measure the quality of contrast-distorted images cannot be used for IELA. The root reason is that though these algorithms can quantify quality distortion of images, they do not know whether the distortion is due to underexposure or overexposure. This paper aims to resolve the issue of IELA to some extent and contributes to two aspects. Firstly, an Image Exposure Database (IEpsD) is constructed to facilitate the study of IELA. IEpsD comprises 24,500 images with various exposure levels, and for each image a subjective exposure score is provided, which represents its perceptual exposure level. Secondly, as IELA can be naturally formulated as a regression problem, we thoroughly evaluate the performance of modern deep CNN architectures for solving this specific task. Our evaluation results can serve as a baseline when the other researchers develop even more sophisticated IELA approaches. To facilitate the other researchers to reproduce our results, we have released the dataset and the relevant source code at https://cslinzhang.github.io/imgExpo/.

Blind Omnidirectional Image Quality Assessment With Viewport Oriented Graph Convolutional Networks

Quality assessment of omnidirectional images has become increasingly urgent due to the rapid growth of virtual reality applications. Different from traditional 2D images and videos, omnidirectional contents can provide consumers with freely changeable viewports and a larger field of view covering the $360^{\circ }\times 180^{\circ }$ spherical surface, which makes the objective quality assessment of omnidirectional images more challenging. In this paper, motivated by the characteristics of the human vision system (HVS) and the viewing process of omnidirectional contents, we propose a novel Viewport oriented Graph Convolution Network (VGCN) for blind omnidirectional image quality assessment (IQA). Generally, observers tend to give the subjective rating of a 360-degree image after passing and aggregating different viewports information when browsing the spherical scenery. Therefore, in order to model the mutual dependency of viewports in the omnidirectional image, we build a spatial viewport graph. Specifically, the graph nodes are first defined with selected viewports with higher probabilities to be seen, which is inspired by the HVS that human beings are more sensitive to structural information. Then, these nodes are connected by spatial relations to capture interactions among them. Finally, reasoning on the proposed graph is performed via graph convolutional networks. Moreover, we simultaneously obtain global quality using the entire omnidirectional image without viewport sampling to boost the performance according to the viewing experience. Experimental results demonstrate that our proposed model outperforms state-of-the-art full-reference and no-reference IQA metrics on two public omnidirectional IQA databases.

No-reference image quality assessment of authentically distorted images with global and local statistics

The development of digital image processing techniques requires reliable image quality assessment (IQA) methods. Since images acquired by a camera often contain various distortions and their non-distorted versions are not available, a no-reference IQA (NR-IQA) technique should be used. Many popular methods are developed to assess artificially distorted images, available in benchmark databases. In this paper, a new large benchmark database, containing naturally distorted images captured with a digital camera, is introduced along with a new NR-IQA metric. The method uses a wide spectrum of local and global image features and their statistics to address a diversity of distortions. Among 80 employed features, 56 are introduced to the IQA for the first time, while the remaining statistics are used to further improve the quality prediction performance of the method. The obtained perceptual feature vector is used to provide a quality model with support vector regression technique. The experimental comparison of the method with the state-of-the-art IQA measures on the database reveals its superiority in terms of correlation with human scores.

No-reference quality assessment for image-based assessment of economically important tropical woods.

Image Quality Assessment (IQA) is essential for the accuracy of systems for automatic recognition of tree species for wood samples. In this study, a No-Reference IQA (NR-IQA), wood NR-IQA (WNR-IQA) metric was proposed to assess the quality of wood images. Support Vector Regression (SVR) was trained using Generalized Gaussian Distribution (GGD) and Asymmetric Generalized Gaussian Distribution (AGGD) features, which were measured for wood images. Meanwhile, the Mean Opinion Score (MOS) was obtained from the subjective evaluation. This was followed by a comparison between the proposed IQA metric, WNR-IQA, and three established NR-IQA metrics, namely Blind/Referenceless Image Spatial Quality Evaluator (BRISQUE), deepIQA, Deep Bilinear Convolutional Neural Networks (DB-CNN), and five Full Reference-IQA (FR-IQA) metrics known as MSSIM, SSIM, FSIM, IWSSIM, and GMSD. The proposed WNR-IQA metric, BRISQUE, deepIQA, DB-CNN, and FR-IQAs were then compared with MOS values to evaluate the performance of the automatic IQA metrics. As a result, the WNR-IQA metric exhibited a higher performance compared to BRISQUE, deepIQA, DB-CNN, and FR-IQA metrics. Highest quality images may not be routinely available due to logistic factors, such as dust, poor illumination, and hot environment present in the timber industry. Moreover, motion blur could occur due to the relative motion between the camera and the wood slice. Therefore, the advantage of WNR-IQA could be seen from its independency from a "perfect" reference image for the image quality evaluation.

No-reference image quality assessment focusing on human facial region

A person’s face is the part that we are most interested in when we look at images. If the quality of the human facial regions in focus is poor, even if the quality of other regions is quite better, we are likely to badly evaluate the quality of the image. Thus, to exactly predict perceptual quality of images, the facial regions have to be specially tackled. However, the previous image quality assessment (IQA) methods did not specifically address the facial area.In this paper, we propose a no-reference (NR) IQA metric that uses global features of an image and local features extracted from detected facial regions by a convolutional neural network (CNN) based tiny face detector. Two regressors are used for the global features and the local facial features, respectively. The computed two scores are combined into a total quality score. Promising six IQA databases are utilized for experiments, and it is demonstrated that the performance of the proposed algorithm is competitive with the previous 2D NR IQA metrics.

Blind quality assessment metric and degradation classification for degraded document images

Epochal documents suffer from several types of noises that accumulate and evolve over time. This significantly affects their quality and makes their storage and the interpretation of their visual content problematic. Digital preservation seems the most viable and the most promising. Moreover, measuring the amount of degradation and quality assessment of degraded documents is highly desirable for applications such as selecting the proper algorithms for enhancement and analysis of document images, filtering the damaged images, tuning the processing algorithms parameters, document repairing, psychological study, etc. The first contribution of this work is the proposition of an efficient Multi-distortion Document Quality Measure (MDQM) for quality assessment of physically degraded document images. The proposed MDQM metric is based on three sets of spatial and frequency image features. These features are extracted from two layers of text and non-text and mapped to the mean opinion scores (MOS) using the regression function. The second contribution of this work is to estimate the probability of four common document image distortion types, namely, paper translucency, stain, readers annotations and worn holes in the degraded images. In our experiment, the correlations of seven no-reference image quality assessment (NR-IQA) metrics with the MOS values are evaluated on two available datasets. It is shown that the performance of MDQM metric is significantly better than the state-of-the-art NR-IQA metrics. Moreover, the experimental results demonstrate that MDQM metric not only leads to high efficacy for classification of the various degradations but also maintains a remarkable run-time efficiency. It is worth to mention that the proposed method has been conducted for Arabic documents.

No-reference quality assessment for contrast-altered images using an end-to-end deep framework

No-reference image quality assessment (NR-IQA) aims to predict image quality consistently with subjective scores with no prior knowledge of reference images. However, contrast distortion, which is an uncommon distortion, has been largely overlooked. To address this issue, we explore the NR-IQA metric by predicting the quality of contrast-altered images, using deep-learning techniques. We adopt a two-stage training strategy due to a gap between the deep learning’s sample requirements and the insufficiency of samples in the IQA domain. A deep convolutional neural network (CNN) is first designed and is pretrained to the classification task with the help of an additional synthetic contrast-distorted dataset. Then, the pretrained CNN is fine-tuned on the target IQA dataset using an end-to-end training approach. An effective pooling method is employed to map the image representation into a subjective quality score during the fine-tuning stage. Experimental results on five public IQA databases containing contrast-altered images show that the proposed method achieves competitive results and has good generalization ability compared to other NR-IQA methods.

Improving the Sharpness of Digital Images Using a Modified Laplacian Sharpening Technique

Many imaging systems produce images with deficient sharpness due to different real limitations. Hence, various image sharpening techniques have been used to improve the acutance of digital images. One of such is the well-known Laplacian sharpening technique. When implementing the basic Laplacian technique for image sharpening, two main drawbacks were detected. First, the amount of introduced sharpness cannot be increased or decreased. Second, in many situations, the resulted image suffers from a noticeable increase in brightness around the sharpened edges. In this article, an improved version of the basic Laplacian technique is proposed, wherein it contains two key modifications of weighting the Laplace operator to control the introduced sharpness and tweaking the second order derivatives to provide adequate brightness for recovered edges. To perform reliable experiments, only real-degraded images were used, and their accuracies were measured using a specialized no-reference image quality assessment metric. From the obtained experimental results, it is evident that the proposed technique outperformed the comparable techniques in terms of recorded accuracy and visual appearance.

A No-Reference Image Quality Assessment Metric by Multiple Characteristics of Light Field Images

Evaluation of light field image (LFI), especially micro-lens camera light field (LF), is a new and challenging work. The development of image quality assessment (IQA) metric of LFIs relies on the subjective quality assessment database. In this paper, we establish a perceptual quality assessment dataset consisting of 240 distorted images from 8 source images with five distortion types. Furthermore, a no-reference IQA metric is proposed by combining 2D and 3D characteristics of LFI with the Support Vector Regression (SVR) model. The performance of the proposed metric is demonstrated by comparing with some classical full reference IQA metrics both on the presented dataset and a third-party dataset. The experiment results show that our method has a better performance than others.

Blind quality assessment for screen content images by combining local and global features

Recently, several no-reference image quality assessment (NR-IQA) metrics have been developed for the quality evaluation of screen content images (SCIs). While, most of them are opinion-aware methods, which are limited by the subjective opinion scores of training data. Hence, in this paper, we propose a novel opinion-unaware method to predict the quality of SCIs without any prior information. Firstly, an union feature is proposed by considering the local and global visual characteristics of human visual system simultaneously. Specifically, a local structural feature is extracted from the rough and smooth regions of SCIs by leveraging a sparse representation model. As a supplement, a global feature is obtained by combining the luminance statistical feature and local binary pattern (LBP) feature of entire SCIs. Secondly, to get rid of the limitation of subjective opinion scores, a new large-scale training dataset contained 80,000 distorted SCIs is constructed, and the quality labels of those distorted SCIs are derived by an advanced full-reference IQA metric. Thirdly, a regression model between image features and image quality labels is learned from the training dataset by employing a learning-based framework. And then, the quality scores of test SCIs can be predicted by the pre-trained regression model. The experimental results on two largest SCI-oriented databases show that the proposed method is superior to the state-of-the-art NR-IQA metrics.

Blind Quality Assessment Based on Pseudo-Reference Image

Traditional full-reference image quality assessment (IQA) metrics generally predict the quality of the distorted image by measuring its deviation from a perfect quality image called reference image. When the reference image is not fully available, the reduced-reference and no-reference IQA metrics may still be able to derive some characteristics of the perfect quality images, and then measure the distorted image's deviation from these characteristics. In this paper, contrary to the conventional IQA metrics, we utilize a new “reference” called pseudo-reference image (PRI) and a PRI-based blind IQA (BIQA) framework. Different from a traditional reference image, which is assumed to have a perfect quality, PRI is generated from the distorted image and is assumed to suffer from the severest distortion for a given application. Based on the PRI-based BIQA framework, we develop distortion-specific metrics to estimate blockiness, sharpness, and noisiness. The PRI-based metrics calculate the similarity between the distorted image's and the PRI's structures. An image suffering from severer distortion has a higher degree of similarity with the corresponding PRI. Through a two-stage quality regression after a distortion identification framework, we then integrate the PRI-based distortion-specific metrics into a general-purpose BIQA method named blind PRI-based (BPRI) metric. The BPRI metric is opinion-unaware (OU) and almost training-free except for the distortion identification process. Comparative studies on five large IQA databases show that the proposed BPRI model is comparable to the state-of-the-art opinion-aware- and OU-BIQA models. Furthermore, BPRI not only performs well on natural scene images, but also is applicable to screen content images. The MATLAB source code of BPRI and other PRI-based distortion-specific metrics will be publicly available.