Non-reference Image Quality Assessment Research Articles

DMvLNet: deep multiview learning network for blindly accessing image quality

In image quality assessment (IQA), the quality of an image is essentially dependent on both semantic contents and image distortions. Conventional nonreference IQA (NR-IQA) methods either encode the semantic contents only or combine them in a straightforward manner without exploring their intrinsic correlation. A unified deep multiview learning network (DMvLNet) is proposed to combine both semantic contents and image distortions for NR-IQA. Using ResNet50 as the backbone, we design IQA-aware semantic features instead of directly using a recognition-specific CNN model for IQA, as in many existing methods. To this end, a high-level feature refining module built on deep layers is developed for generating refined high-level feature maps. Meanwhile, shallow layers are employed for low-level descriptions. Thus, both the refined high-level and low-level features are used for characterizing semantic contents and image distortions, respectively. Furthermore, the refined semantic features and distortion features are simultaneously integrated into DMvLNet by exploring their mutual correlation, leading to the projected multiview embedding passing through the subsequent transformer network for image quality prediction. Extensive experiments conducted on both our assembled large-scale dataset and seven public benchmark datasets demonstrate the superiority of DMvLNet over the state-of-the-art methods.

Real‐world super‐resolution of face‐images from surveillance cameras

Most existing face image Super-Resolution (SR) methods assume that the Low-Resolution (LR) images were artificially downsampled from High-Resolution (HR) images with bicubic interpolation. This operation changes the natural image characteristics and reduces noise. Hence, SR methods trained on such data most often fail to produce good results when applied to real LR images. To solve this problem, a novel framework for the generation of realistic LR/HR training pairs is proposed. The framework estimates realistic blur kernels, noise distributions, and JPEG compression artifacts to generate LR images with similar image characteristics as the ones in the source domain. This allows to train an SR model using high-quality face images as Ground-Truth (GT). For better perceptual quality, a Generative Adversarial Network (GAN) based SR model is used, where the commonly used VGG-loss [1] is exchanged with LPIPS-loss [2]. Experimental results on both real and artificially corrupted face images show that our method results in more detailed reconstructions with less noise compared to the existing State-of-the-Art (SoTA) methods. In addition, it is shown that the traditional non-reference Image Quality Assessment (IQA) methods fail to capture this improvement and demonstrate that the more recent NIMA metric [3] correlates better with human perception via Mean Opinion Rank (MOR).

Medical image super-resolution using a relativistic average generative adversarial network

The medical imaging technique, e.g., positron emission tomography (PET), X-ray computed tomography (CT), and magnetic resonance imaging (MRI) is essential for clinical diagnosis and nuclear medicine. However, due to the hardware limitations of scanners, it is always clinically challenging to obtain high-resolution (HR) medical images. With the development of artificial intelligence, image super-resolution has been an effective technique to enhance the spatial resolution of medical images. In this paper, we propose a novel medical image super-resolution method using a relativistic average generative adversarial network (GAN), which consists of a generator and a discriminator for enhancing medical imaging quality in terms of both numerical criteria and visual results. The generator is trained to reconstruct HR images according to low-resolution (LR) counterparts. In contrast, the discriminator is trained to discriminate the probability of whether real HR images are more realistic than reconstructed images, further enhancing visual results. We apply our proposed method to two different public medical datasets, and experimental results show that our proposed method outperforms in terms of visual results, peak signal-to-noise ratio (PSNR), structural similarity index (SSIM), model complexity and an additional non-reference image quality assessment metric, compared with other state-of-the-art medical image super-resolution methods.

An Underwater Image Enhancement Algorithm Based on MSR Parameter Optimization

The quality of underwater images is often affected by the absorption of light and the scattering and diffusion of floating objects. Therefore, underwater image enhancement algorithms have been widely studied. In this area, algorithms based on Multi-Scale Retinex (MSR) represent an important research direction. Although the visual quality of underwater images can be improved to some extent, the enhancement effect is not good due to the fact that the parameters of these algorithms cannot adapt to different underwater environments. To solve this problem, based on classical MSR, we propose an underwater image enhancement optimization (MSR-PO) algorithm which uses the non-reference image quality assessment (NR-IQA) index as the optimization index. First of all, in a large number of experiments, we choose the Natural Image Quality Evaluator (NIQE) as the NR-IQA index and determine the appropriate parameters in MSR as the optimization object. Then, we use the Gravitational Search Algorithm (GSA) to optimize the underwater image enhancement algorithm based on MSR and the NIQE index. The experimental results show that this algorithm has an excellent adaptive ability to environmental changes.

Real distorted images quality assessment based on multi-layer visual perception mechanism and high-level semantics

Most of the existing image quality assessment (IQA) methods are directed to artificially synthesized distorted images, in which the types and characteristics of distortion are different from those in the real world. In view of the fact that the existing non-reference IQA methods can not accurately evaluate the quality of the real distortion image, combined with the theoretical analysis of multi-layer visual perception mechanism, we propose a real image distortion IQA method based on image underlying features and high-level semantics. Considering non-linear hierarchical structure of human visual perception, firstly, k-means clustering algorithm is performed according to the underlying feature indexs of the image so that the used image database can be divided into several groups, which aims to improve the accuracy of predicted quality score. Secondly, the deep convolutional neural network (DCNN) is used to extract the first-grade high-level semantic features in each group. Then, second-grade high-level semantic features that can provide better representation of image features are obtained by performing multiple statistical functions on first-grade high-level semantics. Besides, we establish an effective high-capacity regressor with high-level semantics and subjective mean opinion scores (MOS) values of the human eyes. The experimental results show that the proposed model on the KonIQ-10 k image database can predict the quality score effectively and achieve a high consistency with the corresponding MOS value, which is helpful for the subsequent image enhancement.

Non-reference image quality assessment based on deep clustering

Image quality assessment (IQA) is an indispensable technique in computer vision and pattern recognition Existing deep IQA methods have achieved remarkable performance. As far as we know, these deep learning-based IQA algorithms lack an adaptive features extraction mechanism toward input images with varying sizes and the stability in avoiding disturbance from data noises and model deviation. To solve these problems, we propose a non-reference IQA method by designing a novel unsupervised deep clustering framework, where a 13-layer network structure is proposed that upgrades the fully-connected layers to produce high-level features with adaptive sizes. Moreover, we add a contracted regular term with a contracted autoencoder into the clustering loss function to form a quality model reflecting the clustering structure. Compared to the other IQA algorithms, our model with simple structure exhibits more stable and robust performance by the initial configuration of network parameters during end-to-end training. The experimental results on the LIVE and CSIP databases have shown that our method not only performs better than the state-of-the-art IQA algorithms, but also has a simpler structure and better adaptability.

Image quality assessment based on adaptive multiple Skyline query

Non-reference image quality assessment has attracted great emphasis in recent years. Traditional image quality assessment algorithms based on structural similarity cannot make full use of the image gradient features, and the contrast similarity features often ignore the consistency of continuous color blocks within the image, which leads to large discrepancy between the evaluation result and the subjective judgment of human vision system. In this paper, we propose a deep model for image quality assessment where the spatial and visual features of image are both considered. For better feature fusion, we design an adaptive multiple Skyline query algorithm named MSFF, which takes as input multiple features of images, and learns the feature weights through end-to-end training. Extensive experiments on image quality assessment tasks prove that the proposed model exhibits superior performance compared with existing solutions.

Blind Measurement of Blocking Artifacts of Images Based on Edge and Flat-region Detection

Image compression based on DCT often produces distortion of blocking effects, which seriously affect the visual quality of images. In order to detect the distortion of blocking effect, a new approach based on the flat region is presented in this article. The approach has a good correlation to characteristics of human perception, because human eyes are sensitive to the blocking effect which is produced at the flat regions of images, and it is also obvious in images. Furthermore, the approach is a non-reference image quality assessment. It does not only improve the efficiency, but also enhance the practicality of the algorithm. The approach has low complexity, high accuracy and good transplantation.