No-reference Image Quality Assessment Algorithms Research Articles

Dual-stream complex-valued convolutional network for authentic dehazed image quality assessment.

Effectively evaluating the perceptual quality of dehazed images remains an under-explored research issue. In this paper, we propose a no-reference complex-valued convolutional neural network (CV-CNN) model to conduct automatic dehazed image quality evaluation. Specifically, a novel CV-CNN is employed that exploits the advantages of complex-valued representations, achieving better generalization capability on perceptual feature learning than real-valued ones. To learn more discriminative features to analyze the perceptual quality of dehazed images, we design a dual-stream CV-CNN architecture. The dual-stream model comprises a distortion-sensitive stream that operates on the dehazed RGB image, and a haze-aware stream on a novel dark channel difference image. The distortion-sensitive stream accounts for perceptual distortion artifacts, while the haze-aware stream addresses the possible presence of residual haze. Experimental results on three publicly available dehazed image quality assessment (DQA) databases demonstrate the effectiveness and generalization of our proposed CV-CNN DQA model as compared to state-of-the-art no-reference image quality assessment algorithms.

Progressive photon mapping algorithm for digital imaging of a space target.

Space target imaging simulation holds great significance for the development of space-based imaging systems and the prediction of imaging tasks. Currently, commonly used rendering algorithms in space target imaging simulation suffer from issues such as radiance calculation results lacking actual physical significance and rendered images containing high-frequency noise. To address these issues, this study proposes a rendering algorithm based on progressive photon mapping in the context of space imaging scenarios, aiming to enhance the accuracy of energy calculations and image rendering for space targets. This algorithm generates multiple photon maps on the target surface through multiple iterations, retrieves photon information near the observation point based on these photon maps, and thus obtains the radiance of the observation direction vector. This study evaluates the quality of rendered images using no-reference image quality assessment algorithms. The results demonstrate that this algorithm can enhance image rendering quality in specific imaging scenarios, consequently improving the accuracy of space target recognition. By comparing the calculated values of this algorithm with the theoretical radiance values for diffuse material, the accuracy of the radiance calculation results of this algorithm is verified, which can provide significant reference values for the selection of backend detectors.

Research on No-reference Image Quality Assessment Algorithm Based on Generative Adversarial Networks

Currently, with the massive generation and transmission of digital images, especially the rapid development of the Internet and mobile Internet industries where images are widely used, the study of reference-free image quality assessment has attracted much attention in academia and practical applications, and is a popular research direction in the field of computer vision. In response to the low performance of existing reference-free image quality assessment algorithms in the face of real distorted images, a reference-free image quality assessment algorithm based on generative adversarial networks is proposed. Firstly, the generator structure is changed, the U-Net structure is improved, and the channel attention mechanism SeNet structure is introduced to update the feature map after down sampling. Secondly, a feature similarity measurement system is incorporated and a dual discriminator structure is used to discriminate multiple groups of images. The FPN structure is combined in the feature extractor to produce a multi-scale feature representation. Experiments are conducted on KonIQ-10k dataset and LIVEC dataset, and the experimental results show that the algorithm exhibits good prediction accuracy as well as good generalization performance in the face of real distorted images.

No-Reference Image Quality Assessment Using the Statistics of Global and Local Image Features

Methods of image quality assessment are widely used for ranking computer vision algorithms or controlling the perceptual quality of video and streaming applications. The ever-increasing number of digital images has encouraged the research in this field at an accelerated pace in recent decades. After the appearance of convolutional neural networks, many researchers have paid attention to different deep architectures to devise no-reference image quality assessment algorithms. However, many systems still rely on handcrafted features to ensure interpretability and restrict the consumption of resources. In this study, our efforts are focused on creating a quality-aware feature vector containing information about both global and local image features. Specifically, the research results of visual physiology indicate that the human visual system first quickly and automatically creates a global perception before gradually focusing on certain local areas to judge the quality of an image. Specifically, a broad spectrum of statistics extracted from global and local image features is utilized to represent the quality-aware aspects of a digital image from various points of view. The experimental results demonstrate that our method’s predicted quality ratings relate strongly with the subjective quality ratings. In particular, the introduced algorithm was compared with 16 other well-known advanced methods and outperformed them by a large margin on 9 accepted benchmark datasets in the literature: CLIVE, KonIQ-10k, SPAQ, BIQ2021, TID2008, TID2013, MDID, KADID-10k, and GFIQA-20k, which are considered de facto standards and generally accepted in image quality assessment.

No-Reference Image Quality Assessment Algorithm Based on Semi-Supervised Learning

No-Reference Image Quality Assessment Algorithm Based on Semi-Supervised Learning

No-Reference Quality Assessment of Authentically Distorted Images Based on Local and Global Features.

With the development of digital imaging techniques, image quality assessment methods are receiving more attention in the literature. Since distortion-free versions of camera images in many practical, everyday applications are not available, the need for effective no-reference image quality assessment algorithms is growing. Therefore, this paper introduces a novel no-reference image quality assessment algorithm for the objective evaluation of authentically distorted images. Specifically, we apply a broad spectrum of local and global feature vectors to characterize the variety of authentic distortions. Among the employed local features, the statistics of popular local feature descriptors, such as SURF, FAST, BRISK, or KAZE, are proposed for NR-IQA; other features are also introduced to boost the performances of local features. The proposed method was compared to 12 other state-of-the-art algorithms on popular and accepted benchmark datasets containing RGB images with authentic distortions (CLIVE, KonIQ-10k, and SPAQ). The introduced algorithm significantly outperforms the state-of-the-art in terms of correlation with human perceptual quality ratings.

A technical review of no-reference image quality assessment algorithms for contrast distorted images

Automatic image quality assessment similar to human vision perception is an essential process for real-time image processing applications to perform perceptual image assessments for effectively achieving their goals. As no-reference image quality assessment (NR-IQA) schemes perform perceptual assessments of images without any information about their original version, these algorithms suit real-time computer vision techniques because of the non-availability of reference images. Contrast and colorfulness play important roles in determining the quality of color images. By combining many IQA metrics, a number of combined metrics had been devised. This study provides an insight into major NR-IQA methods and their effectiveness in assessing contrast, colorfulness, and overall quality of contrast-degraded images with technical analysis. The effectiveness of top-ranking NR-IQA methods is experimentally assessed with benchmark assessment methods on images from benchmarked databases. The study provides insight into open research challenges in the area of NR-IQA for developing new promising methods by clearly demarcating the difficulties of top-ranking NR-IQA methods.

No‐reference image quality assessment based on multiscale feature representation

The no-reference image quality assessment (NR-IQA) method can evaluate the distortions in an image without the reference image. However, due to the diversity of the image contents and distortion types, it is hard for the existing NR-IQA algorithms to obtain competitive performance on both synthetically and authentically distorted images. To address the problem, a multiscale feature representation-based NR-IQA method that performs well for both synthetic and authentic distortions is proposed. This model consists of two parts: The feature extraction part and the feature fusion part. First, part of the Res2Net-50 network is chosen as the feature extraction part due to its high ability in increasing the range of receptive fields. Then, the feature fusion part consisting of a novel residual block and two fully connected layers is designed to fuse the extracted features and realize the quality score mapping. After a series of stepwise optimization experiments, the most competitive network architecture consisting of the feature extraction part and the feature fusion part is obtained. Comprehensive experiments on the LIVE, TID2013, CSIQ, KADID-10k, KonIQ-10K, and LIVE challenge databases demonstrate that the proposed method can work powerfully on both the synthetic and authentic distortions and also has a strong generalization ability.

Gauss-guided patch-based deep convolutional neural networks for no-reference image quality assessment

Patch-based deep convolutional neural network (DCNN) has been proved to have advanced performance in no-reference image quality assessment (NR-IQA). However, these methods generally take global quality score as the quality score of each patch mainly since local quality score is not provided. Unfortunately, the perceived quality of image patch is difficult to maintain a high degree of consistency. Thus, the use of the same global quality score in different patches of the same image may hinder training of DCNNs. In this paper, we propose a universal and nearly cost-free model called Gaussian Random Jitter (GRJ). According to the uncertainty of the perceived quality, GRJ divided the training images into high-confidence distorted images and low-confidence distorted images, and reasonably assigned different local quality scores to each patch through specific gaussian functions with the global quality score as the mean value and the undetermined hyperparameter as the standard deviation. We took one of the most advanced patch-based DCNNs models as backbone and tested the improved performance over three widely used image quality databases. We show that our model can further improve the performance of patch-based models and even help them comparable with those of state-of-the-art NR-IQA algorithms.

MoNET: no-reference image quality assessment based on a multi-depth output network

When deep convolutional neural networks perform feature extraction, the features computed at each layer express different abstractions of visual information. The earlier layers extract highly compact low-level features such as bandpass and directional primitives, whereas deeper layers extract structural features of increasing abstraction, similar to contours, shapes, and edges, becoming less effable as the depth increases. We propose a different kind of end-to-end no-reference (NR) image quality assessment (IQA) model, which is defined as a multi-depth output convolutional neural network (MoNET). It accomplishes this by mapping both shallow and deep features to perceived quality. MoNET delivers three outputs that express shallow (lower-level) and deep (high-level) features, and maps them to subjective quality scores. The multiple outputs are combined into a single, final quality score. MoNET does this by combining the responses of three learning machines, so it may be viewed as a form of ensemble learning. The experimental results on three public image quality databases show that our proposed model achieves better performance than other state-of-the-art NR IQA algorithms.

A Document Image Dataset for Quality Assessment

Mobile device plays an very important role in capturing document image. However, the quality of the captured image is influenced by many factors, such as device quality and shooting conditions. In this context, it is necessary to automatically assess the quality of captured document image. Although there has a lot of work in the filed of image quality assessment (IQA), insufficient attention has been paid to the establishment of document images dataset. Thus, we propose a large dataset of document images containing 19,943 images which are collected by mobile devices. During the process of image acquisition, many factors such as light intensity, distortion type, document material are considered. After capturing images, multiple volunteers participated in the evaluation and collection of Mean Opinion Score (MOS) of the document images. We use two no-reference image quality assessment algorithms to test the proposed dataset. The experimental results show the validity of our dataset and the reliability of MOS. The proposed dataset can be used in the field of image quality assessment and Optical Character Recognition.

Mixed distortion image enhancement method based on joint of deep residuals learning and reinforcement learning

The images distortion leads to the loss of image information and the degradation of perceptual quality. To solve this problem, we investigate a novel mixed distortion image enhancement method based on the parallel network combines deep residual and reinforcement learning. The no-reference image quality assessment algorithm is used to determine the type and level of distorted images accurately. According to the type of distortion, the mixed distortion images enter one of the subsequent parallel joint learning networks automatically. In the joint learning framework, we prepare different residual networks to handle specialized restoration assignments including deblurring, denoising, or JPEG compression. Simultaneously, reinforcement learning then learns a policy to select the next best restoration tasks to progressively restore the quality of a corrupted image. Our method is capable of restoring images corrupted with complex mixed distortions in a more parameter-efficient manner in comparison to conventional networks. The extensive experiments on synthetic and real-world images validate the superior performances of the proposed method.

Effect of Contrast Measures on the Performance of No-Reference Image Quality Assessment Algorithm for Contrast-Distorted Images

Effect of Contrast Measures on the Performance of No-Reference Image Quality Assessment Algorithm for Contrast-Distorted Images

Towards Automatic Image Exposure Level Assessment

The quality of acquired images can be surely reduced by improper exposures. Thus, in many vision-related industries, such as imaging sensor manufacturing and video surveillance, an approach that can routinely and accurately evaluate exposure levels of images is in urgent need. Taking an image as input, such a method is expected to output a scalar value, which can represent the overall perceptual exposure level of the examined image, ranging from extremely underexposed to extremely overexposed. However, studies focusing on image exposure level assessment (IELA) are quite sporadic. It should be noted that blind NR-IQA (no-reference image quality assessment) algorithms or metrics used to measure the quality of contrast-distorted images cannot be used for IELA. The root reason is that though these algorithms can quantify quality distortion of images, they do not know whether the distortion is due to underexposure or overexposure. This paper aims to resolve the issue of IELA to some extent and contributes to two aspects. Firstly, an Image Exposure Database (IEpsD) is constructed to facilitate the study of IELA. IEpsD comprises 24,500 images with various exposure levels, and for each image a subjective exposure score is provided, which represents its perceptual exposure level. Secondly, as IELA can be naturally formulated as a regression problem, we thoroughly evaluate the performance of modern deep CNN architectures for solving this specific task. Our evaluation results can serve as a baseline when the other researchers develop even more sophisticated IELA approaches. To facilitate the other researchers to reproduce our results, we have released the dataset and the relevant source code at https://cslinzhang.github.io/imgExpo/.

No-Reference Image Quality Assessment Based on the Fusion of Statistical and Perceptual Features.

The goal of no-reference image quality assessment (NR-IQA) is to predict the quality of an image as perceived by human observers without using any pristine, reference images. In this study, an NR-IQA algorithm is proposed which is driven by a novel feature vector containing statistical and perceptual features. Different from other methods, normalized local fractal dimension distribution and normalized first digit distributions in the wavelet and spatial domains are incorporated into the statistical features. Moreover, powerful perceptual features, such as colorfulness, dark channel feature, entropy, and mean of phase congruency image, are also incorporated to the proposed model. Experimental results on five large publicly available databases (KADID-10k, ESPL-LIVE HDR, CSIQ, TID2013, and TID2008) show that the proposed method is able to outperform other state-of-the-art methods.

Quality Assessment on Authentically Distorted Images by Expanding Proxy Labels

In this paper, we propose a no-reference image quality assessment (NR-IQA) approach towards authentically distorted images, based on expanding proxy labels. In order to distinguish from the human labels, we define the quality score, which is generated by using a traditional NR-IQA algorithm, as “proxy labels”. “Proxy” means that the objective results are obtained by computer after the extraction and assessment of the image features, instead of human judging. To solve the problem of limited image quality assessment (IQA) dataset size, we adopt a cascading transfer-learning method. First, we obtain large numbers of proxy labels which denote the quality score of authentically distorted images by using a traditional no-reference IQA method. Then the deep network is trained by the proxy labels, in order to learn IQA-related knowledge from the amounts of images with their scores. Ultimately, we use fine-tuning to inherit knowledge represented in the trained network. During the procedure, the mapping relationship fits in with human visual perception closer. The experimental results demonstrate that the proposed algorithm shows an outstanding performance as compared with the existing algorithms. On the LIVE In the Wild Image Quality Challenge database and KonIQ-10k database (two standard databases for authentically distorted image quality assessment), the algorithm realized good consistency between human visual perception and the predicted quality score of authentically distorted images.

Quality improvement of adaptive optics retinal images using conditional adversarial networks.

The adaptive optics (AO) technique is widely used to compensate for ocular aberrations and improve imaging resolution. However, when affected by intraocular scatter, speckle noise, and other factors, the quality of the retinal image will be degraded. To effectively improve the image quality without increasing the imaging system's complexity, the post-processing method of image deblurring is adopted. In this study, we proposed a conditional adversarial network-based method for directly learning an end-to-end mapping between blurry and restored AO retinal images. The proposed model was validated on synthetically generated AO retinal images and real retinal images. The restoration results of synthetic images were evaluated with the metrics of peak signal-to-noise ratio (PSNR), structural similarity (SSIM), perceptual distance, and error rate of cone counting. Moreover, the blind image quality index (BIQI) was used as the no-reference image quality assessment (NR-IQA) algorithm to evaluate the restoration results on real AO retinal images. The experimental results indicate that the images restored by the proposed method have sharper quality and higher signal-to-noise ratio (SNR) when compared with other state-of-the-art methods, which has great practical significance for clinical research and analysis.

Feature-segmentation strategy based convolutional neural network for no-reference image quality assessment

Convolutional neural networks (CNN) have been shown to deliver outstanding performance for image quality assessment (IQA). Most CNN models are trained using small image patches with a fixed resolution of 32 × 32. However, more information of image content and human visual system should be taken into account. This paper proposes a post segmentation based CNN model for no-reference quality assessment without any pre-processing. The network consists of five convolutional layers with max pooling, one special fully connected layer feature-segmentation and one output layer. This paper adopts the feature-segmentation strategy to assure enough training data. We modify the structure of fully connected layer and regard every feature vector of the last pooling maps as an independent sample to train our proposed model. In this way, the raw images can be fed into the input and do not need to be split into patches, to avoid any hand-crafted features. Moreover, the size of the input images is not fixed, and the size of the extracted feature vector is invariant. Experiments on LIVE, CSIQ and TID2008 databases demonstrate that our approach has high consistency with the subjective evaluation scores. The experimental results show that the proposed network outperforms state-of-the-art no-reference IQA algorithms and is comparable to some full-reference IQA algorithms.

Dual Head Network for No-Reference Quality Assessment Towards Realistic Night-Time Images

No-reference image quality assessment (NR-IQA), which devotes to predicting image quality without relying on the corresponding pristine counterpart, develops rapidly in recent years. However, little investigation has been dedicated to quality assessment of realistic night-time images. Existing NR-IQA algorithms laboriously cope with this night-time scenario since complicated authentic distortions such as low contrast, blurred details, and reduced visibility usually appear on it. In this paper, we propose an end-to-end NR-IQA model to meet this challenge based on a multi-stream deep convolutional neural network (DCNN). Two streams, brightness-aware CNN and naturalness-aware CNN are constructed respectively by a brightness-altered image identification task with a self-established dataset and a quality-prediction regression task with an existing authentically-distorted IQA dataset to improve quality-aware initializations. In this case, given the quick convergence and little transformation in the lower layers, a shallow-layer-shared architecture is explored to reduce computational cost. Finally, the features of these two pipelines are collected by an effective pooling method and then concatenated as the image representation for fine-tuning. The effectiveness and efficiency of the proposed method are verified by several different experiments on the NNID, CCRIQ and LIVE Challenge databases. Furthermore, the superiority of wide applications such as for contrast-distorted and driving scenarios is demonstrated on the CID2013, CCID2014 and BBD-100k databases.

A Highly Efficient Blind Image Quality Assessment Metric of 3-D Synthesized Images Using Outlier Detection

With multitudes of image processing applications, image quality assessment (IQA) has become a prerequisite for obtaining maximally distinctive statistics from images. Despite the widespread research in this domain over several years, existing IQA algorithms have a number of key limitations concerning different image distortion types and algorithms’ computational efficiency. Images that are synthesized using depth image-based rendering have applications in various disciplines, such as free viewpoint videos, which enable synthesis of novel realistic images in the referenceless environment. In the literature, very few no-reference (NR) quality assessment metrics of three-dimensional (3-D) synthesized images are proposed, and most of them are computationally expensive, which makes it difficult for them to be deployed in real-time applications. In this paper, we attribute the geometrically distorted pixels as outliers in 3-D synthesized images. This assumption is validated using the three $sigma$ rule-based robust outlyingness ratio. We propose a novel fast and accurate blind IQA metric of 3-D synthesized images using nonlinear median filtering since the median filtering has the capability of identifying and removing outliers. The advantages of the proposed algorithm are twofold. First, it uses a simple technique, i.e., median filtering, to capture the level of geometric and structural distortions (up to some extend). Second, the proposed algorithm has higher computational efficiency. Experiments show the superiority of the proposed NR IQA algorithm over existing state-of-the-art full-, reduced-, and NR IQA methods, in terms of both predicting accuracy and computational complexity.