Perceptual Image Quality Assessment Research Articles

Low-dose computed tomography perceptual image quality assessment

In computed tomography (CT) imaging, optimizing the balance between radiation dose and image quality is crucial due to the potentially harmful effects of radiation on patients. Although subjective assessments by radiologists are considered the gold standard in medical imaging, these evaluations can be time-consuming and costly. Thus, objective methods, such as the peak signal-to-noise ratio and structural similarity index measure, are often employed as alternatives. However, these metrics, initially developed for natural images, may not fully encapsulate the radiologists’ assessment process. Consequently, interest in developing deep learning-based image quality assessment (IQA) methods that more closely align with radiologists’ perceptions is growing. A significant barrier to this development has been the absence of open-source datasets and benchmark models specific to CT IQA. Addressing these challenges, we organized the Low-dose Computed Tomography Perceptual Image Quality Assessment Challenge in conjunction with the Medical Image Computing and Computer Assisted Intervention 2023. This event introduced the first open-source CT IQA dataset, consisting of 1,000 CT images of various quality, annotated with radiologists’ assessment scores. As a benchmark, this challenge offers a comprehensive analysis of six submitted methods, providing valuable insight into their performance. This paper presents a summary of these methods and insights. This challenge underscores the potential for developing no-reference IQA methods that could exceed the capabilities of full-reference IQA methods, making a significant contribution to the research community with this novel dataset. The dataset is accessible at https://zenodo.org/records/7833096.

ConIQA: A deep learning method for perceptual image quality assessment with limited data

Effectively assessing the realism and naturalness of images in virtual (VR) and augmented (AR) reality applications requires Full Reference Image Quality Assessment (FR-IQA) metrics that closely align with human perception. Deep learning-based IQAs that are trained on human-labeled data have recently shown promise in generic computer vision tasks. However, their performance decreases in applications where perfect matches between the reference and the distorted images should not be expected, or whenever distortion patterns are restricted to specific domains. Tackling this issue necessitates training a task-specific neural network, yet generating human-labeled FR-IQAs is costly, and deep learning typically demands substantial labeled data. To address these challenges, we developed ConIQA, a deep learning-based IQA that leverages consistency training and a novel data augmentation method to learn from both labeled and unlabeled data. This makes ConIQA well-suited for contexts with scarce labeled data. To validate ConIQA, we considered the example application of Computer-Generated Holography (CGH) where specific artifacts such as ringing, speckle, and quantization errors routinely occur, yet are not explicitly accounted for by existing IQAs. We developed a new dataset, HQA1k, that comprises 1000 natural images each paired with an image rendered using various popular CGH algorithms, and quality-rated by thirteen human participants. Our results show that ConIQA achieves superior Pearson (0.98), Spearman (0.965), and Kendall’s tau (0.86) correlations over fifteen FR-IQA metrics by up to 5%, showcasing significant improvements in aligning with human perception on the HQA1k dataset.

Enhanced Solar Coronal Imaging: A GAN Approach with Fused Attention and Perceptual Quality Enhancement

The activity of the solar corona has a significant impact on all aspects of human life. People typically use images obtained from astronomical telescopes to observe coronal activities, among which the Atmospheric Imaging Assembly (AIA) of the Solar Dynamics Observatory (SDO) is particularly widely used. However, due to resolution limitations, we have begun to study the application of generative adversarial network super-resolution techniques to enhance the image data quality for a clearer observation of the fine structures and dynamic processes in the solar atmosphere, which improves the prediction accuracy of solar activities. We aligned SDO/AIA images with images from the High-Resolution Coronal Imager (Hi-C) to create a dataset. This research proposes a new super-resolution method named SAFCSRGAN, which includes a spatial attention module that incorporates channel information, allowing the network model to better capture the corona’s features. A Charbonnier loss function was introduced to enhance the perceptual quality of the super-resolution images. Compared to the original method using ESRGAN, our method achieved an 11.9% increase in Peak Signal-to-Noise Ratio (PSNR) and a 4.8% increase in Structural Similarity (SSIM). Additionally, we introduced two perceptual image quality assessment metrics, the Natural Image Quality Evaluator (NIQE) and Learned Perceptual Image Patch Similarity (LPIPS), which improved perceptual quality by 10.8% and 1.3%, respectively. Finally, our experiments demonstrated that our improved model surpasses other models in restoring the details of coronal images.

Age-specific perceptual image quality assessment

With the development of image-based applications, assessing the quality of images has become increasingly important. Although our perception of image quality changes as we age, most existing image quality assessment (IQA) metrics make simplifying assumptions about the age of observers, thus limiting their use for age-specific applications. In this work, we propose a personalized IQA metric to assess the perceived image quality of observers from different age groups. Firstly, we apply an age simulation algorithm to compute how an observer with a particular age would perceive a given image. More specifically, we process the input image according to an age-specific contrast sensitivity function (CSF), which predicts the reduction of contrast visibility associated with the aging eye. We combine age simulation with existing IQA metrics to calculate the age-specific perceived image quality score. To validate the effectiveness of our combined model, we conducted a psychophysical experiment in a controlled laboratory environment with young (18-31 y.o.), middle-aged (32-52 y.o.), and older (53+ y.o.) adults, measuring their image quality preferences for 84 test images. Our analysis shows that the predictions by our age-specific IQA metric are well correlated with the collected subjective IQA results from our psychophysical experiment.

A no-reference perceptual image quality assessment database for learned image codecs

The drastic growth of research in image compression, especially deep learning-based image compression techniques, poses new challenges to objective image quality assessment (IQA). Typical artifacts encountered in the emerging image codecs are significantly different from that produced by traditional block-based codecs, leading to inapplicability of the existing objective IQA algorithms. Towards advancing the development of objective IQA algorithms for recent compression artifacts, we built a learning-based compressed image quality assessment (LCIQA) database involving traditional block-based image codecs, hybrid neural network based image codecs, convolutional neural network based and generative adversarial network (GAN) based end-to-end optimized image coding approaches. Our study confirms the statistical difference and human perception difference between reconstructions of learned compression and traditional block-based compression. We propose a two-step deep learning model for learning-based compressed image quality assessment. Extensive experiments on LCIQA database demonstrate that our proposed model performs better than other counterparts on learning-based compressed images, especially on GAN compressed images, and achieves competitive performance to the state-of-the-art IQA metrics on traditional compressed images.

Face anti-spoofing detection based on multi-scale image quality assessment

This article introduces an effective face PAD algorithm based on multiscale perceptual image quality assessment features. Unique hand-crafted texture features extracted from face images are exploited for spoofing detection. The proposed features are classified into three major models: generalized Gaussian density-based, asymmetric generalized Gaussian density-based, and top gradient similarity deviation features. In light of the essential attributes of these models, a total of 21 multiscale features are acquired for classification, which is performed through a support vector machine (SVM). Extensive experiments on five benchmark databases, CASIA, Replay-Attack, UVAD, OULU-NPU, and SiW along with our new dataset demonstrated the effectiveness of the proposed framework. Experimental results indicated that our face PAD algorithm produced satisfactory detection accuracy on the tested datasets based on both intra-dataset and cross-dataset protocols. While outperforming a number of traditional face PAD methods, the proposed scheme achieved comparable results with many state-of-the-art deep learning-based networks. The introduction of the image quality assessment features with multiscale analysis into face PAD is promising for detection accuracy improvement.

Deep ensembling for perceptual image quality assessment

Deep ensembling for perceptual image quality assessment

Image Quality Assessment of No Reference JPEG Compressed Images Using Various Spatial Domain Features

In today's world of technological advancement higher data rate and data transmission with minimal memoryrequirement is gaining importance. At the same time it also important to preserve quality of data to be transferredfrom various types of distortions and to assess the quality at the receiving end. Hence perceptual image qualityassessment is becoming more popular. This paper introduces a new approach in domain of no reference imagequality assessment of jpeg compressed images using spatial features .Considering the fact that pixel distortion,blurring and edge information are important aspects as far as distortions are concern. A novel approach is presentedin this paper to address the distortion categories. The results are obtained in conjunction with the subjectiveimage quality assessment of train and test image categories. Also results are compared with full reference imagequality assessment technique and parameters of full reference quality assessment. To achieve better accuracyalgorithm is tested on LIVE Texas' image database and our own developed database. Results are found to be wellcorrelating to the various parameters considered for the comparison purpose.

GSW: A High Performance IQA Index Based on Global and Saliency Window Similarity

Perceptual image quality assessment (IQA) attracts significant attention in recent years. It is proved that both global score and an image’s visual saliency (VS) are consistent with subjective evaluation. The global quality score reflects the consistency of the overall structure between two images, and VS map contributes complementarily in evaluating perceptual quality. This paper presents an effective IQA index based on global and saliency window similarity, namely GSW. It chooses VS map as an image feature and uses a special strategy to draw the saliency window with largest significancy. Meanwhile, the background area is taken into account to guarantee the robustness of the quality score. Experimental results on four most widely used databases verify that, compared with state-of-the-art IQA methods, GSW performs consistently well can provide more accurate quality prediction with a low computational complexity.

Perceptual image quality assessment based on gradient similarity and Ruderman operator

Perceptual image quality assessment based on gradient similarity and Ruderman operator

Perceptual Image Quality Assessment Technique for Color Images based on HSI Colour Space

A new full-reference Perceptual Image Quality Assessment (PIQA) technique based on Hue-Saturation-Intensity (HSI) transformation method for colour images is offered in this work. Fundamentally, it is combined with three approaches (colour transformation, histogram processing and Human Visual System (HVS) based weighting) and it uses the Discrete Cosine Transform (DCT) domain. The PIQA is composed of four phases. At the first phase, a colour image is transformed using the HSI because an important characteristic of eyes reaction to intensity of light and colour knowledge is used for quality assessment processes. All channels' DCT coefficients have been calculated at the second phase, because many specifications cannot be assessed in the spatial domain. At the third phase, histogram based quality assessment results are obtained by using histogram of each channel (Hue, Saturation and Intensity). These results are weighted for obtaining perceptual quality result taking into account the HVS specifications at the last phase because the human eye has different sensitivities to the intensity and the others (i.e., hue and saturation). Experimental outcomes about feasibility of the PIQA on test images under different deteriorations, both sensible by the HVS and with the same Peak Signal to Noise Ratio (PSNR) result are offered. The PIQA shows better performance in comparison to state-of-the-art techniques

Perceptual Image Quality Assessment Technique for Color Images based on HSI Colour Space

A new full-reference Perceptual Image Quality Assessment (PIQA) technique based on Hue-Saturation-Intensity (HSI) transformation method for colour images is offered in this work. Fundamentally, it is combined with three approaches (colour transformation, histogram processing and Human Visual System (HVS) based weighting) and it uses the Discrete Cosine Transform (DCT) domain. The PIQA is composed of four phases. At the first phase, a colour image is transformed using the HSI because an important characteristic of eyes reaction to intensity of light and colour knowledge is used for quality assessment processes. All channels' DCT coefficients have been calculated at the second phase, because many specifications cannot be assessed in the spatial domain. At the third phase, histogram based quality assessment results are obtained by using histogram of each channel (Hue, Saturation and Intensity). These results are weighted for obtaining perceptual quality result taking into account the HVS specifications at the last phase because the human eye has different sensitivities to the intensity and the others (i.e., hue and saturation). Experimental outcomes about feasibility of the PIQA on test images under different deteriorations, both sensible by the HVS and with the same Peak Signal to Noise Ratio (PSNR) result are offered. The PIQA shows better performance in comparison to state-of-the-art techniques

Perceptual image quality assessment based on gradient similarity and Ruderman operator

In this work, a new metric for image quality assessment is suggested, which provides more suppleness than previous measures in using Ruderman operator, visual region of interest and gradient similarity. Firstly, the luminance distortion between the reference and test images is determined. Secondly, the gradient similarity is computed by using canny filter and proposed gradient mask. Thirdly, the test and reference images are transformed using Ruderman operator. Fourthly, the visual region of interest is calculated by employing entropy operator. Lastly, the dissimilarity between the reference and test images is obtained, by combining all previous metrics: luminance distortion measure, gradient similarity measures, Ruderman measure and visual region of interest measure. Experimental comparison demonstrates the effectiveness of the proposed method.

Quality Assessment of SAR-to-Optical Image Translation

Synthetic aperture radar (SAR) images contain severe speckle noise and weak texture, which are unsuitable for visual interpretation. Many studies have been undertaken so far toward exploring the use of SAR-to-optical image translation to obtain near optical representations. However, how to evaluate the translation quality is a challenge. In this paper, we combine image quality assessment (IQA) with SAR-to-optical image translation to pursue a suitable evaluation approach. Firstly, several machine-learning baselines for SAR-to-optical image translation are established and evaluated. Then, extensive comparisons of perceptual IQA models are performed in terms of their use as objective functions for the optimization of image restoration. In order to study feature extraction of the images translated from SAR to optical modes, an application in scene classification is presented. Finally, the attributes of the translated image representations are evaluated using visual inspection and the proposed IQA methods.

Perceptual image quality assessment: a survey

Perceptual quality assessmentplays a vital role in the visual communication systems owing to theexistence of quality degradations introduced in various stages of visual signalacquisition, compression, transmission and display.Quality assessment for visual signals can be performed subjectively andobjectively, and objective quality assessment is usually preferred owing to itshigh efficiency and easy deployment. A large number of subjective andobjective visual quality assessment studies have been conducted during recent years.In this survey, we give an up-to-date and comprehensivereview of these studies.Specifically, the frequently used subjective image quality assessment databases are firstreviewed, as they serve as the validation set for the objective measures.Second, the objective image quality assessment measures are classified and reviewed according to the applications and the methodologies utilized in the quality measures.Third, the performances of the state-of-the-artquality measures for visual signals are compared with an introduction of theevaluation protocols.This survey provides a general overview of classical algorithms andrecent progresses in the field of perceptual image quality assessment.

Perceptual Image Quality Assessment for Various Viewing Conditions and Display Systems

From complete darkness to direct sunlight, real-world displays operate in various viewing conditions often resulting in a non-optimal viewing experience. Most existing Image Quality Assessment (IQA) methods, however, assume ideal environments and displays, and thus cannot be used when viewing conditions differ from the standard. In this paper, we investigate the influence of ambient illumination level and display luminance on human perception of image quality. We conduct a psychophysical study to collect a novel dataset of over 10000 image quality preference judgments performed in illumination conditions ranging from 0 lux to 20000 lux. We also propose a perceptual IQA framework that allows most existing image quality metrics (IQM) to accurately predict image quality for a wide range of illumination conditions and display parameters. Our analysis demonstrates strong correlation between human IQA and the predictions of our proposed framework combined with multiple prominent IQMs and across a wide range of luminance values.

Analysis of Structural Characteristics for Quality Assessment of Multiply Distorted Images

Perceptual image quality assessment (IQA) plays an important role in numerous applications, including image restoration, compression, enhancement, and others. Although many works have been conducted on individually distorted IQA problems and have achieved encouraging results, few studies have been conducted on multiple distorted (MD) IQA problems. Thus, limited progress has been made. In this paper, we propose a novel no reference image quality assessment (NR-IQA) method, named improved multiscale local binary pattern (IMLBP), for addressing multiply distorted IQA problems. The image structures are sensitive to image distortions, which motivates us to utilize the structural characteristics for overall image quality prediction. We improved the local binary pattern (LBP) by considering the human visual mechanism to better extract the structural information. The IMLBP contains two parts, the LBP and the radius difference LBP (DLBP). The DLBP reflects the values’ changes in the radial direction. Specifically, when the radius value is small, the proposed descriptor is computed to represent microstructural information. Conversely, it represents macrostructural information when the radius becomes large. Moreover, to better mimick the human visual mechanism, the IMLBP is computed with the multiscale strategy and the operation is based on a patch unit whose size is proportional to the radius value. The frequency histogram of feature maps is transformed to feature vectors. Subsequently, a predictable function trained by the support vector regression is used to infer the overall quality score. Experimental results show that the proposed method outperforms most state-of-the-art IQA metrics on publicly available multiply distorted image databases.

Perceptual image quality assessment through spectral analysis of error representations

In this paper, we analyze the statistics of error signals to assess the perceived quality of images. Specifically, we focus on the magnitude spectrum of error images obtained from the difference of reference and distorted images. Analyzing spectral statistics over grayscale images partially models interference in spatial harmonic distortion exhibited by the visual system but it overlooks color information, selective and hierarchical nature of visual system. To overcome these shortcomings, we introduce an image quality assessment algorithm based on the Spectral Understanding of Multi-scale and Multi-channel Error Representations, denoted as SUMMER. We validate the quality assessment performance over 3 databases with around 30 distortion types. These distortion types are grouped into 7 main categories as compression artifact, image noise, color artifact, communication error, blur, global and local distortions. In total, we benchmark the performance of 17 algorithms along with the proposed algorithm using 5 performance metrics that measure linearity, monotonicity, accuracy, and consistency. In addition to experiments with standard performance metrics, we analyze the distribution of objective and subjective scores with histogram difference metrics and scatter plots. Moreover, we analyze the classification performance of quality assessment algorithms along with their statistical significance tests. Based on our experiments, SUMMER significantly outperforms majority of the compared methods in all benchmark categories.

Contrast and Visual Saliency Similarity-Induced Index for Assessing Image Quality

Image quality that is consistent with human opinion is assessed by a perceptual image quality assessment (IQA) that defines/utilizes a computational model. A good model should take effectiveness and efficiency into consideration, but most of the previously proposed IQA models do not simultaneously consider these factors. Therefore, this study attempts to develop an effective and efficient IQA metric. Contrast is an inherent visual attribute that indicates image quality, and visual saliency (VS) is a quality that attracts the attention of human beings. The proposed model utilized these two features to characterize the image local quality. After obtaining the local contrast quality map and the global VS quality map, we added the weighted standard deviation of the previous two quality maps together to yield the final quality score. The experimental results for three benchmark databases (LIVE, TID2008, and CSIQ) demonstrated that our model performs the best in terms of a correlation with the human judgment of visual quality. Furthermore, compared with competing IQA models, this proposed model is more efficient.

A Fast Reliable Image Quality Predictor by Fusing Micro- and Macro-Structures

A fast reliable computational quality predictor is eagerly desired in practical image/video applications, such as serving for the quality monitoring of real-time coding and transcoding. In this paper, we propose a new perceptual image quality assessment (IQA) metric based on the human visual system (HVS). The proposed IQA model performs efficiently with convolution operations at multiscales, gradient magnitude, and color information similarity, and a perceptual-based pooling. Extensive experiments are conducted using four popular large-size image databases and two multiply distorted image databases, and results validate the superiority of our approach over modern IQA measures in efficiency and efficacy. Our metric is built on the theoretical support of the HVS with lately designed IQA methods as special cases.