No-reference Image Quality Assessment Research Articles

Saliency-guided convolution neural network–transformer fusion network for no-reference image quality assessment

Saliency-guided convolution neural network–transformer fusion network for no-reference image quality assessment

MRI Super-Resolution Analysis via MRISR: Deep Learning for Low-Field Imaging

Abstract: This paper presents a novel MRI super-resolution analysis model, MRISR. Through the utilization of generative adversarial networks for the estimation of degradation kernels and the injection of noise, we have constructed a comprehensive dataset of high-quality paired high- and low-resolution MRI images. The MRISR model seamlessly integrates VMamba and Transformer technologies, demonstrating superior performance across various no-reference image quality assessment metrics compared with existing methodologies. It effectively reconstructs high-resolution MRI images while meticulously preserving intricate texture details, achieving a fourfold enhancement in resolution. This research endeavor represents a significant advancement in the field of MRI super-resolution analysis, contributing a cost-effective solution for rapid MRI technology that holds immense promise for widespread adoption in clinical diagnostic applications.

No-reference image quality assessment based on global awareness.

In the field of computer vision, the application of hand-crafted as well as computer-learning-based methods in the field of image quality assessment has yielded remarkable results. However, in the field of no-reference image distortion, it is still challenging to accurately perceive and determine the quality of an image. To address the difficulties of Image Quality Assessment (IQA) in the field of authentic distorted images, we consider the use of the Swin Transformer (ST) to extract features. To enable the model to focus on both spatial and channel information of features, we design a plug-and-play Global Self-Attention Block (GSAB). At the same time, we introduce a Transformer block in the model to enhance the model's ability to capture long-range dependencies. Finally, we derive the prediction of image quality scores through a Dual-Branching structure. Our method is experimented on four synthetic datasets as well as two authentic datasets, and the results of the experiments are weighted according to the size of the datasets, and the results show that our method outperforms all the current methods and works well in the Generalization ability test, which proves that our method has a good generalization ability. The code will be posted subsequently at https://github.com/yanggege-new/NR-IQA-based-on-global-awareness.

Black-box adversarial attacks against image quality assessment models

The problem of No-Reference Image Quality Assessment (NR-IQA) is to predict the perceptual quality of an image in line with its subjective evaluation. However, the vulnerabilities of NR-IQA models to the adversarial attacks have not been thoroughly studied for model refinement. This paper aims to investigate the potential loopholes of NR-IQA models via black-box adversarial attacks. Specifically, we first formulate the attack problem as maximizing the deviation between the estimated quality scores of original and perturbed images, while restricting the perturbed image distortions for visual quality preservation. Under such formulation, we then design a Bi-directional loss function to mislead the estimated quality scores of adversarial examples towards an opposite direction with maximum deviation. On this basis, we finally develop an efficient and effective black-box attack method for NR-IQA models based on a random search paradigm. Comprehensive experiments on three benchmark datasets show that all evaluated NR-IQA models are significantly vulnerable to the proposed attack method. After being attacked, the average change rates in terms of two well-known IQA performance metrics achieved by victim models reach 97% and 101%, respectively. In addition, our attack method also outperforms a newly introduced black-box attack approach on IQA models. We also observe that the generated perturbations are not transferable, which points out a new research direction in NR-IQA community. The source code is available at https://github.com/GZHU-DVL/AttackIQA.

Low-dose computed tomography perceptual image quality assessment

In computed tomography (CT) imaging, optimizing the balance between radiation dose and image quality is crucial due to the potentially harmful effects of radiation on patients. Although subjective assessments by radiologists are considered the gold standard in medical imaging, these evaluations can be time-consuming and costly. Thus, objective methods, such as the peak signal-to-noise ratio and structural similarity index measure, are often employed as alternatives. However, these metrics, initially developed for natural images, may not fully encapsulate the radiologists’ assessment process. Consequently, interest in developing deep learning-based image quality assessment (IQA) methods that more closely align with radiologists’ perceptions is growing. A significant barrier to this development has been the absence of open-source datasets and benchmark models specific to CT IQA. Addressing these challenges, we organized the Low-dose Computed Tomography Perceptual Image Quality Assessment Challenge in conjunction with the Medical Image Computing and Computer Assisted Intervention 2023. This event introduced the first open-source CT IQA dataset, consisting of 1,000 CT images of various quality, annotated with radiologists’ assessment scores. As a benchmark, this challenge offers a comprehensive analysis of six submitted methods, providing valuable insight into their performance. This paper presents a summary of these methods and insights. This challenge underscores the potential for developing no-reference IQA methods that could exceed the capabilities of full-reference IQA methods, making a significant contribution to the research community with this novel dataset. The dataset is accessible at https://zenodo.org/records/7833096.

UIEDP: Boosting underwater image enhancement with diffusion prior

Underwater image enhancement (UIE) aims to generate clear images from low-quality underwater images. Due to the unavailability of clear reference images, researchers often synthesize them to construct paired datasets for training deep models. However, these synthesized images may sometimes lack quality, adversely affecting training outcomes. To address this issue, we propose to boost UIE with Diffusion Prior (UIEDP). It is a novel framework treating UIE as a posterior distribution sampling process of clear images conditioned on degraded underwater inputs. Specifically, UIEDP combines a pre-trained diffusion model capturing natural image priors with any existing UIE algorithm, leveraging the latter to guide conditional generation. The diffusion prior mitigates the drawbacks of inferior synthetic images, resulting in higher-quality image generation. Extensive experiments have demonstrated that our UIEDP yields significant improvements across various metrics, especially no-reference image quality assessment. And the generated enhanced images also exhibit a more natural appearance.

No-Reference Image Quality Assessment Combining Swin-Transformer and Natural Scene Statistics.

No-reference image quality assessment aims to evaluate image quality based on human subjective perceptions. Current methods face challenges with insufficient ability to focus on global and local information simultaneously and information loss due to image resizing. To address these issues, we propose a model that combines Swin-Transformer and natural scene statistics. The model utilizes Swin-Transformer to extract multi-scale features and incorporates a feature enhancement module and deformable convolution to improve feature representation, adapting better to structural variations in images, apply dual-branch attention to focus on key areas, and align the assessment more closely with human visual perception. The Natural Scene Statistics compensates information loss caused by image resizing. Additionally, we use a normalized loss function to accelerate model convergence and enhance stability. We evaluate our model on six standard image quality assessment datasets (both synthetic and authentic), and show that our model achieves advanced results across multiple datasets. Compared to the advanced DACNN method, our model achieved Spearman rank correlation coefficients of 0.922 and 0.923 on the KADID and KonIQ datasets, respectively, representing improvements of 1.9% and 2.4% over this method. It demonstrated outstanding performance in handling both synthetic and authentic scenes.

An effective no-reference image quality index prediction with a hybrid Artificial Intelligence approach for denoised MRI images

As the quantity and significance of digital pictures in the medical industry continue to increase, Image Quality Assessment (IQA) has recently become a prevalent subject in the research community. Due to the wide range of distortions that Magnetic Resonance Images (MRI) can experience and the wide variety of information they contain, No-Reference Image Quality Assessment (NR-IQA) has always been a challenging study issue. In an attempt to address this issue, a novel hybrid Artificial Intelligence (AI) is proposed to analyze NR-IQ in massive MRI data. First, the features from the denoised MRI images are extracted using the gray level run length matrix (GLRLM) and EfficientNet B7 algorithm. Next, the Multi-Objective Reptile Search Algorithm (MRSA) was proposed for optimal feature vector selection. Then, the Self-evolving Deep Belief Fuzzy Neural network (SDBFN) algorithm was proposed for the effective NR-IQ analysis. The implementation of this research is executed using MATLAB software. The simulation results are compared with the various conventional methods in terms of correlation coefficient (PLCC), Root Mean Square Error (RMSE), Spearman Rank Order Correlation Coefficient (SROCC) and Kendall Rank Order Correlation Coefficient (KROCC), and Mean Absolute Error (MAE). In addition, our proposed approach yielded a quality number approximately we achieved significant 20% improvement than existing methods, with the PLCC parameter showing a notable increase compared to current techniques. Moreover, the RMSE number decreased by 12% when compared to existing methods. Graphical representations indicated mean MAE values of 0.02 for MRI knee dataset, 0.09 for MRI brain dataset, and 0.098 for MRI breast dataset, showcasing significantly lower MAE values compared to the baseline models.

Binning of biological image based on unsupervised semantic segmentation and no-reference image quality assessment

Binning of biological image based on unsupervised semantic segmentation and no-reference image quality assessment

No reference image quality assessment using gabor convolutional neural networks

With the emergence of image capturing devices and increase in usage of internet, massive volume of network data was occupied with digital images. For efficient data transmission, the image may undergo several processing units in from the point it captures to the display/storage device. It may result in loss of the perceptual image quality. Therefore, it is necessary to estimate the image quality to measure the quality of experience. It was found that the convolutional neural networks serve as potential tool for effective feature extraction in many image processing applications. Particularly, with the first layer as Gabor filters, the robustness of the network can be reinforced with learnable Gabor parameters. This paper proposes Gabor Convolutional Neural Network method for No-Reference Image Quality Assessment. Their well-defined spatial structured filters are promising in extracting quality features from the local patches and maps them to perceptual quality scores. Our proposed architecture was tested over synthetic and authentic databases such as LIVE, TID2013, CSIQ, LIVE-MD, MDID2016, LIVE Wild and KoNiQ-10k. The proposed approach was also tested on the Waterloo 3D phase-II database, which contains high-resolution images of both the eyes individually with their respective DMOS scores. The proposed approach out performs over LIVE-MD and LIVE Wild and competes with existing algorithm over other databases.

DistilIQA: Distilling Vision Transformers for no-reference perceptual CT image quality assessment

No-reference image quality assessment (IQA) is a critical step in medical image analysis, with the objective of predicting perceptual image quality without the need for a pristine reference image. The application of no-reference IQA to CT scans is valuable in providing an automated and objective approach to assessing scan quality, optimizing radiation dose, and improving overall healthcare efficiency. In this paper, we introduce DistilIQA, a novel distilled Vision Transformer network designed for no-reference CT image quality assessment. DistilIQA integrates convolutional operations and multi-head self-attention mechanisms by incorporating a powerful convolutional stem at the beginning of the traditional ViT network. Additionally, we present a two-step distillation methodology aimed at improving network performance and efficiency. In the initial step, a ”teacher ensemble network” is constructed by training five vision Transformer networks using a five-fold division schema. In the second step, a ”student network”, comprising of a single Vision Transformer, is trained using the original labeled dataset and the predictions generated by the teacher network as new labels. DistilIQA is evaluated in the task of quality score prediction from low-dose chest CT scans obtained from the LDCT and Projection data of the Cancer Imaging Archive, along with low-dose abdominal CT images from the LDCTIQAC2023 Grand Challenge. Our results demonstrate DistilIQA‘s remarkable performance in both benchmarks, surpassing the capabilities of various CNNs and Transformer architectures. Moreover, our comprehensive experimental analysis demonstrates the effectiveness of incorporating convolutional operations within the ViT architecture and highlights the advantages of our distillation methodology.

Semi-supervised correction model for turbulence-distorted images

Significant progress has been made in addressing turbulence distortion in recent years, but persistent challenges remain. Firstly, existing methods heavily rely on fully supervised optimization strategies and synthetic datasets, posing difficulties in effectively utilizing unlabeled real data for training. Secondly, most approaches construct networks in a straightforward manner, overlooking the representation model of phase distortion and point spread function (PSF) in spatial and channel dimensions. This oversight restricts the potential for distortion correction. To address these challenges, this paper proposes a semi-supervised atmospheric turbulence correction method based on the mean-teacher framework. Our approach imposes constraints on the unlabeled data of student networks using pseudo-labels generated by teacher networks, thereby enhancing the generalization ability by leveraging information from unlabeled data. Furthermore, we introduce to use no-reference image quality assessment criterion to select the most reliable pseudo-label for each unlabeled sample by predicting physical parameters that indicating the level of degradation. Additionally, we propose to combine sliding window-based self-attention with channel attention to facilitate local-global context interaction. This design is inspired by the representation of phase distortion and PSF, which can be characterized by coefficients and basis functions corresponding to the channel-wise representation of convolutional neural network features. Moreover, the base functions exhibit spatial correlation, akin to Zenike and Airy disks. Experimental results show that the proposed method surpasses state-of-the-art models.

Joint dual-teacher distillation and unsupervised fusion for unpaired real-world image dehazing

Existing learning-based dehazing algorithms struggle to deal with real world hazy images for lack of paired clean data. Moreover, most dehazing methods require significant computation and memory. To address the above problems, we propose a joint dual-teacher knowledge distillation and unsupervised fusion framework for single image dehazing in this paper. First, considering the complex degradation factors in real-world hazy images, two synthetic-to-real dehazing networks are explored to generate two preliminary dehazing results with the heterogeneous distillation strategy. Second, to get more qualified ground truth, an unsupervised adversarial fusion network is proposed to refine the preliminary outputs of teachers with unpaired clean images. In particular, the unpaired clean images are enhanced to deal with the dim artifacts. Furthermore, to alleviate the structure distortion in the unsupervised adversarial training, we constructed an intermediate image to constrain the output of the fusion network. Finally, considering the memory storage and computation overhead, an end-to-end lightweight student network is trained to learn the mapping from the original hazy image to the output of the fusion network. Experimental results demonstrate that the proposed method achieves state-of-the-art performance on real-world hazy images in terms of no-reference image quality assessment and the parameters.

Semi-TSGAN: Semi-Supervised Learning for Highlight Removal Based on Teacher-Student Generative Adversarial Network.

Despite recent notable advancements in highlight image restoration techniques, the dearth of annotated data and the lightweight deployment of highlight removal networks pose significant impediments to further advancements in the field. In this paper, to the best of our knowledge, we first propose a semi-supervised learning paradigm for highlight removal, merging the fusion version of a teacher-student model and a generative adversarial network, featuring a lightweight network architecture. Initially, we establish a dependable repository to house optimal predictions as pseudo ground truth through empirical analyses guided by the most reliable No-Reference Image Quality Assessment (NR-IQA) method. This method serves to assess rigorously the quality of model predictions. Subsequently, addressing concerns regarding confirmation bias, we integrate contrastive regularization into the framework to curtail the risk of overfitting on inaccurate labels. Finally, we introduce a comprehensive feature aggregation module and an extensive attention mechanism within the generative network, considering a balance between network performance and computational efficiency. Our experimental evaluations encompass comprehensive assessments on both full-reference and non-reference highlight benchmarks. The results demonstrate conclusively the substantive quantitative and qualitative enhancements achieved by our proposed algorithm in comparison to state-of-the-art methodologies.

Quality-aware blind image motion deblurring

Recent mapping-based motion deblurring methods lack the regularization of prior knowledge, resulting in an over-reliance on the training data and limited generalization ability. As deblurring aims to improve image quality, we quantitatively analyze and further discover the strong correlation between image quality and sharpness. Motivated by the above facts and notable accomplishments of recent no-reference image quality assessment (NR-IQA), we present a novel framework that incorporates quality knowledge into mapping-based deblurring models. Specifically, we extract quality-aware features from NR-IQA models as prior knowledge, and subsequently propose a prediction-based strategy and an encoder-reuse strategy to integrate knowledge into the encoder and decoder, respectively. After training, the model can simultaneously deblur images and predict quality features, indicating that it has grasped the knowledge and validating the effectiveness of the proposed embedding strategies. Extensive experimental results show that embedding quality knowledge consistently improves model performance and the model achieves state-of-the-art intra/cross-dataset results. Code and pre-trained models are available at https://github.com/esnthere/QAMD.

AM-BQA: Enhancing blind image quality assessment using attention retractable features and multi-dimensional learning

In the realm of no-reference image quality assessment (NR-IQA), acquiring pristine source content for reference is often unattainable. This absence of reference presents challenges in accurately estimating perceptual scores due to the diversity and complexity of distortion patterns. To tackle these challenges, we introduce a novel approach named AM-BQA: Enhancing Blind Image Quality Assessment using Attention Retractable Features and Multi-Dimensional Learning, designed to capture and analyze complex patterns. Our method involves several steps. Firstly, we extract crucial and intricate features using a vision transformer. Next, we employ a multi-head transpose attention block with a dual key, incorporating overlap convolution patches and transpose attention into these extracted features. Finally, the attention maps generated by this process pass through an attention retractable block and a weighted multi-head layer to calculate the final quality score. By employing this architecture, we enhance both global and local interactions between complex patches. To validate the effectiveness of our approach, we assess it on four standard datasets (LIVE, TID2013, CSIQ, and KADID-10 K), including both synthetic datasets. Additionally, we conduct experiments on authentic datasets and demonstrate that our model achieves state-of-the-art performance across multiple datasets. The source code and pretrained models are available on this GitHub repository: https://github.com/adhikariastha5/AM-BQA.

Attention-driven residual-dense network for no-reference image quality assessment

Attention-driven residual-dense network for no-reference image quality assessment

Joint Distortion Restoration and Quality Feature Learning for No-reference Image Quality Assessment

No-reference image quality assessment (NR-IQA) methods, inspired by the free energy principle, improve the accuracy of image quality prediction by simulating the human brain’s repair process for distorted images. However, existing methods use separate optimization schemes for distortion restoration and quality prediction, which undermines the accurate mapping of feature representations to quality scores. To address this issue, we propose a joint restoration and quality feature learning NR-IQA (RQFL-IQA) method to jointly tackle distortion image restoration and quality prediction within a unified framework. To accurately establish the quality reconstruction relationship between distorted and restored images, a hybrid loss function based on pixel-wise and structure-wise representations is used to improve the restoration capability of the image restoration network. The proposed RQFL-IQA exploits rich labels, including restored images and quality scores, to enable the model to learn more discriminative features and establish a more accurate mapping from feature representation to quality scores. In addition, to avoid the impact of poor restoration on quality prediction, we propose a module with a cleaning function to reweight the fusion of restored and primitive features to achieve more perceptual consistency in feature fusion. Experimental results on public IQA datasets show that the proposed RQFL-IQA is superior over existing methods.

Transformer-Based No-Reference Image Quality Assessment via Supervised Contrastive Learning

Image Quality Assessment (IQA) has long been a research hotspot in the field of image processing, especially No-Reference Image Quality Assessment (NR-IQA). Due to the powerful feature extraction ability, existing Convolution Neural Network (CNN) and Transformers based NR-IQA methods have achieved considerable progress. However, they still exhibit limited capability when facing unknown authentic distortion datasets. To further improve NR-IQA performance, in this paper, a novel supervised contrastive learning (SCL) and Transformer-based NR-IQA model SaTQA is proposed. We first train a model on a large-scale synthetic dataset by SCL (no image subjective score is required) to extract degradation features of images with various distortion types and levels. To further extract distortion information from images, we propose a backbone network incorporating the Multi-Stream Block (MSB) by combining the CNN inductive bias and Transformer long-term dependence modeling capability. Finally, we propose the Patch Attention Block (PAB) to obtain the final distorted image quality score by fusing the degradation features learned from contrastive learning with the perceptual distortion information extracted by the backbone network. Experimental results on six standard IQA datasets show that SaTQA outperforms the state-of-the-art methods for both synthetic and authentic datasets. Code is available at https://github.com/I2-Multimedia-Lab/SaTQA.

An image quality assessment method based on edge extraction and singular value for blurriness

The automatic assessment of perceived image quality is crucial in the field of image processing. To achieve this idea, we propose an image quality assessment (IQA) method for blurriness. The features of gradient and singular value were extracted in this method instead of the single feature in the traditional IQA algorithms. According to the insufficient size of existing public image quality assessment datasets to support deep learning, machine learning was introduced to fuse the features of multiple domains, and a new no-reference (NR) IQA method for blurriness denoted Feature fusion IQA(Ffu-IQA) was proposed. The Ffu-IQA uses a probabilistic model to estimate the probability of each edge detection blur in the image, and then uses machine learning to aggregate the probability information to obtain the edge quality score. After that uses the singular value obtained by singular value decomposition of the image matrix to calculate the singular value score. Finally, machine learning pooling is used to obtain the true quality score. Ffu-IQA achieves PLCC scores of 0.9570 and 0.9616 on CSIQ and TID2013, respectively, and SROCC scores of 0.9380 and 0.9531, which are better than most traditional image quality assessment methods for blurriness.