Image Quality Assessment Methods Research Articles

Application of Binary Image Quality Assessment Methods to Predict the Quality of Optical Character Recognition Results

One of the continuous challenges related to the growing popularity of mobile devices and embedded systems with limited memory and computational power is the development of relatively fast methods for real-time image and video analysis. One such example is Optical Character Recognition (OCR), which is usually too complex for such devices. Considering that images captured by cameras integrated into mobile devices may be acquired in uncontrolled lighting conditions, some quality issues related to non-uniform illumination may affect the image binarization results and further text recognition results. The solution proposed in this paper is related to a significant reduction in the computational burden, preventing the necessity of full text recognition. Conducting only the initial image binarization using various thresholding methods, the computation of the mutual similarities of binarization results is proposed, making it possible to build a simple model of binary image quality for a fast prediction of the OCR results’ quality. The experimental results provided in the paper obtained for the dataset of 1760 images, as well as the additional verification for a larger dataset, confirm the high correlation of the proposed quality model with text recognition results.

A new method for judging thermal image quality with applications

Infrared thermal imaging, a non-destructive testing technology, measures the surface temperature of objects. Assessing thermal image quality is crucial for image monitoring, system design, algorithm optimization, and benchmarking. However, developing objective metrics that align with human perception is challenging due to the distinct structure of thermal images, which often feature high background temperatures and minimal variance between objects and the background. Existing methods typically target specific local features or overall image contrast, but new measures are needed to bridge the gap between objective performance and the unique characteristics of thermal images.We propose a novel image quality assessment (IQA) method inspired by the human vision system, specifically designed for thermal images, harmonizing local and global data. The primary contributions include (1) innovative local, global, and hybrid thermal quality assessment methods that deliver precise image quality predictions without needing reference images, (2) an experimental analysis evaluating the developed blind thermal IQA measure’s applicability to various thermal images, and (3) a comprehensive analysis of traditional IQA measure-based methods applied to publicly accessible thermal databases. Extensive simulations demonstrate our method’s competitive performance and strong alignment with human perception of image quality.

Automated quality control analysis for American College of Radiology (ACR) digital mammography (DM) phantom images.

To develop and validate an automated software analysis method for mammography image quality assessment of the American College of Radiology (ACR) digital mammography (DM) phantom images. Twenty-seven DICOM images were acquired using Fuji mammography systems. All images were evaluated by three expert medical physicists using the Royal Australian and New Zealand College of Radiologists (RANZCR) mammography quality control guideline. To enhance the robustness and sensitivity assessment of our algorithm, an additional set of 12 images from a Hologic mammography system was included to test various phantom positional adjustments. The software automatically chose multiple regions of interest (ROIs) for analysis. A template matching method was primarily used for image analysis, followed by an additional method that locates and scores each target object (speck groups, fibers, and masses). The software performance shows a good to excellent agreement with the average scoring of observers (intraclass correlation coefficient [ICC] of 0.75, 0.79, 0.82 for speck groups, fibers, and masses, respectively). No significant differences were found in the scoring of target objects between human observers and the software. Both methods achieved scores meeting the pass criteria for speck groups and masses. Expert observers allocated lower scores to fiber objects, with diameters less than 0.61mm, when compared to the software. The software was able to accurately score objects when the phantom position changed by up to 25mm laterally, up to 5 degrees rotation, and overhanging the chest wall edge by up to 15mm. Automated software analysis is a feasible method that may help improve the consistency and reproducibility of mammography image quality assessment with reduced reliance on human interaction and processing time.

VISTA: vision improvement via split and reconstruct deep neural network for fundus image quality assessment

AbstractWidespread eye conditions such as cataracts, diabetic retinopathy, and glaucoma impact people worldwide. Ophthalmology uses fundus photography for diagnosing these retinal disorders, but fundus images are prone to image quality challenges. Accurate diagnosis hinges on high-quality fundus images. Therefore, there is a need for image quality assessment methods to evaluate fundus images before diagnosis. Consequently, this paper introduces a deep learning model tailored for fundus images that supports large images. Our division method centres on preserving the original image’s high-resolution features while maintaining low computing and high accuracy. The proposed approach encompasses two fundamental components: an autoencoder model for input image reconstruction and image classification to classify the image quality based on the latent features extracted by the autoencoder, all performed at the original image size, without alteration, before reassembly for decoding networks. Through post hoc interpretability methods, we verified that our model focuses on key elements of fundus image quality. Additionally, an intrinsic interpretability module has been designed into the network that allows decomposing class scores into underlying concepts quality such as brightness or presence of anatomical structures. Experimental results in our model with EyeQ, a fundus image dataset with three categories (Good, Usable, and Rejected) demonstrate that our approach produces competitive outcomes compared to other deep learning-based methods with an overall accuracy of 0.9066, a precision of 0.8843, a recall of 0.8905, and an impressive F1-score of 0.8868. The code is publicly available at https://github.com/saifalkhaldiurv/VISTA_-Image-Quality-Assessment.

Intensity-sensitive Quality Assessment of Extended Sources in Astronomical Images

Radio astronomy studies the Universe by observing the radio emissions of celestial bodies. Different methods can be used to recover the sky brightness distribution (SBD), which describes the distribution of celestial sources from recorded data, with the output dependent on the method used. Image quality assessment (IQA) indexes can be used to compare the differences between restored SBDs produced by different image reconstruction techniques to evaluate their effectiveness. However, reconstructed images (for the same SBD) can appear to be very similar, especially when observed by the human visual system (HVS). Hence, current structural similarity methods, inspired by the HVS, are not effective. In the past, we have proposed two methods to assess point-source images, where low amounts of concentrated information are present in larger regions of noise-like data. But for images that include extended source(s), the increase in complexity of the structure makes the IQA methods for point sources oversensitive because the important objects cannot be described by isolated point sources. Therefore, in this article we propose the augmented low-information similarity index (augLISI), an improved version of LISI, to assess images including extended source(s). Experiments have been carried out to illustrate how this new IQA method can help with the development and study of astronomical imaging techniques. Note that although we focus on radio astronomical images herein, these IQA methods are also applicable to other astronomical images and imaging techniques.

Overview of High-Dynamic-Range Image Quality Assessment.

In recent years, the High-Dynamic-Range (HDR) image has gained widespread popularity across various domains, such as the security, multimedia, and biomedical fields, owing to its ability to deliver an authentic visual experience. However, the extensive dynamic range and rich detail in HDR images present challenges in assessing their quality. Therefore, current efforts involve constructing subjective databases and proposing objective quality assessment metrics to achieve an efficient HDR Image Quality Assessment (IQA). Recognizing the absence of a systematic overview of these approaches, this paper provides a comprehensive survey of both subjective and objective HDR IQA methods. Specifically, we review 7 subjective HDR IQA databases and 12 objective HDR IQA metrics. In addition, we conduct a statistical analysis of 9 IQA algorithms, incorporating 3 perceptual mapping functions. Our findings highlight two main areas for improvement. Firstly, the size and diversity of HDR IQA subjective databases should be significantly increased, encompassing a broader range of distortion types. Secondly, objective quality assessment algorithms need to identify more generalizable perceptual mapping approaches and feature extraction methods to enhance their robustness and applicability. Furthermore, this paper aims to serve as a valuable resource for researchers by discussing the limitations of current methodologies and potential research directions in the future.

Image Quality Difference Perception Ability: A BIQA model effectiveness metric based on model falsification method

Blind Image Quality Assessment (BIQA) methods refer to algorithms that predict image quality scores without reference images. This challenging area is crucial for preprocessing and optimizing visual tasks. Since BIQA is a small-sample task, it has become a common practice to apply transfer learning by using models pre-trained on other visual tasks, such as semantic recognition, to BIQA. This paper is interested in whether a backbone that performs better in semantic recognition also improves predictive accuracy in BIQA tasks. Comparative experiments showed that different semantic backbones with varying precision exhibit minimal differences in PLCC (Pearson Linear Correlation Coefficient) and SRCC (Spearman Rank Order Correlation Coefficient). However, further comparison shows that the semantic backbone does enhance the model’s IQA abilities, which traditional metrics fail to capture due to their oversight of quality differences between images. We term this ability as Image Quality Difference Perception Ability (IQDP Ability). Based on this, we propose an IQDP comparison method and a new metric, which can effectively compare and measure a model’s IQDP Ability, supplementing traditional metrics and providing an effective means of identifying superior models.

Low-dose computed tomography perceptual image quality assessment

In computed tomography (CT) imaging, optimizing the balance between radiation dose and image quality is crucial due to the potentially harmful effects of radiation on patients. Although subjective assessments by radiologists are considered the gold standard in medical imaging, these evaluations can be time-consuming and costly. Thus, objective methods, such as the peak signal-to-noise ratio and structural similarity index measure, are often employed as alternatives. However, these metrics, initially developed for natural images, may not fully encapsulate the radiologists’ assessment process. Consequently, interest in developing deep learning-based image quality assessment (IQA) methods that more closely align with radiologists’ perceptions is growing. A significant barrier to this development has been the absence of open-source datasets and benchmark models specific to CT IQA. Addressing these challenges, we organized the Low-dose Computed Tomography Perceptual Image Quality Assessment Challenge in conjunction with the Medical Image Computing and Computer Assisted Intervention 2023. This event introduced the first open-source CT IQA dataset, consisting of 1,000 CT images of various quality, annotated with radiologists’ assessment scores. As a benchmark, this challenge offers a comprehensive analysis of six submitted methods, providing valuable insight into their performance. This paper presents a summary of these methods and insights. This challenge underscores the potential for developing no-reference IQA methods that could exceed the capabilities of full-reference IQA methods, making a significant contribution to the research community with this novel dataset. The dataset is accessible at https://zenodo.org/records/7833096.

Full-reference calibration-free image quality assessment

Objective Image Quality Assessment (IQA) methods often lack of linearity of their quality estimates with respect to scores expressed by human subjects and therefore IQA metrics undergo a calibration process based on subjective quality examples. However, example-based training presents a challenge in terms of generalization hampering result comparison across different applications and operative conditions. In this paper, new Full Reference (FR) techniques, providing estimates linearly correlated with human scores without using calibration are introduced. We show that on natural images, application of estimation theory and psychophysical principles to images degraded by Gaussian blur leads to a so-called canonical IQA method, whose estimates are linearly correlated to both the subjective scores and the viewing distance. Then, we show that any mainstream IQA methods can be reconducted to the canonical method by converting its metric based on a unique specimen image. The proposed scheme is extended to wide classes of degraded images, e.g. noisy and compressed images. The resulting calibration-free FR IQA methods allows for comparability and interoperability across different imaging systems and on different viewing distances. A comparison of their statistical performance with respect to state-of-the-art calibration prone methods is finally provided, showing that the presented model is a valid alternative to the final 5-parameter calibration step of IQA methods, and the two parameters of the model have a clear operational meaning and are simply determined in practical applications. The enhanced performance are achieved across multiple viewing distance databases by independently realigning the blur values associated with each distance.

ConIQA: A deep learning method for perceptual image quality assessment with limited data

Effectively assessing the realism and naturalness of images in virtual (VR) and augmented (AR) reality applications requires Full Reference Image Quality Assessment (FR-IQA) metrics that closely align with human perception. Deep learning-based IQAs that are trained on human-labeled data have recently shown promise in generic computer vision tasks. However, their performance decreases in applications where perfect matches between the reference and the distorted images should not be expected, or whenever distortion patterns are restricted to specific domains. Tackling this issue necessitates training a task-specific neural network, yet generating human-labeled FR-IQAs is costly, and deep learning typically demands substantial labeled data. To address these challenges, we developed ConIQA, a deep learning-based IQA that leverages consistency training and a novel data augmentation method to learn from both labeled and unlabeled data. This makes ConIQA well-suited for contexts with scarce labeled data. To validate ConIQA, we considered the example application of Computer-Generated Holography (CGH) where specific artifacts such as ringing, speckle, and quantization errors routinely occur, yet are not explicitly accounted for by existing IQAs. We developed a new dataset, HQA1k, that comprises 1000 natural images each paired with an image rendered using various popular CGH algorithms, and quality-rated by thirteen human participants. Our results show that ConIQA achieves superior Pearson (0.98), Spearman (0.965), and Kendall’s tau (0.86) correlations over fifteen FR-IQA metrics by up to 5%, showcasing significant improvements in aligning with human perception on the HQA1k dataset.

Joint Luminance-Saliency Prior and Attention for Underwater Image Quality Assessment

Underwater images, as a crucial medium for storing ocean information in underwater sensors, play a vital role in various underwater tasks. However, they are prone to distortion due to the imaging environment, which leads to a decline in visual quality, which is an urgent issue for various marine vision systems to address. Therefore, it is necessary to develop underwater image enhancement (UIE) and corresponding quality assessment methods. At present, most underwater image quality assessment (UIQA) methods primarily rely on extracting handcrafted features that characterize degradation attributes, which struggle to measure complex mixed distortions and often exhibit discrepancies with human visual perception in practical applications. Furthermore, current UIQA methods lack the consideration of the perception perspective of enhanced effects. To this end, this paper employs luminance and saliency priors as critical visual information for the first time to measure the enhancement effect of global and local quality achieved by the UIE algorithms, named JLSAU. The proposed JLSAU is built upon an overall pyramid-structured backbone, supplemented by the Luminance Feature Extraction Module (LFEM) and Saliency Weight Learning Module (SWLM), which aim at obtaining perception features with luminance and saliency priors at multiple scales. The supplement of luminance priors aims to perceive visually sensitive global distortion of luminance, including histogram statistical features and grayscale features with positional information. The supplement of saliency priors aims to perceive visual information that reflects local quality variation both in spatial and channel domains. Finally, to effectively model the relationship among different levels of visual information contained in the multi-scale features, the Attention Feature Fusion Module (AFFM) is proposed. Experimental results on the public UIQE and UWIQA datasets demonstrate that the proposed JLSAU outperforms existing state-of-the-art UIQA methods.

Omnidirectional image quality assessment with local–global vision transformers

With the rising popularity of omnidirectional images (ODIs) in virtual reality applications, the need for specialized image quality assessment (IQA) methods becomes increasingly critical. Traditional IQA approaches, designed for rectilinear images, often fail to evaluate ODIs accurately due to their 360-degree scene representation. Addressing this, we introduce the Local–Global Transformer for 360-degree Image Quality Assessment (LGT360IQ). This novel framework features dual branches tailored to mimic top-down and bottom-up visual attention mechanisms, adapted for the spherical characteristics of ODIs. The local branch processes tangent viewports from salient regions within the equirectangular projection image, extracting detailed features for granular quality assessment. In parallel, the global branch utilizes a task-dependent token sampling strategy for holistic image feature processing and quality score prediction. This integrated approach combines local and global information, offering an effective IQA method for ODIs. Our extensive evaluation across three benchmark ODI datasets, CVIQ, OIQA, and ODI, demonstrates LGT360IQ superior performance and establishes its role in advancing the field of IQA for omnidirectional images.

Machine Learning and Deep Learning Methods for Fast and Accurate Assessment of Transthoracic Echocardiogram Image Quality.

High-quality echocardiogram images are the cornerstone of accurate and reliable measurements of the heart. Therefore, this study aimed to develop, validate and compare machine learning and deep learning algorithms for accurate and automated assessment of transthoracic echocardiogram image quality. In total, 4090 single-frame two-dimensional transthoracic echocardiogram images were used from apical 4-chamber, apical 2-chamber and parasternal long-axis views sampled from 3530 adult patients. The data were extracted from CAMUS and Unity Imaging open-source datasets. For every raw image, additional grayscale block histograms were developed. For block histogram datasets, six classic machine learning algorithms were tested. Moreover, convolutional neural networks based on the pre-trained EfficientNetB4 architecture were developed for raw image datasets. Classic machine learning algorithms predicted image quality with 0.74 to 0.92 accuracy (AUC 0.81 to 0.96), whereas convolutional neural networks achieved between 0.74 and 0.89 prediction accuracy (AUC 0.79 to 0.95). Both approaches are accurate methods of echocardiogram image quality assessment. Moreover, this study is a proof of concept of a novel method of training classic machine learning algorithms on block histograms calculated from raw images. Automated echocardiogram image quality assessment methods may provide additional relevant information to the echocardiographer in daily clinical practice and improve reliability in clinical decision making.

Is the development of objective image quality assessment methods keeping pace with technological developments?

Is the development of objective image quality assessment methods keeping pace with technological developments?

No reference image quality assessment using gabor convolutional neural networks

With the emergence of image capturing devices and increase in usage of internet, massive volume of network data was occupied with digital images. For efficient data transmission, the image may undergo several processing units in from the point it captures to the display/storage device. It may result in loss of the perceptual image quality. Therefore, it is necessary to estimate the image quality to measure the quality of experience. It was found that the convolutional neural networks serve as potential tool for effective feature extraction in many image processing applications. Particularly, with the first layer as Gabor filters, the robustness of the network can be reinforced with learnable Gabor parameters. This paper proposes Gabor Convolutional Neural Network method for No-Reference Image Quality Assessment. Their well-defined spatial structured filters are promising in extracting quality features from the local patches and maps them to perceptual quality scores. Our proposed architecture was tested over synthetic and authentic databases such as LIVE, TID2013, CSIQ, LIVE-MD, MDID2016, LIVE Wild and KoNiQ-10k. The proposed approach was also tested on the Waterloo 3D phase-II database, which contains high-resolution images of both the eyes individually with their respective DMOS scores. The proposed approach out performs over LIVE-MD and LIVE Wild and competes with existing algorithm over other databases.

A Systematic Survey of Deep Learning-Based Single-Image Super-Resolution

Single-image super-resolution (SISR) is an important task in image processing, which aims to enhance the resolution of imaging systems. Recently, SISR has made a huge leap and has achieved promising results with the help of deep learning (DL). In this survey, we give an overview of DL-based SISR methods and group them according to their design targets. Specifically, we first introduce the problem definition, research background, and the significance of SISR. Secondly, we introduce some related works, including benchmark datasets, upsampling methods, optimization objectives, and image quality assessment methods. Thirdly, we provide a detailed investigation of SISR and give some domain-specific applications of it. Fourthly, we present the reconstruction results of some classic SISR methods to intuitively know their performance. Finally, we discuss some issues that still exist in SISR and summarize some new trends and future directions. This is an exhaustive survey of SISR, which can help researchers better understand SISR and inspire more exciting research in this field. An investigation project for SISR is provided at https://github.com/CV-JunchengLi/SISR-Survey.

Semi-TSGAN: Semi-Supervised Learning for Highlight Removal Based on Teacher-Student Generative Adversarial Network.

Despite recent notable advancements in highlight image restoration techniques, the dearth of annotated data and the lightweight deployment of highlight removal networks pose significant impediments to further advancements in the field. In this paper, to the best of our knowledge, we first propose a semi-supervised learning paradigm for highlight removal, merging the fusion version of a teacher-student model and a generative adversarial network, featuring a lightweight network architecture. Initially, we establish a dependable repository to house optimal predictions as pseudo ground truth through empirical analyses guided by the most reliable No-Reference Image Quality Assessment (NR-IQA) method. This method serves to assess rigorously the quality of model predictions. Subsequently, addressing concerns regarding confirmation bias, we integrate contrastive regularization into the framework to curtail the risk of overfitting on inaccurate labels. Finally, we introduce a comprehensive feature aggregation module and an extensive attention mechanism within the generative network, considering a balance between network performance and computational efficiency. Our experimental evaluations encompass comprehensive assessments on both full-reference and non-reference highlight benchmarks. The results demonstrate conclusively the substantive quantitative and qualitative enhancements achieved by our proposed algorithm in comparison to state-of-the-art methodologies.

G2NPAN: GAN-guided nuance perceptual attention network for multimodal medical fusion image quality assessment.

Multimodal medical fusion images (MMFI) are formed by fusing medical images of two or more modalities with the aim of displaying as much valuable information as possible in a single image. However, due to the different strategies of various fusion algorithms, the quality of the generated fused images is uneven. Thus, an effective blind image quality assessment (BIQA) method is urgently required. The challenge of MMFI quality assessment is to enable the network to perceive the nuances between fused images of different qualities, and the key point for the success of BIQA is the availability of valid reference information. To this end, this work proposes a generative adversarial network (GAN) -guided nuance perceptual attention network (G2NPAN) to implement BIQA for MMFI. Specifically, we achieve the blind evaluation style via the design of a GAN and develop a Unique Feature Warehouse module to learn the effective features of fused images from the pixel level. The redesigned loss function guides the network to perceive the image quality. In the end, the class activation mapping supervised quality assessment network is employed to obtain the MMFI quality score. Extensive experiments and validation have been conducted in a database of medical fusion images, and the proposed method is superior to the state-of-the-art BIQA method.

Review of Image Quality Assessment Methods for Compressed Images.

The compression of images for efficient storage and transmission is crucial in handling large data volumes. Lossy image compression reduces storage needs but introduces perceptible distortions affected by content, compression levels, and display environments. Each compression method generates specific visual anomalies like blocking, blurring, or color shifts. Standardizing efficient lossy compression necessitates evaluating perceptual quality. Objective measurements offer speed and cost efficiency, while subjective assessments, despite their cost and time implications, remain the gold standard. This paper delves into essential research queries to achieve visually lossless images. The paper describes the influence of compression on image quality, appropriate objective image quality metrics (IQMs), and the effectiveness of subjective assessment methods. It also provides an overview of the existing literature, surveys, and subjective and objective image quality assessment (IQA) methods. Our aim is to offer insights, identify challenges in existing methodologies, and assist researchers in selecting the most effective assessment approach for their needs.

Hierarchical prior-guided quality assessment method for underwater images

Many underwater image enhancement (UIE) algorithms have been proposed to improve the quality of the underwater image. However, the lack of effective objective image quality assessment (IQA) methods has hindered the further development of UIE algorithms. Unlike traditional distorted images, underwater images usually contain natural distortions such as color cast and fog effects induced by light absorption and scattering. In addition, due to the complexity of underwater scenes, the enhanced results generated using existing UIE algorithms may have artificial distortions such as under-enhancement, over-enhancement, and detail loss. Thus, it is difficult for traditional IQA methods to comprehensively capture quality-related features in complicated underwater conditions. Based on this, this paper proposes a new hierarchical prior-guided quality assessment method for underwater images with the aid of deep learning technology, which first decomposes the underwater image into the base layer and detail layer. Then, the two layers are taken as input of feature extraction network for extracting the luminance information and texture information of the image, respectively. Meanwhile, two pseudo-reference images are used to guide model training so as to obtain luminance and texture features more accurately and efficiently. Finally, the two types of features are fused to obtain the quality score of the underwater image. Extensive experiments have been conducted on two publicly available underwater image datasets, and the experimental results show that the proposed method outperforms other underwater image quality assessment methods.