Traditional Image Quality Metrics Research Articles

Key-Point-Descriptor-Based Image Quality Evaluation in Photogrammetry Workflows

Photogrammetry depends critically on the quality of the images used to reconstruct accurate and detailed 3D models. Selection of high-quality images not only improves the accuracy and resolution of the resulting 3D models, but also contributes to the efficiency of the photogrammetric process by reducing data redundancy and computational demands. This study presents a novel approach to image quality evaluation tailored for photogrammetric applications that uses the key point descriptors typically encountered in image matching. Using a LightGBM ranker model, this research evaluates the effectiveness of key point descriptors such as SIFT, SURF, BRISK, ORB, KAZE, FREAK, and SuperPoint in predicting image quality. These descriptors are evaluated for their ability to indicate image quality based on the image patterns they capture. Experiments conducted on various publicly available image datasets show that descriptor-based methods outperform traditional no-reference image quality metrics such as BRISQUE, NIQE, PIQE, and BIQAA and a simple sharpness-based image quality evaluation method. The experimental results highlight the potential of using key-point-descriptor-based image quality evaluation methods to improve the photogrammetric workflow by selecting high-quality images for 3D modeling.

Deep learning generation of preclinical positron emission tomography (PET) images from low-count PET with task-based performance assessment.

Preclinical low-count positron emission tomography (LC-PET) imaging offers numerous advantages such as facilitating imaging logistics, enabling longitudinal studies of long- and short-lived isotopes as well as increasing scanner throughput. However, LC-PET is characterized by reduced photon-count levels resulting in low signal-to-noise ratio (SNR), segmentation difficulties, and quantification uncertainties. We developed and evaluated a novel deep-learning (DL) architecture-Attention based Residual-Dilated Net (ARD-Net)-to generate standard-count PET (SC-PET) images from LC-PET images. The performance of the ARD-Net framework was evaluated for numerous low count realizations using fidelity-based qualitative metrics, task-based segmentation, and quantitative metrics. Patient Derived tumor Xenograft (PDX) with tumors implanted in the mammary fat-pad were subjected to preclinical [18F]-Fluorodeoxyglucose (FDG)-PET/CT imaging. SC-PET images were derived from a 10min static FDG-PET acquisition, 50min post administration of FDG, and were resampled to generate four distinct LC-PET realizations corresponding to 10%, 5%, 1.6%, and 0.8% of SC-PET count-level. ARD-Net was trained and optimized using 48 preclinical FDG-PET datasets, while 16 datasets were utilized to assess performance. Further, the performance of ARD-Net was benchmarked against two leading DL-based methods (Residual UNet, RU-Net; and Dilated Network, D-Net) and non-DL methods (Non-Local Means, NLM; and Block Matching 3D Filtering, BM3D). The performance of the framework was evaluated using traditional fidelity-based image quality metrics such as Structural Similarity Index Metric (SSIM) and Normalized Root Mean Square Error (NRMSE), as well as human observer-based tumor segmentation performance (Dice Score and volume bias) and quantitative analysis of Standardized Uptake Value (SUV) measurements. Additionally, radiomics-derived features were utilized as a measure of quality assurance (QA) in comparison to true SC-PET. Finally, a performance ensemble score (EPS) was developed by integrating fidelity-based and task-based metrics. Concordance Correlation Coefficient (CCC) was utilized to determine concordance between measures. The non-parametric Friedman Test with Bonferroni correction was used to compare the performance of ARD-Net against benchmarked methods with significance at adjusted p-value ≤0.01. ARD-Net-generated SC-PET images exhibited significantly better (p ≤ 0.01 post Bonferroni correction) overall image fidelity scores in terms of SSIM and NRMSE at majority of photon-count levels compared to benchmarked DL and non-DL methods. In terms of task-based quantitative accuracy evaluated by SUVMeanand SUVPeak, ARD-Net exhibited less than 5% median absolute bias for SUVMean compared to true SC-PET and lower degree of variability compared to benchmarked DL and non-DL based methods in generating SC-PET. Additionally, ARD-Net-generated SC-PET images displayed higher degree of concordance to SC-PET images in terms of radiomics features compared to non-DL and other DL approaches. Finally, the ensemble score suggested that ARD-Net exhibited significantly superior performance compared to benchmarked algorithms (p ≤ 0.01 post Bonferroni correction). ARD-Net provides a robust framework to generate SC-PET from LC-PET images. ARD-Net generated SC-PET images exhibited superior performance compared other DL and non-DL approaches in terms of image-fidelity based metrics, task-based segmentation metrics, and minimal bias in terms of task-based quantification performance for preclinical PET imaging.

Image Quality Assessment for Realistic Zoom Photos

New CMOS imaging sensor (CIS) techniques in smartphones have helped user-generated content dominate our lives over traditional DSLRs. However, tiny sensor sizes and fixed focal lengths also lead to more grainy details, especially for zoom photos. Moreover, multi-frame stacking and post-sharpening algorithms would produce zigzag textures and over-sharpened appearances, for which traditional image-quality metrics may over-estimate. To solve this problem, a real-world zoom photo database is first constructed in this paper, which includes 900 tele-photos from 20 different mobile sensors and ISPs. Then we propose a novel no-reference zoom quality metric which incorporates the traditional estimation of sharpness and the concept of image naturalness. More specifically, for the measurement of image sharpness, we are the first to combine the total energy of the predicted gradient image with the entropy of the residual term under the framework of free-energy theory. To further compensate for the influence of over-sharpening effect and other artifacts, a set of model parameters of mean subtracted contrast normalized (MSCN) coefficients are utilized as the natural statistics representatives. Finally, these two measures are combined linearly. Experimental results on the zoom photo database demonstrate that our quality metric can achieve SROCC and PLCC over 0.91, while the performance of single sharpness or naturalness index is around 0.85. Moreover, compared with the best tested general-purpose and sharpness models, our zoom metric outperforms them by 0.072 and 0.064 in SROCC, respectively.

What are we looking at? An investigation on the use of deep learning models for image quality assessment

In recent years several different Image Quality Metrics (IQMs) have been introduced which are focused on comparing the feature maps extracted from different pre-trained deep learning models[1-3]. While such objective IQMs have shown a high correlation with the subjective scores little attention has been paid on how they could be used to better understand the Human Visual System (HVS) and how observers evaluate the quality of images. In this study, by using different pre-trained Convolutional Neural Networks (CNN) models we identify the most relevant features in Image Quality Assessment (IQA). By visualizing these feature maps we try to have a better understanding about which features play a dominant role when evaluating the quality of images. Experimental results on four benchmark datasets show that the most important feature maps represent repeated textures such as stripes or checkers, and feature maps linked to colors blue, or orange also play a crucial role. Additionally, when it comes to calculating the quality of an image based on a comparison of different feature maps, a higher accuracy can be reached when only the most relevant feature maps are used in calculating the image quality instead of using all the extracted feature maps from a CNN model. [1] Amirshahi, Seyed Ali, Marius Pedersen, and Stella X. Yu. "Image quality assessment by comparing CNN features between images." Journal of Imaging Science and Technology 60.6 (2016): 60410-1. [2] Amirshahi, Seyed Ali, Marius Pedersen, and Azeddine Beghdadi. "Reviving traditional image quality metrics using CNNs." Color and imaging conference. Vol. 2018. No. 1. Society for Imaging Science and Technology, 2018. [3] Gao, Fei, et al. "Deepsim: Deep similarity for image quality assessment." Neurocomputing 257 (2017): 104-114.

CT Noise-Reduction Methods for Lower-Dose Scanning: Strengths and Weaknesses of Iterative Reconstruction Algorithms and New Techniques.

Iterative reconstruction (IR) algorithms are the most widely used CT noise-reduction method to improve image quality and have greatly facilitated radiation dose reduction within the radiology community. Various IR methods have different strengths and limitations. Because IR algorithms are typically nonlinear, they can modify spatial resolution and image noise texture in different regions of the CT image; hence traditional image-quality metrics are not appropriate to assess the ability of IR to preserve diagnostic accuracy, especially for low-contrast diagnostic tasks. In this review, the authors highlight emerging IR algorithms and CT noise-reduction techniques and summarize how these techniques can be evaluated to help determine the appropriate radiation dose levels for different diagnostic tasks in CT. In addition to advanced IR techniques, we describe novel CT noise-reduction methods based on convolutional neural networks (CNNs). CNN-based noise-reduction techniques may offer the ability to reduce image noise while maintaining high levels of image detail but may have unique drawbacks. Other novel CT noise-reduction methods are being developed to leverage spatial and/or spectral redundancy in multiphase or multienergy CT. Radiologists and medical physicists should be familiar with these different alternatives to adapt available CT technology for different diagnostic tasks. The scope of this article is (a) to review the clinical applications of IR algorithms as well as their strengths, weaknesses, and methods of assessment and (b) to explore new CT image reconstruction and noise-reduction techniques that promise to facilitate radiation dose reduction. ©RSNA, 2021.

Application of the generalized contrast-to-noise ratio to assess photoacoustic image quality.

The generalized contrast-to-noise ratio (gCNR) is a relatively new image quality metric designed to assess the probability of lesion detectability in ultrasound images. Although gCNR was initially demonstrated with ultrasound images, the metric is theoretically applicable to multiple types of medical images. In this paper, the applicability of gCNR to photoacoustic images is investigated. The gCNR was computed for both simulated and experimental photoacoustic images generated by amplitude-based (i.e., delay-and-sum) and coherence-based (i.e., short-lag spatial coherence) beamformers. These gCNR measurements were compared to three more traditional image quality metrics (i.e., contrast, contrast-to-noise ratio, and signal-to-noise ratio) applied to the same datasets. An increase in qualitative target visibility generally corresponded with increased gCNR. In addition, gCNR magnitude was more directly related to the separability of photoacoustic signals from their background, which degraded with the presence of limited bandwidth artifacts and increased levels of channel noise. At high gCNR values (i.e., 0.95-1), contrast, contrast-to-noise ratio, and signal-to-noise ratio varied by up to 23.7-56.2 dB, 2.0-3.4, and 26.5-7.6×1020, respectively, for simulated, experimental phantom, and in vivo data. Therefore, these traditional metrics can experience large variations when a target is fully detectable, and additional increases in these values would have no impact on photoacoustic target detectability. In addition, gCNR is robust to changes in traditional metrics introduced by applying a minimum threshold to image amplitudes. In tandem with other photoacoustic image quality metrics and with a defined range of 0 to 1, gCNR has promising potential to provide additional insight, particularly when designing new beamformers and image formation techniques and when reporting quantitative performance without an opportunity to qualitatively assess corresponding images (e.g., in text-only abstracts).

No-reference synthetic image quality assessment with convolutional neural network and local image saliency

Depth-image-based rendering (DIBR) is widely used in 3DTV, free-viewpoint video, and interactive 3D graphics applications. Typically, synthetic images generated by DIBR-based systems incorporate various distortions, particularly geometric distortions induced by object dis-occlusion. Ensuring the quality of synthetic images is critical to maintaining adequate system service. However, traditional 2D image quality metrics are ineffective for evaluating synthetic images as they are not sensitive to geometric distortion. In this paper, we propose a novel no-reference image quality assessment method for synthetic images based on convolutional neural networks, introducing local image saliency as prediction weights. Due to the lack of existing training data, we construct a new DIBR synthetic image dataset as part of our contribution. Experiments were conducted on both the public benchmark IRCCyN/IVC DIBR image dataset and our own dataset. Results demonstrate that our proposed metric outperforms traditional 2D image quality metrics and state-of-the-art DIBR-related metrics.

Prospective Image Quality Analysis and Control for Prior-Image-Based Reconstruction of Low-Dose CT.

Prior-image-based reconstruction (PIBR) is a powerful tool for low-dose CT, however, the nonlinear behavior of such approaches are generally difficult to predict and control. Similarly, traditional image quality metrics do not capture potential biases exhibited in PIBR images. In this work, we identify a new bias metric and construct an analytical framework for prospectively predicting and controlling the relationship between prior image regularization strength and this bias in a reliable and quantitative fashion. Bias associated with prior image regularization in PIBR can be described as the fraction of actual contrast change (between the prior image and current anatomy) that appears in the reconstruction. Using local approximation of the nonlinear PIBR objective, we develop an analytical relationship between local regularization, fractional contrast reconstructed, and true contrast change. This analytic tool allows prediction bias properties in a reconstructed PIBR image and includes the dependencies on the data acquisition, patient anatomy and change, and reconstruction parameters. Predictions are leveraged to provide reliable and repeatable image properties for varying data fidelity in simulation and physical cadaver experiments. The proposed analytical approach permits accurate prediction of reconstructed contrast relative to a gold standard based on exhaustive search based on numerous iterative reconstructions. The framework is used to control regularization parameters to enforce consistent change reconstructions over varying fluence levels and varying numbers of projection angles - enabling bias properties that are less location- and acquisition-dependent. While PIBR methods have demonstrated a substantial ability for dose reduction, image properties associated with those images have been difficult to express and quantify using traditional metrics. The novel framework presented in this work not only quantifies this bias in an intuitive fashion, but it gives a way to predict and control the bias. Reliable and predictable reconstruction methods are a requirement for clinical imaging systems and the proposed framework is an important step translating PIBR methods to clinical application.

Reduced-reference quality assessment of DIBR-synthesized images based on multi-scale edge intensity similarity

Depth-image-based-rendering (DIBR) plays an important role in view synthesis for free-viewpoint videos. The warping process in DIBR causes geometric displacement, which distributes intensively around edges, and the subsequent rendering process results in the impairment of edges. Traditional 2D image quality metrics are limited in the quality evaluation of DIBR-synthesized images. In this paper, we present a reduced-reference quality metric for DIBR-synthesized images by only extracting several feature values, namely multi-scale Edge Intensity Similarity (EIS). The original and synthesized images are first downsampled to generate images with different resolutions. Then an edge detection process is conducted on each scale and the edge intensity is calculated. The similarity of the edge intensity between each downsampled original image and the corresponding synthesized image is computed. Finally, the average similarity is calculated as the quality score of the DIBR-synthesized image. Experiments conducted on IRCCyN/IVC DIBR image and video databases demonstrate that the proposed method overall outperforms traditional 2D and existing DIBR-targeted quality metrics.

Objective and Optical Evaluation of Despeckle Filters in Breast Ultrasound Images

ABSTRACTSpeckle removal from breast ultrasound images result in blurring of lesion margin, echo pattern, and other sharp details which may carry important diagnostic information. Traditional image quality metrics fail to quantify the losses, dislocation of edges, and other sharp image features. Hence, an image quality metric is needed to measure the edge-retrieval capability of filter so that blurring, loss, and dislocation of edges can be quantified. In this paper, a novel image quality metric called edge retrieval index (in short, ERI) is proposed and utilized to evaluate state-of-art speckle removal techniques in breast ultrasound images along with traditional image quality measures. Besides the objective evaluation, optical evaluation is also carried out by two experts in extracting important diagnostic information from breast ultrasound images. Experiments were conducted on 28 real-time breast ultrasound images on MATLAB® software platform. The results of objective evaluation show that wavelet filter with first-level decomposition and eliminated HH band (WAV (HH/1)) gives the best performance followed by homomorphic filter with wavelet filter function (WAV (HH/1)) and anisotropic diffusion filter (PM2). On the other hand, wavelet filter (WAV (HL/1)) performed best in restoring edges while providing the highest smoothness. Furthermore, anisotropic diffusion filter (PM2) outperformed others, followed by homomorphic filter with wavelet filter function (WAV (HH/1)), in optical evaluation carried out by two experts. It was shown that ERI can be successfully used to measure the edge-preserving ability of the filter and can give significant information when conventional metrics fail to assess the image quality obtained with filtering. Future work is needed to evaluate various filters in clinical practice along with segmentation, feature extraction, and classification.

An object-level strategy for pan-sharpening quality assessment of high-resolution satellite imagery

Current satellite imaging systems offer a trade-off between high spatial and high spectral resolution providing panchromatic images at a higher spatial resolution and multispectral images at a lower spatial resolution but rich in spectral information while a wide range of applications need the highest level of this information, simultaneously. Image fusion techniques as means of enhancing the information content of initial panchromatic and multispectral images produce new images, titled pan-sharpened, which inherent the advantages of the initial images. Considering the impact of fusion accuracy on the quality of corresponding applications, it is necessary to evaluate the quality of these processed images. During the last decade, a lot of quality evaluation metrics have been proposed which are mostly inspired by traditional image quality metrics. These methods are mostly based on applying quality metrics at the pixel level and evaluating final quality value based on averaging of obtained metric values through the whole image. However, obtained results clearly show that the behaviour of image fusion quality is inconsistent amongst different image objects. In this article, by applying image fusion quality metrics (IFQMs) to image objects, an object-level strategy for quality assessment of the image fusion process is proposed. The proposed strategy is applied to different satellite imagery covering residential and agricultural areas. Experimental results show higher capabilities of object-level quality assessment strategy in the quality assessment of the fusion process. Evaluating fusion quality at the object level provides the potential of fusion quality assessment for each individual image object in compliance with different parameters such as the type of objects and the effective size of objects in data set.

ICAM framework for image appearance, differences, and quality

Traditional color appearance modeling has recently ma- tured to the point that available, internationally recommended mod- els such as CIECAM02 are capable of making a wide range of pre- dictions, to within the observer variability in color matching and color scaling of stimuli, in somewhat simplified viewing conditions. It is proposed that the next significant advances in the field of color ap- pearance modeling and image quality metrics will not come from evolutionary revisions of colorimetric color appearance models alone. Instead, a more revolutionary approach will be required to make appearance and difference predictions for more complex stimuli in a wider array of viewing conditions. Such an approach can be considered image appearance modeling, since it extends the concepts of color appearance modeling to stimuli and viewing envi- ronments that are spatially and temporally at the level of complexity of real natural and man-made scenes, and extends traditional image quality metrics into the color appearance domain. Thus, two previ- ously parallel and evolving research areas are combined in a new way as an attempt to instigate a significant advance. We review the concepts of image appearance modeling, present iCAM as one ex- ample of such a model, and provide a number of examples of the use of iCAM in image reproduction and image quality evaluation. © 2004 SPIE and IS&T. (DOI: 10.1117/1.1635368)

Developing operational performance metrics using image comparison metrics and the concept of degradation space

. A technique for determining relative degradations from image metrics is presented along with a technique for predicting sensor performance from metrics. These techniques are illustrated with degradations of blur and noise in thermal imagery. These uses of metrics are depicted as mappings among a degradation space, an image quality metric space, and an operational performance space. This technique has utility in sampled imagery applications where input and output image comparison is possible, e.g., validation of an infrared scene projector (IRSP), testing image compression algorithms, image simulation, etc. Such applications have a known input image and a degraded output image. With the input image, one can characterize the output image in terms of its degradations relative to the input. The concept of a degradation space leads to developing an Operational Performance Metric (OPM) in terms of more traditional Image Quality Metrics (IQMs). The technique is illustrated using empirical results for human observers performing recognition tasks with thermal imagery in a degradation space of blur and noise.