Full-reference Image Quality Metrics Research Articles

Impact of using sinogram domain data in the super-resolution of CT images on diagnostic information.

In recent times, deep-learning-based super-resolution (DL-SR) techniques for computed tomography (CT) images have shown outstanding results in terms of full-reference image quality (FR-IQ) metrics (e.g., root mean square error and structural similarity index metric), which assesses IQ by measuring its similarity to the high-resolution (HR) image. In addition, IQ can be evaluated via task-based IQ (Task-IQ) metrics that evaluate the ability to perform specific tasks. Ironically, most proposed image domain-based SR techniques are not possible to improve a Task-IQ metric, which assesses the amount of information related todiagnosis. In the case of CT imaging systems, sinogram domain data can be utilized for SR techniques. Therefore, this study aims to investigate the impact of utilizing sinogram domain data on diagnostic information restorationability. We evaluated three DL-SR techniques: using image domain data (Image-SR), using sinogram domain data (Sinogram-SR), and using sinogram as well as image domain data (Dual-SR). For Task-IQ evaluation, the Rayleigh discrimination task was used to evaluate diagnostic ability by focusing on the resolving power aspect, and an ideal observer (IO) can be used to perform the task. In this study, we used a convolutional neural network (CNN)-based IO that approximates the IO performance. We compared the IO performances of the SR techniques according to the data domain to evaluate the discriminative information restorationability. Overall, the low-resolution (LR) and SR exhibit lower IO performances compared with that of HR owing to their degraded discriminative information when detector binning is used. Next, between the SR techniques, Image-SR does not show superior IO performances compared to the LR image, but Sinogram-SR and Dual-SR show superior IO performances than the LR image. Furthermore, in Sinogram-SR, we confirm that FR-IQ and IO performance are positively correlated. These observations demonstrate that sinogram domain upsampling improves the representation ability for discriminative information in the image domain compared to the LR andImage-SR. Unlike Image-SR, Sinogram-SR can improve the amount of discriminative information present in the image domain. This demonstrates that to improve the amount of discriminative information on the resolving power aspect, it is necessary to employ sinogram domainprocessing.

Examining effect of super-resolution on AVIRIS-NG data: A precursor to generation of large-scale urban material and natural cover maps

The two-phase Airborne Visible InfraRed Imaging Spectrometer – Next Generation (AVIRIS-NG) campaign over India has generated datasets of numerous natural and artificial surroundings with high signal-to-noise ratios and higher spatial and spectral consistencies. However, the first phase AVIRIS-NG datasets of urban areas have 8.1 m spatial resolution, which is insufficient for producing level-4 urban material and natural cover maps required to model Earth’s processes at the micro-scale or constantly monitor urban processes. The current work addresses this drawback by demonstrating a part of the proposed proof-of-concept approach on the AVIRIS-NG dataset of Ahmedabad, India, wherein super-resolution (SR) is proposed to enhance the hyperspectral image’s spatial resolution and preserve its rich spectral content at the targeted spatial scale before utilizing it for producing detailed urban land cover maps. A step-wise account of the SR process has been provided herein to aid the reader in reproducing these experiments. Super-resolved products generated using four SR algorithms have been compared and validated using a composite framework based on processing time, qualitative visual investigation of selected image patches and spectral profiles, re-interpretation of Wald’s reduced resolution protocol and full-reference image quality metrics. Results suggest the AVIRIS-NG dataset’s successful spectral and spatial fidelity at a larger spatial scale, albeit varying in each super-resolved output. Sparse regression and natural prior (SRP), and anchored neighbourhood regression (ANR) produce the best and poorest super-resolved outputs among the tested SR algorithms. Although gradient profile prior (GPP) generated super-resolved product is the second best, it is also the most computationally expensive algorithm. An additional yardstick in the form of a spectral similarity analysis of randomly selected unknown spectral signatures on super-resolved output against a reference spectral library has been proposed to ascertain spectral content preservation further before utilizing the super-resolved product for urban information extraction. This exercise’s findings reveal the presence of the target urban land cover feature at the unknown pixels with matching scores of greater than 0.5 in both best and worst super-resolved products, thereby advocating the extension of the proposed proof-of-concept to recently launched spaceborne hyperspectral missions’ datasets.

BLUR KERNEL’S EFFECT ON PERFORMANCE OF SINGLE-FRAME SUPER-RESOLUTION ALGORITHMS FOR SPATIALLY ENHANCING HYPERION AND PRISMA DATA

Abstract. Single-frame super-resolution (SFSR) achieves the goal of generating a high-resolution image from a single low-resolution input in a three-step process, namely, noise removal, up-sampling and deblurring. Scale factor and blur kernel are essential parameters of the up-sampling and deblurring steps. Few studies document the impact of these parameters on the performance of SFSR algorithms for improving the spatial resolution of real-world remotely-sensed datasets. Here, the effect of changing blur kernel has been studied on the behaviour of two classic SFSR algorithms: iterative back projection (IBP) and gaussian process regression (GPR), which are applied to two spaceborne hyperspectral datasets for scale factors 2, 3 and 4. Eight full-reference image quality metrics and algorithm processing time are deployed for this purpose. A literature-based re-interpretation of Wald’s reduced resolution protocol has also been used in this work for choosing the reference image. Intensive intra-algorithm comparisons of various simulation scenarios reveal each algorithm’s best performing Gaussian blur kernel parameters. Inter-algorithm comparison shows the better performing algorithm out of the two, thereby paving the way for further research in SFSR of remotely-sensed images.

Automated image quality appraisal through partial least squares discriminant analysis.

Automatic retinal fundus image quality analysis is one of the most essential preliminary stages in automatic computer-aided retinal disease diagnosis system, which allows good-quality fundus images for accurate disease prediction through localization and segmentation of retinal regions. This paper presents new feature extraction methods using full-reference and no-reference image quality metrics for image quality classification. Basic image features, reference and no-reference features are extracted from the fundus image and applied through different classification techniques to determine the image quality for further diagnosis. In this paper, human-made categorization including good and non-good-quality fundus image classification is constructed by considering major features of retinal fundus images are illumination, clarity, image intensity, contrast and region visibility. The proposed system presented fundus image quality classification by automatic extraction of features from fundus images through image processing techniques and automatic classification of image quality through different classification algorithm. This system was thoroughly investigated on 2674 retinal fundus images from publically available datasets, namely MESSIDOR, Drishti-GS1, DRIVE, HRF, DRIONS-DB, DIARETDB0, DIARETDB1, IDRiD, INSPIRE-AVR, CHASE-DB1, ONHSD, DRIMDB and e-ophtha-EX with better performance results in terms of sensitivity, accuracy, precision and F1 score of 99.36%, 96.79%, 96.29% and 97.79%, respectively. The proposed system results were compared to the existing state-of-the-art approaches and outperform existing methods for image quality assessment representing the efficiency and robustness of our system is most suitable for automatic image analysis during retinal disease diagnosis.

Combined Full-Reference Image Quality Metrics for Objective Assessment of Multiply Distorted Images

In the recent years, many objective image quality assessment methods have been proposed by different researchers, leading to a significant increase in their correlation with subjective quality evaluations. Although many recently proposed image quality assessment methods, particularly full-reference metrics, are in some cases highly correlated with the perception of individual distortions, there is still a need for their verification and adjustment for the case when images are affected by multiple distortions. Since one of the possible approaches is the application of combined metrics, their analysis and optimization are discussed in this paper. Two approaches to metrics’ combination have been analyzed that are based on the weighted product and the proposed weighted sum with additional exponential weights. The validation of the proposed approach, carried out using four currently available image datasets, containing multiply distorted images together with the gathered subjective quality scores, indicates a meaningful increase of correlations of the optimized combined metrics with subjective opinions for all datasets.

RAVA: Region-Based Average Video Quality Assessment.

Video has become the most popular medium of communication over the past decade, with nearly 90 percent of the bandwidth on the Internet being used for video transmission. Thus, evaluating the quality of an acquired or compressed video has become increasingly important. The goal of video quality assessment (VQA) is to measure the quality of a video clip as perceived by a human observer. Since manually rating every video clip to evaluate quality is infeasible, researchers have attempted to develop various quantitative metrics that estimate the perceptual quality of video. In this paper, we propose a new region-based average video quality assessment (RAVA) technique extending image quality assessment (IQA) metrics. In our experiments, we extend two full-reference (FR) image quality metrics to measure the feasibility of the proposed RAVA technique. Results on three different datasets show that our RAVA method is practical in predicting objective video scores.

Quality Assessment of 2.5D Prints Using 2D Image Quality Metrics

Quality assessment is an important aspect in a variety of application areas. In this work, the objective quality assessment of 2.5D prints was performed. The work is done on camera captures under both diffuse (single-shot) and directional (multiple-shot) illumination. Current state-of-the-art 2D full-reference image quality metrics were used to predict the quality of 2.5D prints. The results showed that the selected metrics can detect differences between the prints as well as between a print and its 2D reference image. Moreover, the metrics better detected differences in the multiple-shot set-up captures than in the single-shot set-up ones. Although the results are based on a limited number of images, they show existing metrics’ ability to work with 2.5D prints under limited conditions.

A New Full-Reference Image Quality Metric for Motion Blur Profile Characterization

A New Full-Reference Image Quality Metric for Motion Blur Profile Characterization

Single Image Dehazing by Predicting Atmospheric Scattering Parameters

In this work we propose a method for single image dehazing that exploits a physical model to recover the haze-free image by estimating the atmospheric scattering parameters. Cycle consistency is used to further improve the reconstruction quality of local structures and objects in the scene as well. Experimental results on four real and synthetic hazy image datasets show the effectiveness of the proposed method in terms of two commonly used full-reference image quality metrics.

An expandable image database for evaluation of full-reference image visual quality metrics

Traditional approach to collect mean opinion score (MOS) values for evaluation of full-reference image quality metrics has two serious drawbacks. The first drawback is a nonlinearity of MOS, only partially compensated by the use of rank order correlation coefficients in a further analysis. The second drawback are limitations on number of distortion types and distortion levels in image database imposed by a maximum allowed time to carry out an experiment. One of the largest of databases used for this purpose, TID2013, has almost reached these limitations, which makes an extension of TID2013 within the boundaries of this approach to be practically unfeasible. In this paper, a novel methodology to collect MOS values, with a possibility to infinitely increase a size of a database by adding new types of distortions, is proposed. For the proposed methodology, MOS values are collected for pairs of distortions, one of them being a signal dependent Gaussian noise. A technique of effective linearization and normalization of MOS is described. Extensive experiments for linearization of MOS values to extend TID2013 database are carried out.

Full-reference image quality metric for blurry images and compressed images using hybrid dictionary learning

The image quality degradation due to the loss of high-frequency components of images is often seen in real scenarios, such as artifacts caused by image compression and image blur caused by camera shake or out of focus. Quantifying such degradation is very useful for many tasks that are related to image quality. In this paper, an effective approach is proposed for the image quality assessment on images with blur as well as images with compression artifacts. Based on the relation between the dictionaries of the degraded image and the reference image, we build up a hybrid dictionary learning model to characterize the space of patches of the reference image as well as that of the degraded image. The image quality is then measured by the difference between the two resulting dictionaries. Combined with a simple sparse-coding-based metric, the proposed method shows competitive performance on five benchmark datasets, which demonstrates its effectiveness.

Effective quality metric for contrast-distorted images based on SVD

Proper contrast usually makes an object distinguishable, while contrast degradation will affect our understanding of the object. Many experiments found that the human visual system is more sensitive to contrast than to luminance in distinguishing an object. However, few studies have been conducted to assess the quality of contrast change effectively. In our study, we first reveal some characteristics of contrast-distorted images by analyzing image data. Based on these characteristics, we then present two novel and fast full-reference image quality metrics: FC-SVD for a full-reference model and RC-SVD for a reduced-reference image quality assessment model, both using singular value decomposition (SVD). The proposed methods are evaluated through four public contrast-related databases (TID2013, TID2008, CCID2014, and CSIQ). The results show that they have excellent performance in prediction accuracy and much lower computation complexity than other state-of-the-art image quality assessment methods.

Low-light image enhancement of high-speed endoscopic videos using a convolutional neural network.

Laryngeal endoscopy is one of the primary diagnostic tools for laryngeal disorders. The main techniques are videostroboscopy and lately high-speed video endoscopy. Unfortunately, due to the restricting anatomy of the larynx and technical limitations of the recording equipment, many videos suffer from insufficient illumination, which complicates clinical examination and analysis. This work presents an approach to enhance low-light images from high-speed video endoscopy using a convolutional neural network. We introduce a new technique to generate realistically darkened training samples using Perlin noise. Extensive data augmentation is employed to cope with the limited training data allowing training with just 55 videos. The approach is compared against four state-of-the-art low-light enhancement methods and statistically significantly outperforms each on a no-reference (NIQE) and two full-reference (PSNR, SSIM) image quality metrics. The presented approach can be run on consumer-grade hardware and is thereby directly applicable in a clinical context. It is likely transferable to similar techniques such as videostroboscopy. Graphical Abstract The basic setup for training and employing an improved fully convolutional U-Net neural network to predict a brightness map used to enhance the lighting of ill-lit endoscopic high-speed videos - Artificially darkened training data are created using Perlin noise to allow region-specific darkening.

I3D: a new dataset for testing denoising and demosaicing algorithms

In this paper we present a dataset of images to test the performance of image processing algorithms, in particular demosaicing and denoising methods. Despite the plethora of demosaicing and denoising algorithms present in the literature, only few benchmarks are available to test their performance, and most of them are quite old, thus inadequate to represent the images captured by modern devices. The proposed dataset is composed by twenty 16 bit-depth images that can be used to test full-reference image quality metrics. More specifically, twelve pictures have been synthetically created by means of 2D or 3D softwares, while eight images have been captured by a high-end digital camera.

Study of Visual Quality Assessment on Pattern Images: Subjective Evaluation and Visual Saliency Effects

This paper presents a comparison study on visual quality scores obtained from single-stimulus and double-stimulus approaches for pattern images, respectively. We also conduct the comparison on general (non-pattern) images, which serve as a control group. The pattern and non-pattern (PNP) images are collected and built by the Perceptual Data Analysis and Processing Laboratory, National Chung Hsing University, and evaluated by a group of people with both single-stimulus and double-stimulus approaches. Then, we examine the difference of mean opinion scores obtained by both the methods with respect to the image types, image contents (scenes), and distortion types. The hypothesis tests are used to determine whether there is a significant difference between both the sets of scores. The test results suggest that the differences exist and are significant for some specific distortion types and image contents. For example, the pattern images with simple colors should be viewed by the double-stimulus method in order to reduce the variance of subjective scores. In the end, we test three well-known full-reference image quality metrics (IQMs) on both the PNP images. In addition, we investigate whether the visual saliency (VS) plays a more important role for pattern images compared with non-pattern images. We discover that the most salient regions are more influential on pattern images than non-pattern images. Also, by introducing the VS information into the IQMs with a newly proposed Otsu’s weighted VS mask, the correlation performance between objective quality scores obtained from IQMs and subjective scores from human’s visual perception can be boosted further.

An extensive performance evaluation of full-reference HDR image quality metrics

High dynamic range (HDR) image and video technology has recently attracted a great deal of attention in the multimedia community, as a mean to produce truly realistic video and further improve the quality of experience (QoE) of emerging multimedia services. In this context, measuring the quality of compressed HDR content plays a fundamental role. However, full-reference (FR) HDR visual quality assessment poses new challenges with respect to the conventional low dynamic range case. Quality metrics have to be redesigned or adapted to HDR, and understanding their reliability to predict users’ judgments is even more critical due to the still limited availability of HDR displays to perform subjective evaluations. The goal of this paper is to provide a complete and thorough survey of the performance of the most popular HDR FR image quality metrics. To this end, we gather several existing HDR image databases with subjective quality annotations, in addition to a new one created by ourselves. After aligning the scores in these databases, we obtain an extensive set of 690 compressed HDR images, along with their subjective quality. Next, we analyze in depth many FR metrics, including those used in MPEG standardization, using both classical correlation analyses and classification accuracy. We believe that our results could serve as the most complete and comprehensive benchmark of image quality metrics in the field of HDR image compression.

A weighted full-reference image quality assessment based on visual saliency

In full reference image quality assessment (IQA), the images without distortion are usually employed as reference, while the structures in both reference images and distorted images are ignored and all pixels are equally treated. In addition, the role of human visual system (HVS) is not taken account into subjective IQA metric. In this paper, a weighted full-reference image quality metric is proposed, where a weight imposed on each pixel indicates its importance in IQA. Furthermore, the weights can be estimated via visual saliency computation, which can approximate the subjective IQA via exploiting the HVS. In the experiments, the proposed metric is compared with several objective IQA metrics on LIVE release 2 and TID 2008 database. The results demonstrate that SROCC and PLCC of the proposed metric are 0.9647 and 0.9721, respectively,which are higher than other methods and it only takes 427.5s, which is lower than that of most other methods.

Reduced Reference Quality Metric for Synthesized Virtual Views in 3DTV

Multi-view video plus depth (MVD) has been widely used owing to its effectiveness in three-dimensional data representation. Using MVD, color videos with only a limited number of real viewpoints are compressed and transmitted along with captured or estimated depth videos. Because the synthesized views are generated from decoded real views, their original reference views do not exist at either the transmitter or receiver. Therefore, it is challenging to define an efficient metric to evaluate the quality of synthesized images. We propose a novel metric—the reduced-reference quality metric. First, the effects of depth distortion on the quality of synthesized images are analyzed. We then employ the high correlation between the local depth distortions and local color characteristics of the decoded depth and color images, respectively, to achieve an efficient depth quality metric for each real view. Finally, the objective quality metric of the synthesized views is obtained by combining all the depth quality metrics obtained from the decoded real views. The experimental results show that the proposed quality metric correlates very well with full reference image and video quality metrics.

A new full-reference image quality metric based on just noticeable difference

An Image Quality Metric (IQM) is used to measure how the image on which modifications are carried out is distorted. Knowing the distortion scale of the image is of significance for the intended purposes (compression, forensic, medical, etc.). The amount of distortion has different implications for each of the application and whether the image is convenient for the relevant application by using the IQM. One of the most basic approaches in the literature which is used in measuring the amount of the distortion is the Full-Reference Image Quality Metric (FR-IQM). The FR-IQM needs both the original and distorted image in the phase of comparison. In this study, taking into consideration the Human Vision System (HVS), a new full-reference image quality metric based on Just Noticeable Difference (JND) has been presented. Through this new metric, the HVS's physiological (color and light sensitivity) and psycho-physiological (texture and edge sensitivity) characteristics have been taken into consideration. Through the JND, the inclusion of the unintended modifications that cannot be sensed by the HVS has been prevented. The near-zero quality results point out the more quality image. The experimental results have showed that the metric in question demonstrates a higher level of performance for the HVS when compared with its counterparts.

An oriented gradient based image quality metric for pedestrian detection performance evaluation

Full reference image quality metrics are important tools for optimizing system design parameters associated with image acquisition, compression and transmission. While optimizing systems for perceptual quality is important, in the automotive environment Advanced Driver Assistance Systems (ADAS) such as automated pedestrian detection are becoming a common feature of in-vehicular vision systems. As such, automotive image quality must also be tuned for optimal machine vision performance.In this paper the effects of transmission artifacts on the performance of a number of state-of-the-art pedestrian detection algorithms are evaluated. We demonstrate that the human visual system may not perceive distortions that adversely affect machine vision performance. As a result, existing full-reference image quality metrics are not necessarily accurate predictors of machine vision performance on transmitted video sequences. To address this problem, a novel, computationally inexpensive, full-reference objective quality metric based on histogram of oriented gradients is proposed. The proposed metric accurately predicts algorithm performance in the presence of transmission artifacts. The metric can be used at the system design stage in order to optimize image capture parameters for machine vision performance without the need for annotated test databases, which are both expensive and time consuming to produce.