Image Quality Assessment Measures Research Articles

Bilateral tone mapping scheme for color correction and contrast adjustment in nearly invisible medical images

AbstractMedical images are commonly used to diagnose and screen various diseases. However, medical images taken by operators with different skill levels have a considerable variation in visual quality. Poor illumination of images increases uncertainty in clinical treatment and lead to the risk of misdiagnosis. This paper proposes a novel bilateral tone‐mapped gamma correction to enhance the visual quality of medical images without affecting the natural color quality. First, the proposed tone‐mapped gamma correction is employed to enhance the intensity values of poorly illuminated medical images. Then, bilateral enhancement curve adjustment is performed to effectively improve the global luminance content of the input image with features and richer information. The image quality assessment measures like entropy, blind/reference less image spatial quality evaluator, natural image quality evaluator, perception‐based image quality evaluator, contrast enhancement based image quality parameter, and color contrast fog density evaluates the superior performance of the proposed technique. The experimental results significantly enhance the intensity difference between dark and bright areas and can aid clinical experts and intelligent health fields.

Mere za estimaciju klatera na slici i performanse akvizicije cilja

Introduction/purpose: Measuring target acquisition performance in imaging systems with human-in-the-loop plays an essential role in military applications. This paper presents an extended review on the application of image clutter metrics for target acquisition, with the aim of using objective measures to predict the detection probability, false alarm probability and mean search time of the target in the image. Methods: To determine the degree of clutter, simple features on the global (picture-wise) and local (target-wise) level were used as well as contrast-based clutter metrics, target size and metrics derived from image quality assessment measures. Along with the standard ones, the features derived from the distribution of mean subtracted contrast normalized coefficients were also used. To compare the results of the objective scores and the experimental results obtained on the publicly available Search_2 dataset, regression laws accepted in the literature were applied. Linear correlations and rank correlations were used as quantitative measures of agreement. Results: It is shown that the best agreement with target acquisition indicators is obtained by applying clutter metrics derived from image quality assessment measures. The correlation with the results of subjective tests is up to 90%, which indicates the need for further research. A special contribution of the paper is the analysis of the target acquisition prediction performance using simple features at the global and local level, where it is shown that the prediction performance can be improved by determining the features around the target. Furthermore, it was shown that the false alarm probability and the probability of detection can be predicted based on the mean target search time in the image with a probability higher than 90%. Conclusion: In addition to obtaining a high degree of agreement between the objective metrics of clutter and the results of subjective tests (up to 90%), there is a need to improve the existing and develop new metrics as well as to conduct new subjective tests.

Study of subjective and objective quality assessment of infrared compressed images

Abstract Given the lack of accessible infrared compressed images’ benchmarks annotated by human subjects, this work presents a new database with the aim of studying both subjective and objective image quality assessment (IQA) on compressed long wavelength infrared (LWIR) images. The database contains 20 reference (pristine) images and 200 distorted (degraded) images obtained by application of the most known compression algorithms used in multimedia and communication fields, namely: JPEG and JPEG-2000. Each compressed image is evaluated by 31 subjects having different levels of experience in LWIR images. Mean opinion scores (MOS) and natural scene statistics (NSS) of pristine and compressed images are elaborated to study the performance of the database. Five analyses are conducted on collected images and subjective scores, namely: analysis by compression type, analysis by file size, analysis by reference image, analysis by quality level and analysis by subject. Moreover, a wide set of objective IQA metrics is applied on the images and the obtained scores are compared with the collected subjective scores. Results show that objective IQA measures correlate with human subjective results with a degree of agreement up to 95 %, so this benchmark is promising to improve existing and develop new IQA measures for compressed LWIR images. Thanks to a real-world surveillance original images based on which we analyze how image compression and quality level affect the quality of compressed images, this database is primarily suitable for (military and civilian) surveillance applications. The database is accessible via the link: https://github.com/azedomar/compressed-LWIR-images-IQA-database. As a follow-up to this work, an extension of the database is underway to study other types of distortion in addition to compression.

An efficient unsupervised image quality metric with application for condition recognition in kiln

In this paper, we propose an unsupervised textural-intensity-based natural image quality evaluator (TI-NIQE) by modelling the texture, structure and naturalness of an image. In detail, an effective quality-aware feature named as textural intensity (TI) is proposed in this paper to detect image texture. The image structure is captured by the distribution of gradients and basis images. The naturalness is characterized through the distributions of the locally mean subtracted and contrast normalized (MSCN) coefficients and the products of pairs of the adjacent MSCN coefficients. Furthermore, a new application pattern of image quality assessment (IQA) measures is proposed by taking the quality scores as the essential input of the recognition model. Using statistics of video quality scores computed by TI-NIQE as input features, an automatic IQA-based visual recognition model is proposed for the condition recognition in rotary kiln. Extensive experiments on benchmark datasets demonstrate that TI-NIQE shows better performance both in accuracy and computational complexity than other state-of-the-art unsupervised IQA methods, and experimental results on real-world data show that the recognition model has high prediction accuracy for condition recognition in rotary kiln.

Structural similarity index (SSIM) revisited: A data-driven approach

Several contemporaneous image processing and computer vision systems rely upon the full-reference image quality assessment (IQA) measures. The single-scale structural similarity index (SS-SSIM) is one of the most popular measures, and it owes its success to the mathematical simplicity, low computational complexity, and implicit incorporation of Human Visual System’s (HVS) characteristics. In this paper, we revise the original parameters of SSIM and its multi-scale counterpart (MS-SSIM) to increase their correlation with subjective evaluation. More specifically, we exploit the evolutionary computation and the swarm intelligence methods on five popular IQA databases, two of which are dedicated distance-changed databases, to determine the best combination of parameters efficiently. Simultaneously, we explore the effect of different scale selection approaches in the context of SS-SSIM. The experimental results show that with a proper fine-tuning (1) the performance of SS-SSIM and MS-SSIM can be improved, in average terms, by 8% and by 3%, respectively, (2) the SS-SSIM after the so-called standard scale selection achieves similar performance as if applying computationally more expensive state-of-the-art scale selection methods or MS-SSIM; moreover, (3) there is evidence that the parameters learned on a given database can be successfully transferred to other (previously unseen) databases; finally, (4) we propose a new set of reference parameters for SSIM’s variants and provide their interpretation.

Evaluation of quality measures for color quantization

Visual quality evaluation is one of the challenging basic problems in image processing. It also plays a central role in the shaping, implementation, optimization, and testing of many methods. The existing image quality assessment methods focused on images corrupted by common degradation types while little attention was paid to color quantization. This in spite there is a wide range of applications requiring color quantization assessment being used as a preprocessing step when color-based tasks are more efficiently accomplished on a reduced number of colors. In this paper, we propose and carry-out a quantitative performance evaluation of nine well-known and commonly used full-reference image quality assessment measures. The evaluation is done by using two publicly available and subjectively rated image quality databases for color quantization degradation and by considering suitable combinations or subparts of them. The results indicate the quality measures that have closer performances in terms of their correlation to the subjective human rating and show that the evaluation of the statistical performance of the quality measures for color quantization is significantly impacted by the selected image quality database while maintaining a similar trend on each database. The detected strong similarity both on individual databases and on databases obtained by integration provides the ability to validate the integration process and to consider the quantitative performance evaluation on each database as an indicator for performance on the other databases. The experimental results are useful to address the choice of suitable quality measures for color quantization and to improve their future employment.

An Improved SPSIM Index for Image Quality Assessment

Objective image quality assessment (IQA) measures are playing an increasingly important role in the evaluation of digital image quality. New IQA indices are expected to be strongly correlated with subjective observer evaluations expressed by Mean Opinion Score (MOS) or Difference Mean Opinion Score (DMOS). One such recently proposed index is the SuperPixel-based SIMilarity (SPSIM) index, which uses superpixel patches instead of a rectangular pixel grid. The authors of this paper have proposed three modifications to the SPSIM index. For this purpose, the color space used by SPSIM was changed and the way SPSIM determines similarity maps was modified using methods derived from an algorithm for computing the Mean Deviation Similarity Index (MDSI). The third modification was a combination of the first two. These three new quality indices were used in the assessment process. The experimental results obtained for many color images from five image databases demonstrated the advantages of the proposed SPSIM modifications.

Structured Computational Modeling of Human Visual System for No-reference Image Quality Assessment

Objective image quality assessment (IQA) plays an important role in various visual communication systems, which can automatically and efficiently predict the perceived quality of images. The human eye is the ultimate evaluator for visual experience, thus the modeling of human visual system (HVS) is a core issue for objective IQA and visual experience optimization. The traditional model based on black box fitting has low interpretability and it is difficult to guide the experience optimization effectively, while the model based on physiological simulation is hard to integrate into practical visual communication services due to its high computational complexity. For bridging the gap between signal distortion and visual experience, in this paper, we propose a novel perceptual no-reference (NR) IQA algorithm based on structural computational modeling of HVS. According to the mechanism of the human brain, we divide the visual signal processing into a low-level visual layer, a middle-level visual layer and a high-level visual layer, which conduct pixel information processing, primitive information processing and global image information processing, respectively. The natural scene statistics (NSS) based features, deep features and free-energy based features are extracted from these three layers. The support vector regression (SVR) is employed to aggregate features to the final quality prediction. Extensive experimental comparisons on three widely used benchmark IQA databases (LIVE, CSIQ and TID2013) demonstrate that our proposed metric is highly competitive with or outperforms the state-of-the-art NR IQA measures.

Orthogonal projections for image quality analyses applied to MRI

AbstractMeasuring digital image quality is of importance in numerous applied research fields, such as image acquisition, image processing or image reconstruction. Subjective evaluation, i.e. assessment by human beings, is often too time‐consuming, expensive and inconsistent in complicated tasks such as medical imaging. In order to better understand the relation between objective image quality assessment measures, we compare a selection in an experimental setup. It is well known that L2 error estimation, i.e. computing the mean squared error (MSE) between two images, does not correspond well to the perceptual evaluation by the human visual system. Here, we illustrate on the example of MRI data the interaction of the MSE with the signal to noise ratio (SNR), the variance and subjective evaluation. We use random coverings of the space of orthogonal projections for coil combination to enable a comprehensive correlation analysis.

No-reference image quality assessment of authentically distorted images with global and local statistics

The development of digital image processing techniques requires reliable image quality assessment (IQA) methods. Since images acquired by a camera often contain various distortions and their non-distorted versions are not available, a no-reference IQA (NR-IQA) technique should be used. Many popular methods are developed to assess artificially distorted images, available in benchmark databases. In this paper, a new large benchmark database, containing naturally distorted images captured with a digital camera, is introduced along with a new NR-IQA metric. The method uses a wide spectrum of local and global image features and their statistics to address a diversity of distortions. Among 80 employed features, 56 are introduced to the IQA for the first time, while the remaining statistics are used to further improve the quality prediction performance of the method. The obtained perceptual feature vector is used to provide a quality model with support vector regression technique. The experimental comparison of the method with the state-of-the-art IQA measures on the database reveals its superiority in terms of correlation with human scores.

Perceptual image quality assessment: a survey

Perceptual quality assessmentplays a vital role in the visual communication systems owing to theexistence of quality degradations introduced in various stages of visual signalacquisition, compression, transmission and display.Quality assessment for visual signals can be performed subjectively andobjectively, and objective quality assessment is usually preferred owing to itshigh efficiency and easy deployment. A large number of subjective andobjective visual quality assessment studies have been conducted during recent years.In this survey, we give an up-to-date and comprehensivereview of these studies.Specifically, the frequently used subjective image quality assessment databases are firstreviewed, as they serve as the validation set for the objective measures.Second, the objective image quality assessment measures are classified and reviewed according to the applications and the methodologies utilized in the quality measures.Third, the performances of the state-of-the-artquality measures for visual signals are compared with an introduction of theevaluation protocols.This survey provides a general overview of classical algorithms andrecent progresses in the field of perceptual image quality assessment.

Texture synthesis quality assessment using perceptual texture similarity

Texture synthesis plays an important role in computer game and movie industries. Although it has been widely studied, the assessment of the quality of the synthesised textures has received little attention. Inspired by the research progress in perceptual texture similarity estimation, we propose a Texture Synthesis Quality Assessment (TSQA) approach. To our knowledge, this is the first attempt to exploit perceptual texture similarity for the TSQA task. In particular, we introduce two perceptual similarity principles for synthesis quality assessment. Correspondingly, we train two Random Forest (RF) regressors. Given a pair of sample and synthesised textures, the two regressors can be used to predict the global and local quality scores of the synthesised texture respectively. An overall score is generated from the two scores. Our results show that the deep Bag-of-Words (BoW) descriptors, extracted by a pre-trained Convolutional Neural Network (CNN), perform better than, or comparably to, the other nine types of hand-crafted or CNN descriptors and an image quality assessment measure, together with the proposed TSQA approach.

Adaptive Near-Infrared and Visible Fusion for Fast Image Enhancement

Near-infrared (NIR) band sensors capture digital images of scenes under special conditions such as haze, fog, overwhelming light or mist, where visible (VS) band sensors get occluded. However, the NIR images contain poor textures and colors of different objects in the scene, on the contrary to the VS images. In this article, we propose a simple yet effective fusion approach that combines both VS and NIR images to produce an enhanced fused image that contains better scene details and similar colors to the VS image. The proposed approach first estimates a fusion map from the relative difference of local contrasts of the VS and NIR images. Then, the approach extracts non-spectral spatial details from the NIR image and finally, the extracted details are weighted according to the fusion map and injected into the VS image to produce the enhanced fused image. The proposed approach adaptively transfers the useful details from the NIR image that contributes to the enhancement of the fused image. It produces realistic fused images by preserving the colors of the VS image and constitutes simple and non-iterative calculations with $\mathcal {O}(n)$ complexity. The effectiveness of the proposed approach is experimentally verified by comparisons to four different state-of-the-art VS-NIR fusion approaches in terms of computational complexity and quality of the obtained enhanced fused images. The quality is evaluated using two-color distortion measures and a novel aggregation of several blind image quality assessment measures. The proposed approach shows superior performance as it produces enhanced fused images and preserves the quality even when the NIR images suffer from loss of texture or blurriness degradations, with acceptable fast execution time. Source code of the proposed approach is available online.

Quality-Aware Unpaired Image-to-Image Translation

Generative Adversarial Networks (GANs) have been widely used for the image-to-image translation task. While these models rely heavily on the labeled image pairs, recently some GAN variants have been proposed to tackle the unpaired image translation task. These models exploited supervision at the domain level with a reconstruction process for unpaired image translation. On the other hand, parallel works have shown that leveraging perceptual loss functions based on high level deep features could enhance the generated image quality. Nevertheless, as these GAN-based models either depended on the pretrained deep network structure or relied on the labeled image pairs, they could not be directly applied to the unpaired image translation task. Moreover, despite the improvement of the introduced perceptual losses from deep neural networks, few researchers have explored the possibility of improving the generated image quality from classical image quality measures. To tackle the above two challenges, in this paper, we propose a unified quality-aware GAN-based framework for unpaired image-to-image translation, where a quality-aware loss is explicitly incorporated by comparing each source image and the reconstructed image at the domain level. Specifically, we design two detailed implementations of the quality loss. The first method is based on a classical image quality assessment measure by defining a classical quality-aware loss. The second method proposes an adaptive deep network based loss. Finally, extensive experimental results on many real-world datasets clearly show the quality improvement of our proposed framework, and the superiority of leveraging classical image quality measures for unpaired image translation compared to the deep network based model.

Quality Evaluation of Image Dehazing Methods Using Synthetic Hazy Images

To enhance the visibility and usability of images captured in hazy conditions, many image dehazing algorithms (DHAs) have been proposed. With so many image DHAs, there is a need to evaluate and compare these DHAs. Due to the lack of the reference haze-free images, DHAs are generally evaluated qualitatively using real hazy images. But it is possible to perform quantitative evaluation using synthetic hazy images since the reference haze-free images are available and full-reference (FR) image quality assessment (IQA) measures can be utilized. In this paper, we follow this strategy and study DHA evaluation using synthetic hazy images systematically. We first build a synthetic haze removing quality (SHRQ) database. It consists of two subsets: regular and aerial image subsets, which include 360 and 240 dehazed images created from 45 and 30 synthetic hazy images using 8 DHAs, respectively. Since aerial imaging is an important application area of dehazing, we create an aerial image subset specifically. We then carry out subjective quality evaluation study on these two subsets. We observe that taking DHA evaluation as an exact FR IQA process is questionable, and the state-of-the-art FR IQA measures are not effective for DHA evaluation. Thus, we propose a DHA quality evaluation method by integrating some dehazing-relevant features, including image structure recovering, color rendition, and over-enhancement of low-contrast areas. The proposed method works for both types of images, but we further improve it for aerial images by incorporating its specific characteristics. Experimental results on two subsets of the SHRQ database validate the effectiveness of the proposed measures.

No-Reference quality assessment of noisy images with local features and visual saliency models

Image quality assessment (IQA) measures predict the perceived quality of evaluated images, aiming to replace time-consuming human evaluation. This is particularly important for the automatic comparison of image processing techniques which often modify image content. Since the presence of noise highly affects the perception of images and only a few IQA measures address this type of distortion, a new no-reference IQA measure is proposed in this paper. In the introduced measure, two aspects of the Human Visual System (HVS) are considered. The sensitivity of the HVS to local image distortions is expressed by similarities between descriptors obtained with Speeded-Up Robust Features (SURF) technique. In order to improve the quality prediction, image patches for description are extracted from a gradient map of an input image. Noise also affects the visual saliency, since it decreases attention caused by image content. In the approach, visual saliency models of image patches are obtained and evaluated using adapted Visual Saliency-based Index (VSI). Experimental evaluation on popular IQA benchmarks reveals that the proposed measure outperforms the related state-of-the-art techniques. The applicability of the proposed method to an automatic selection of the best denoising algorithm is also discussed in the paper.

A New Objective Image Quality Assessment Metric: For Color and Grayscale Images

Image quality assessment (IQA) is a challenging task in digital image processing where the images get distorted in several situations. In recent years, various IQA measures have been developed to assess the quality of images in subjective and objective manner. Although the popular measures like mean square error (MSE) and peak signal to noise ratio (PSNR) work well for grayscale images, they fail to measure the exact difference between two color images on a pixel by pixel basis. To overcome this problem, we made a slight modification in the MSE and PSNR to present new measures mean absolute variance and fidelity ration (FR). Moreover, we formulate a constant ‘k’ of value 9.542425094 to establish a relationship between FR of color and grayscale images. The performance of the proposed metric is validated by comparing its results with state of art metrics against a same set of benchmark dataset. Though the proposed method involves simple mathematical calculations and no human visual system model is employed, the experimental analysis shows that FR is found to be highly effective and robust measure especially for continuous tone, discrete tone, bi-level and inverted images. This measure is highly useful for applications require exact IQA where a change of one-pixel value is also not desirable.

Full Reference Objective Quality Assessment for Reconstructed Background Images

With an increased interest in applications that require a clean background image, such as video surveillance, object tracking, street view imaging and location-based services on web-based maps, multiple algorithms have been developed to reconstruct a background image from cluttered scenes. Traditionally, statistical measures and existing image quality techniques have been applied for evaluating the quality of the reconstructed background images. Though these quality assessment methods have been widely used in the past, their performance in evaluating the perceived quality of the reconstructed background image has not been verified. In this work, we discuss the shortcomings in existing metrics and propose a full reference Reconstructed Background image Quality Index (RBQI) that combines color and structural information at multiple scales using a probability summation model to predict the perceived quality in the reconstructed background image given a reference image. To compare the performance of the proposed quality index with existing image quality assessment measures, we construct two different datasets consisting of reconstructed background images and corresponding subjective scores. The quality assessment measures are evaluated by correlating their objective scores with human subjective ratings. The correlation results show that the proposed RBQI outperforms all the existing approaches. Additionally, the constructed datasets and the corresponding subjective scores provide a benchmark to evaluate the performance of future metrics that are developed to evaluate the perceived quality of reconstructed background images.

COMPRESSION OF HIGH DYNAMIC RANGE OF SAR IMAGES

The paper presents results of our experiments on compression of the high dynamic range SAR images. The range is equal to 16-bit. Objectives of study were comparison of known approaches to compression of the high dynamic range images; selection of optimal parameters for compression algorithms, and selection of a no-reference measure for image quality assessment after compression. Tone-mapping transformations like gamma correction, Ashikhmin-operator, mu-transformation, as well as no-reference image quality assessment measures were tested. The results of the experiments are presented in the article. It was concluded that further research and analysis of various functions and approaches to compression of dynamic range of SAR images is necessary, since including in the article approaches do not give stable and positive results on all SAR images. It was also concluded that after transformation 16-bit images, it is very difficult to estimate which image is better, and it is necessary to use no-reference image quality assessment measure.

Blind Image Quality Estimation via Distortion Aggravation

Traditional blind image quality assessment (IQA) measures generally predict quality from a sole distorted image directly. In this paper, we first introduce multiple pseudo reference images (MPRIs) by further degrading the distorted image in several ways and to certain degrees, and then compare the similarities between the distorted image and the MPRIs. Via such distortion aggravation, we can have some references to compare with, i.e., the MPRIs, and utilize the full-reference IQA framework to compute the quality. Specifically, we apply four types and five levels of distortion aggravation to deal with the commonly encountered distortions. Local binary pattern features are extracted to describe the similarities between the distorted image and the MPRIs. The similarity scores are then utilized to estimate the overall quality. More similar to a specific pseudo reference image (PRI) indicates closer quality to this PRI. Owning to the availability of the created multiple PRIs, we can reduce the influence of image content, and infer the image quality more accurately and consistently. Validation is conducted on four mainstream natural scene image and screen content image quality assessment databases, and the proposed method is comparable to or outperforms the state-of-the-art blind IQA measures. The MATLAB source code of the proposed measure will be publicly available.