Multi-scale Structural Similarity Research Articles

Perceptual image quality assessment based on structural similarity and visual masking

We propose an improved objective image quality assessment method based on the structural similarity and visual masking, called the Perceptual Image Quality Assessment (PIQA). The PIQA contains three similarity measures: the luminance comparison measure, the structure comparison measure, the contrast comparison measure as same as the Structure Similarity (SSIM) and its variants. Firstly, in order to improve the ability of distinguishing the structure information in blurred images and noisy images, we modify the structure comparison measure by using the improved structure tensor which is more efficient for describing the structure information in global areas. Secondly, based on the perceptual characters of Human Visual System (HVS) perceptual process, the contrast masking and neighborhood masking are integrated to the contrast comparison measure. Finally, three measures are pooled together to compute the PIQA metric. Comparing with the state-of-the-art methods including Multi-scale SSIM (MS-SSIM), Visual Signal to Noise Ratio (VSNR) and Visual Information Fidelity (VIF) criterion, simulation results show that our approach is highly consistent with HVS perceptual process, and also delivers better performance.

Structural Similarity Measure for Color Images

images reveal more meaningful information to the human observers rather than grayscale ones. Regardless of the advantages of the existing well-known objective image quality measures, one of the common and major limitations of these measures is that they evaluate the quality of grayscale images only and don't make use of color information. In this paper we propose an improved method for image quality assessment that adds a color comparison to the criteria of the well-known Multiscale Structural Similarity index (MSSIM). We evaluated the new color image quality measure through human subjective experiments. Our human subjective evaluation data contains 25 reference images and 875 test images produced by five popular color quantization algorithms. Each of the quantized images was evaluated by twenty two subjects and more than 19200 individual human quality judgments were carried out to obtain the final mean opinion scores. We also tested the proposed method on TID2008 image database to further verify our results. These results indicate that adding color comparison improves MSSIM for many distortions in TID2008 and for assessing quantized images in our database. Keywordsquality assessment, Structural similarity index, Color quantization.

A new image quality measure for assessment of histogram equalization-based contrast enhancement techniques

Absolute Mean Brightness Error (AMBE) and Entropy are among the two most popular IQMs used to assess Histogram Equalization (HE) based techniques. To the best of [email protected]?s knowledge, there is no evaluation report on how well the two IQMs correlate to human opinion. This paper reviews and discusses the potential flaws in using AMBE and Entropy to assess HE-based techniques. This paper presents results of a subjective quality assessment in which image quality data obtained from 1935 human observer opinion scores were used to evaluate the IQMs. The statistical evaluation results show that the two IQMs have poor correlation with human mean opinion score (MOS); Pearson Correlation Coefficient (PCC) 0.75, Outlier Ratio (OR)>20%. A new IQM which takes into account important properties of human visual perception (HVP) is proposed. It is tested and found to have significantly better correlation (PCC>0.86, RMSE<0.39 and OR=0%). The proposed IQM also outperforms Multi-Scale Structural Similarity (MSSIM) and Information Fidelity Criterion-based (IFC) measure, which are two prominent fidelity-based IQMs.

Weighted Multi-scale Structural Similarity for Image Quality Assessment with Saliency-based Pooling Strategy

Weighted Multi-scale Structural Similarity for Image Quality Assessment with Saliency-based Pooling Strategy

Directional lifting wavelet and universal trellis-coded quantisation-based image coding algorithm and objective quality evaluation

In this study, an image coding algorithm based on directional lifting wavelet transform (DLWT) and universal trellis-coded quantisation (UTCQ) is presented, and the coding performance is evaluated with three objective image quality metrics. Compared to the discrete wavelet transform, DLWT performing prediction and update along the direction of the local region can provide an efficient representation of edges in images, but shows a similar ability in representing the smooth region. To further improve the visual quality of the smooth background regions, UTCQ is adopted to quantising the wavelet coefficients. The proposed algorithm is measured with not only the dominant peak signal-to-noise ratio (PSNR), but also new metrics multi-scale structural similarity index measure (MSSIM) and visual information fidelity (VIF) which provide a better approximation to the perceived image quality than PSNR by taking the property of human visual system (HVS) into account. Experimental results show that the proposed algorithm has the best MSSIM and VIF performance among the compared algorithms (including JPEG2000) for the typical test images, and its decoded images at low bit-rate are visually more appealing in both edges and smooth background regions. For image Barbara, the proposed algorithm outperforms JPEG2000 up to 24.63% relatively in VIF and 1.93%dB in PSNR at 0.5%bpp, at most 3.62% relatively in MSSIM at 0.125%bpp. The experimental results also show that UTCQ does perform better than scalar quantisation (SQ) in MSSIM and VIF and improves the subjective visual quality, although UTCQ is not necessarily better than SQ in PSNR.

Use of the cross‐correlation component of the multiscale structural similarity metric (R* metric) for the evaluation of medical images

The aim of the present work is to analyze the potential of the cross-correlation component of the multiscale structural similarity metric (R*) to predict human performance in detail detection tasks closely related with diagnostic x-ray images. To check the effectiveness of R*, the authors have initially applied this metric to a contrast detail detection task. Threshold contrast visibility using the R* metric was determined for two sets of images of a contrast-detail phantom (CDMAM). Results from R* and human observers were compared as far as the contrast threshold was concerned. A comparison between the R* metric and two algorithms currently used to evaluate CDMAM images was also performed. Similar trends for the CDMAM detection task of human observers and R* were found in this study. Threshold contrast visibility values using R* are statistically indistinguishable from those obtained by human observers (F-test statistics: p > 0.05). These results using R* show that it could be used to mimic human observers for certain tasks, such as the determination of contrast detail curves in the presence of uniform random noise backgrounds. The R* metric could also outperform other metrics and algorithms currently used to evaluate CDMAM images and can automate this evaluation task.

Image Multidistortion Estimation

We present a method for estimating the amount of noise and blur in a distorted image. Our method is based on the multiscale structural similarity (MS-SSIM) framework that, although designed to measure image quality, is used to estimate the amount of blur and noise in a degraded image given a reference image. We show that there exists a bijective mapping between the 2-D noise/blur space and the 3-D MS-SSIM space, which allows to recover distortion parameters. That mapping allows to formulate the multidistortion-estimation problem as a classical optimization problem. Various search strategies such as Newton, simplex, NewUOA, and brute-force search are presented and compared. We also show that a bicubic patch can be used to approximate the bijective mapping between the noise/blur space and the 3-D MS-SSIM space. Interestingly, the use of such a patch reduces the processing time by a factor of 40 without significantly reducing precision. Based on quantitative results, we show that the amount of different types of blur and noise in a distorted image can be recovered with accuracy of roughly 2% and 8%, respectively. Our methods are compared with four state-of-the-art noise- and blur-estimation techniques.

Information Content Weighting for Perceptual Image Quality Assessment

Many state-of-the-art perceptual image quality assessment (IQA) algorithms share a common two-stage structure: local quality/distortion measurement followed by pooling. While significant progress has been made in measuring local image quality/distortion, the pooling stage is often done in ad-hoc ways, lacking theoretical principles and reliable computational models. This paper aims to test the hypothesis that when viewing natural images, the optimal perceptual weights for pooling should be proportional to local information content, which can be estimated in units of bit using advanced statistical models of natural images. Our extensive studies based upon six publicly-available subject-rated image databases concluded with three useful findings. First, information content weighting leads to consistent improvement in the performance of IQA algorithms. Second, surprisingly, with information content weighting, even the widely criticized peak signal-to-noise-ratio can be converted to a competitive perceptual quality measure when compared with state-of-the-art algorithms. Third, the best overall performance is achieved by combining information content weighting with multiscale structural similarity measures.

Fast structural similarity index algorithm

The development of real-time image and video quality assessment algorithms is an important direction on which little research has focused. Towards this end, we present a design of real-time implementable full-reference image/video quality algorithms that are based on the Structural SIMilarity (SSIM) index and multi-scale SSIM (MS-SSIM) index. The proposed algorithms, which modify SSIM/MS-SSIM to achieve speed of execution, were tested on the LIVE Image Quality Database and LIVE Video Quality Database. The experimental results show that the performance of the new, fast algorithms is commensurate with that of SSIM and MS-SSIM, but with much lower computational complexity. Indeed, the proposed Fast MS-SSIM algorithm is 10 times faster (lower complexity) than the MS-SSIM algorithm, while the proposed Fast SSIM is 2.68 times faster than SSIM without parallel computing optimization.

Advantage in image fidelity and additional computing time of JPEG2000 3D in comparison to JPEG2000 in compressing abdomen CT image datasets of different section thicknesses

This study aimed to assess the advantage of the Joint Photographic Experts Group 2000 (JPEG2000) 3D (part 2) over JPEG2000 in compressing abdomen computed tomography (CT) image data sets of different section thicknesses (STs). Twenty CT scans were reconstructed with six STs (0.67, 1, 2, 3, 4, and 5 mm) and were then compressed to seven compression ratios (CRs) (reversible, 6:1, 8:1, 10:1, 12:1, 14:1, and 16:1) using JPEG2000 and JPEG2000 3D algorithms. Computing (encoding and decoding) times were measured. The image fidelity of the compressed images was quantitatively measured with two computerized image fidelity metrics, peak signal-to-noise ratio (PSNR) and multiscale structural similarity (MS-SSIM). For 120 selected case-relevant images (20 patients x one image per patient x 6 STs), five radiologists independently compared original and compressed images and assessed the fidelity of the compressed images on a four-grade scale. Wilcoxon signed-rank tests and Friedman tests with post hoc Dunn tests were used for the comparisons between the two compressions and among the six STs, respectively For each combination of the ST and irreversible CR, JPEG2000 3D showed higher image fidelity than JPEG2000 in terms of PSNR (p < 0.0001), MS-SSIM (p < 0.0001), and five radiologists' grading (p-values ranged from <0.0001 to 0.003). At each CR, the advantage of JPEG2000 3D in image fidelity, measured as the differences in the two computerized image fidelity metrics (PSNR and MS-SSIM), significantly increased as the ST increased from 0.67 to 2 mm, and then slowly decreased as the ST increased from 2 to 5 mm. Similar trends were observed in visual analyses of 120 selected images by five radiologists. At each CR, the 3D-to-2D encoding-time ratio significantly decreased (p < 0.001) as the ST increased from 0.67 to 2 mm, and then slowly increased (p < 0.001) as the ST increased from 2 to 5 mm. The 3D-to-2D decoding-time ratio at each CR did not show a notable biphasic trend across the ST. In compressing abdomen CT image data sets of different STs, the advantage of JPEG2000 3D over JPEG2000 increases as the ST increases from 0.67 to 2 mm, and then slowly decreases as the ST increases from 2 to 5 mm. The practical advantage of JPEG2000 3D is limited for a submillimeter ST due to its greater computing time with only a marginal improvement in image fidelity.

A Comparison of Three Image Fidelity Metrics of Different Computational Principles for JPEG2000 Compressed Abdomen CT Images

This study aimed to evaluate three image fidelity metrics of different computational principles--peak signal-to-noise ratio (PSNR), high-dynamic range visual difference predictor (HDR-VDP), and multiscale structural similarity (MS-SSIM)--in measuring the fidelity of JPEG2000 compressed abdomen computed tomography images from a viewpoint of visually lossless compression. Three hundred images with 0.67- or 5-mm section thickness were compressed to one of five compression ratios ranging from reversible compression to 15:1. The fidelity of each compressed image was measured by five radiologists' visual analyses (distinguishable or indistinguishable from the original) and the three metrics. The Spearman rank correlation coefficients of the PSNR, HDR-VDP, and MS-SSIM values with the number of readers responding as indistinguishable were 0.86, 0.94, and 0.86, respectively. Using the pooled readers' responses as the reference standard, the area under the receiver-operating-characteristic curve for the HDR-VDP (0.99) was significantly greater than that for the PSNR (0.95) (p < 0.001) and for the MS-SSIM (0.96) (p = 0.003), and there was no significant difference between the PSNR and MS-SSIM (p = 0.70). In measuring the image fidelity, the HDR-VDP outperforms the PSNR and MS-SSIM, and the MS-SSIM and PSNR are comparable.

Content-partitioned structural similarity index for image quality assessment

The assessment of image quality is important in numerous image processing applications. Two prominent examples, the Structural Similarity Image (SSIM) index and Multi-scale Structural Similarity (MS-SSIM) operate under the assumption that human visual perception is highly adapted for extracting structural information from a scene. Results in large human studies have shown that these quality indices perform very well relative to other methods. However, the performance of SSIM and other Image Quality Assessment (IQA) algorithms are less effective when used to rate blurred and noisy images. We address this defect by considering a four-component image model that classifies image local regions according to edge and smoothness properties. In our approach, SSIM scores are weighted by region type, leading to modified versions of (G-)SSIM and MS-(G-)SSIM, called four-component (G-)SSIM (4-(G-)SSIM) and four-component MS-(G-)SSIM (4-MS-(G-)SSIM). Our experimental results show that our new approach provides results that are highly consistent with human subjective judgment of the quality of blurred and noisy images, and also deliver better overall performance than (G-)SSIM and MS-(G-)SSIM on the LIVE Image Quality Assessment Database.

Assessment of random-noise contamination in digital images via testing on wavelet coefficients

Full-reference image quality assessment methods seek to measure visual similarity between two images (in practice, one original and the other its altered version). It has been established that traditional methods, such as Mean Square Error and Peak Signal-to-Noise Ratio poorly mimic the human visual system and much of the recent research in image quality assessment has been directed toward developing image similarity measures that are more consistent with assessments from human observers. Some extensively tested popular methods in this regard are Visual Image Fidelity (VIF), Structure Similarity Index (SSIM) and its variants Multi-scale Structure Similarity Index (MS-SSIM) and Information Content Weighted Multi-scale Structure Similarity Index (IW-SSIM). However, experiments show that these methods may produce drastically different similarity indices for different images contaminated with the same source of random noise. In this article, we propose a new full-reference image quality assessment method, namely, Wavelet-based Non-parametric Structure Similarity Index (WNPSSIM), specifically designed to detect visual similarity between images contaminated with all sorts of random noises. WNPSSIM is based on a rank test of the hypothesis of identical images conducted on the wavelet domain. Our experimental comparisons demonstrate that WNPSSIM provides similar ranking as MS-SSIM, IW-SSIM and VIF for images contaminated with different random noises in general though the methodology is very different. In addition, WNPSSIM corrects the aforementioned shortcoming of assigning sharply different similarity indices for different images contaminated with the same source of random noise. AMS 2000 subject classifications: Primary 68U10, 97K80, 62H35; secondary 62G10.