Scale Mixture Model Research Articles

A Probabilistic Joint Sparse Regression Model for Semisupervised Hyperspectral Unmixing

Semisupervised hyperspectral unmixing finds the ratio of spectral library members in the mixture of hyperspectral pixels to find the proportion of pure materials in a natural scene. The two main challenges are noise in observed spectral vectors and high mutual coherence of spectral libraries. To tackle these challenges, we propose a probabilistic sparse regression method for linear hyperspectral unmixing, which utilizes the implicit relations of neighboring pixels. We partition the hyperspectral image into rectangular patches. The sparse coefficients of pixels in each patch are assumed to be generated from a Laplacian scale mixture model with the same latent variables. These latent variables specify the probability of existence of endmembers in the mixture of each pixel. Experiments on synthetic and real hyperspectral images illustrate the superior performance of the proposed method over alternatives.

NMPE: A normalized metric for measuring generalized spatial distortion of multispectral panshapening fusion

Adaptivity is important in remote sensing image fusion because of the data-intensive and mass-driven processing platforms that call for reliable evaluation metrics to assess the runtime fusion procedure. Spatial distortion, including poor detail, visual disorder, or over-injection, has not been measured as effectively as in the spectral domain. A new metric, namely normalized mean potential energy (NMPE), is proposed in this paper to check the generalized spatial distortion of fused images by calculating the potential energy of marginal filtering distributions using information of high-order Markov random fields. NMPE is defined based on the GFoE model, which is a new high-order model that we built for remote sensing image applications. To incorporate the evaluation experience of human vision system into the GFoE model, the real zero-mean Gabor filters with multiple directions and scales are used as feature extractors, and the Gaussian scale mixture model as the expert function. The model parameters are trained from 200 images of the Berkeley segmentation dataset. Poor detail in a fused image tends to result in small NMPE evaluation with respect to small Gabor scales. Our observation shows that scale invariance exists only for “good” details, so we use large scales of Gabor functions to detect visual disorder. Over-injection is checked when the fused NMPE is much higher than 1. In the experimental procedure, satellite images from Quickbird, LandSat-7, and SPOT-5 were put to fusion with five popular methods to produce different images for visual and digital comparison. It can be concluded from the experiment that NMPE is in line with our subjective judgment to measure the pansharpening quality in terms of enhanced detail, visual distortion, and over-injection.

Mixed Noise Removal via Laplacian Scale Mixture Modeling and Nonlocal Low-Rank Approximation.

Recovering the image corrupted by additive white Gaussian noise (AWGN) and impulse noise is a challenging problem due to its difficulties in an accurate modeling of the distributions of the mixture noise. Many efforts have been made to first detect the locations of the impulse noise and then recover the clean image with image in painting techniques from an incomplete image corrupted by AWGN. However, it is quite challenging to accurately detect the locations of the impulse noise when the mixture noise is strong. In this paper, we propose an effective mixture noise removal method based on Laplacian scale mixture (LSM) modeling and nonlocal low-rank regularization. The impulse noise is modeled with LSM distributions, and both the hidden scale parameters and the impulse noise are jointly estimated to adaptively characterize the real noise. To exploit the nonlocal self-similarity and low-rank nature of natural image, a nonlocal low-rank regularization is adopted to regularize the denoising process. Experimental results on synthetic noisy images show that the proposed method outperforms existing mixture noise removal methods.

Frequentist and Bayesian inference for Gaussian–log‐Gaussian wavelet trees and statistical signal processing applications

We introduce new estimation methods for a subclass of the Gaussian scale mixture models for wavelet trees by Wainwright, Simoncelli and Willsky that rely on modern results for composite likelihoods and approximate Bayesian inference. Our methodology is illustrated for denoising and edge detection problems in two‐dimensional images. Copyright © 2017 John Wiley & Sons, Ltd.

A Note on Upper Tail Behavior of Liouville Copulas

The family of Liouville copulas is defined as the survival copulas of multivariate Liouville distributions, and it covers the Archimedean copulas constructed by Williamson’s d-transform. Liouville copulas provide a very wide range of dependence ranging from positive to negative dependence in the upper tails, and they can be useful in modeling tail risks. In this article, we study the upper tail behavior of Liouville copulas through their upper tail orders. Tail orders of a more general scale mixture model that covers Liouville distributions is first derived, and then tail order functions and tail order density functions of Liouville copulas are derived. Concrete examples are given after the main results.

Fast Distribution Fitting for Parameter Estimation of Range-Weighted Neighborhood Filters

The range variance of neighborhood filters is well estimated via distribution fitting of a chi scale mixtures model proposed in our previous work. However, it introduced computation overheads for deriving empirical distributions and performing iterative fitting. In this letter, we discuss how to greatly reduce the overheads for practical usage while maintaining denoising quality. For empirical distributions, a grid-subsampling strategy is adopted for acceleration. Regarding distribution fitting, two different methods are studied: equal-frequency merged distribution and L-moment fitting. The former reformulates the fitting process into entropy optimization for only few merged bins. It provides 6-13x speedup for model fitting with negligible quality loss and 9-20x speedup with $\leq 0.1 \text{dB}$ PSNR drop by using 20 and 10 bins respectively. The latter performs table lookup of L-moments, instead of conventional moments, for robust fitting of heavy-tailed distributions. The fitting time then becomes negligible with $\leq 0.2 \text{dB}$ drop in most cases, e.g. the overall run time for bilateral $9 \times 9$ filtering can be thus accelerated by around 6x. Experiments on bilateral and non-local means filters are also given to show the speedup, quality and robustness.

Restoration approach of 3D ultrasound images using complex wavelet-based Laplacian-TV mixture prior

3D ultrasound imaging has advantages over other medical imaging modalities, as it does not involve invasive measurements, reduces study time and operator dependency and can display volume information in real time. It is a valuable way of examining many of the body’s internal organs; however the visibility of details of the organs are usually degraded by speckle noise and blurring during the image acquisition process. A number of image restoration methods have been used to reduce the effects of blurring and noise. Most of the restoration methods involve the use of a prior model as a regularisation term to solve an ill-posed inverse problem. In this paper, we present a wavelet-based restoration method for 3D ultrasound image restoration where a Laplacian scale mixture model combined with a total variation constraint is used as a regularisation prior. Our results show that the restored 3D ultrasound volumes produced by this approach have improved quality over those restored using other recently proposed approaches.

Leveraging Genomic Annotations and Pleiotropic Enrichment for Improved Replication Rates in Schizophrenia GWAS.

Most of the genetic architecture of schizophrenia (SCZ) has not yet been identified. Here, we apply a novel statistical algorithm called Covariate-Modulated Mixture Modeling (CM3), which incorporates auxiliary information (heterozygosity, total linkage disequilibrium, genomic annotations, pleiotropy) for each single nucleotide polymorphism (SNP) to enable more accurate estimation of replication probabilities, conditional on the observed test statistic (“z-score”) of the SNP. We use a multiple logistic regression on z-scores to combine information from auxiliary information to derive a “relative enrichment score” for each SNP. For each stratum of these relative enrichment scores, we obtain nonparametric estimates of posterior expected test statistics and replication probabilities as a function of discovery z-scores, using a resampling-based approach that repeatedly and randomly partitions meta-analysis sub-studies into training and replication samples. We fit a scale mixture of two Gaussians model to each stratum, obtaining parameter estimates that minimize the sum of squared differences of the scale-mixture model with the stratified nonparametric estimates. We apply this approach to the recent genome-wide association study (GWAS) of SCZ (n = 82,315), obtaining a good fit between the model-based and observed effect sizes and replication probabilities. We observed that SNPs with low enrichment scores replicate with a lower probability than SNPs with high enrichment scores even when both they are genome-wide significant (p < 5x10-8). There were 693 and 219 independent loci with model-based replication rates ≥80% and ≥90%, respectively. Compared to analyses not incorporating relative enrichment scores, CM3 increased out-of-sample yield for SNPs that replicate at a given rate. This demonstrates that replication probabilities can be more accurately estimated using prior enrichment information with CM3.

Gaussian Scale Mixture Models for Robust Linear Multivariate Regression with Missing Data

We present an algorithm for multivariate robust Bayesian linear regression with missing data. The iterative algorithm computes an approximative posterior for the model parameters based on the variational Bayes (VB) method. Compared to the EM algorithm, the VB method has the advantage that the variance for the model parameters is also computed directly by the algorithm. We consider three families of Gaussian scale mixture models for the measurements, which include as special cases the multivariate t distribution, the multivariate Laplace distribution, and the contaminated normal model. The observations can contain missing values, assuming that the missing data mechanism can be ignored. A Matlab/Octave implementation of the algorithm is presented and applied to solve three reference examples from the literature.

A semiparametric scale-mixture regression model and predictive recursion maximum likelihood

To avoid specification of a particular distribution for the error in a regression model, we propose a flexible scale mixture model with a nonparametric mixing distribution. This model contains, among other things, the familiar normal and Student-t models as special cases. For fitting such mixtures, the predictive recursion method is a simple and computationally efficient alternative to existing methods. We define a predictive recursion-based marginal likelihood function, and estimation of the regression parameters proceeds by maximizing this function. A hybrid predictive recursion–EM algorithm is proposed for this purpose. The method’s performance is compared with that of existing methods in simulations and real data analyses.

Variational Bayesian image restoration with group-sparse modeling of wavelet coefficients

In this work, we present a recent wavelet-based image restoration framework based on a group-sparse Gaussian scale mixture model. A hierarchical Bayesian estimation is derived using a combination of variational Bayesian inference and a subband-adaptive majorization–minimization method that simplifies computation of the posterior distribution. We show that both of these iterative methods can converge together without needing nested loops, and thus good solutions can be found rapidly in the non-convex search space. We also integrate our method, variational Bayesian with majorization minimization (VBMM), with tree-structured modeling of the wavelet coefficients. This extension achieves significant gains in performance over the coefficient-sparse version of the algorithm. The experimental results demonstrate that the proposed method and its tree-structured extensions are effective for various imaging applications such as image deconvolution, image superresolution and compressive sensing magnetic resonance imaging (MRI) reconstruction, and that they outperform more conventional sparsity-inducing methods based on the l1-norm.

Image denoising in hybrid wavelet and quincunx diamond filter bank domain based on Gaussian scale mixture model

The main challenge in image denoising is, how to preserve the information such as edges and textures to get satisfactory visual quality when improving the signal to noise ratio. In this paper, we propose a hybrid filter bank for denoising based on wavelet filter bank and quincunx diamond filter bank. The noisy image is decomposed into different subbands of frequency and orientation using DMeyer wavelet. The quincunx diamond filter bank is designed from finite impulse response (FIR) filters using Kaiser window, which is applied on the detail subband of wavelet filter bank. The directional subband coefficients are modeled with Gaussian scale mixture model (GSM). The Bayes least squares estimator is used to obtain the denoised detail coefficients from the noisy image decomposition. Experimental results show that the new method performs spatial averaging without smoothing edges, and thereby enhances the visual quality and peak signal-to-noise-ratio.

Image Restoration via Simultaneous Sparse Coding: Where Structured Sparsity Meets Gaussian Scale Mixture

In image processing, sparse coding has been known to be relevant to both variational and Bayesian approaches. The regularization parameter in variational image restoration is intrinsically connected with the shape parameter of sparse coefficients’ distribution in Bayesian methods. How to set those parameters in a principled yet spatially adaptive fashion turns out to be a challenging problem especially for the class of nonlocal image models. In this work, we propose a structured sparse coding framework to address this issue—more specifically, a nonlocal extension of Gaussian scale mixture (GSM) model is developed using simultaneous sparse coding (SSC) and its applications into image restoration are explored. It is shown that the variances of sparse coefficients (the field of scalar multipliers of Gaussians)—if treated as a latent variable—can be jointly estimated along with the unknown sparse coefficients via the method of alternating optimization. When applied to image restoration, our experimental results have shown that the proposed SSC–GSM technique can both preserve the sharpness of edges and suppress undesirable artifacts. Thanks to its capability of achieving a better spatial adaptation, SSC–GSM based image restoration often delivers reconstructed images with higher subjective/objective qualities than other competing approaches.

Extremes of scale mixtures of multivariate time series

Factor models have large potential in the modeling of several natural and human phenomena. In this paper we consider a multivariate time series Y n , n ≥ 1 , rescaled through random factors T n , n ≥ 1 , extending some scale mixture models in the literature. We analyze its extremal behavior by deriving the maximum domain of attraction and the multivariate extremal index, which leads to new ways to construct multivariate extreme value distributions. The computation of the multivariate extremal index and the characterization of the tail dependence show an interesting property of these models. More precisely, however much it is the dependence within and between factors T n , n ≥ 1 , the extremal index of the model is unit whenever Y n , n ≥ 1 , presents cross-sectional and sequential tail independence. We illustrate with examples of thinned multivariate time series and multivariate autoregressive processes with random coefficients. An application of these latter to financial data is presented at the end.

Spatial Resolution Enhancement for Hyperspectral Image Using Gaussian Scale Mixtures

In this paper, a wavelet-based Bayesian fusion framework is presented, in which a low spatial resolution hyperspectral (HS) image is fused with a high spatial resolution multispectral (MS) image. Particularly, a multivariate model, Gaussian Scale Mixture (GSM) model, is employed, which is believed to be capable of modeling the distribution of wavelet coefficients more accurately. A practical implementation scheme is also presented for feasible calculations. The proposed approach is validated by simulation experiments for HS and MS image fusion. The experimental results of the proposed approach are also compared with its counterpart employing a Gaussian model for performance evaluation.

Natural image statistics based 3D reduced reference image quality assessment in contourlet domain

Designing a reliable and generic perceptual quality metric is a challenging issue in three-dimensional (3D) visual signal processing. In many practical 3D application scenarios, the original stereoscopic images (i.e., the reference image) cannot be fully accessed, leaving difficulties to quality assessment. To handle this problem, 3D reduced-reference image quality assessment (RRIQA) metrics have been investigated, which only extract small amount of information from the original stereoscopic images. In this paper, we propose a novel 3D RRIQA metric based on 3D natural image statistics in contourlet domain. In this metric, the Gaussian scale mixtures (GSM) model is employed to normalize the coefficients in the contourlet subband of luminance image and disparity map of the 3D images. After divisive normalization transform, we find that the marginal distribution of the coefficients is approximately Gaussian distributed. Based on these investigations, the standard derivations of the fitted Gaussian distribution are determined as the feature parameters in our metric for each contourlet subband. Then, the feature similarity index is employed to measure the 3D visual quality at the receiver side without accessing the reference stereoscopic images. Experimental results demonstrate that the proposed metric has good consistency with 3D subjective perception of human, and can be implemented in many end-to-end 3D video systems.

Implementation of No Reference Distortion Patch Features Image Quality Assessment Algorithm Based on Human Visual System

One of the most popular waveform based coding scheme for images is Transform coding. It is also known that the wavelet coefficients of no reference images possess certain regularities by the image distortions. The NRDPF-IQA (No Reference Distortion Patch Features-Image Quality Assessment) is one of the No-Reference Image Quality algorithms, which is the proposed algorithm in this paper. The proposed algorithm estimates quality based on features of the image. However the previous BRISQUE (Blind/Referenceless Image Spatial Quality Evaluator) algorithm operates based on low computational complexity and C-DIIVINE (complex extension of the DIIVINE) algorithm based on complex Gaussian scale mixture model. In this paper, we present NRDPF-IQA algorithm, which blindly assesses image quality based on advanced generalized Gaussian distortion model. All these distortions have characteristic features that are consistent across No-Reference images. We also compare the results with all existing algorithms. The experimental results show that this NRDPF-IQA to yield an improvement to perform the better results compare with other recent No-Reference algorithms.

Improved regularisation constraints for compressed sensing of multi-slice MRI

In magnetic resonance imaging, the long acquisition time required to capture k-space data according to the Nyquist sampling rule is a major limitation. Methods for reducing the scan time for these types of imaging procedures have attracted considerable research interest. Compressed sensing approaches have recently been applied to allow faster acquisition by undersampling the k-space data. However, random undersampling introduces noise-like artefacts. To address this issue, a number of nonlinear reconstruction methods have been proposed that use norm regularisation with a sparsifying transform. In this paper, we present a reconstruction method in which a Gaussian scale mixture model constraint in the wavelet domain is combined with a total variation constraint for use as a regularisation prior. A series of experimental evaluations are conducted to validate our method using synthetic and real multi-slice MRI data for the purposes of faster acquisition. Our results show that the volume reconstructed by our method has superior quality to the volumes reconstructed by other approaches.

C-DIIVINE: No-reference image quality assessment based on local magnitude and phase statistics of natural scenes

It is widely known that the wavelet coefficients of natural scenes possess certain statistical regularities which can be affected by the presence of distortions. The DIIVINE (Distortion Identification-based Image Verity and Integrity Evaluation) algorithm is a successful no-reference image quality assessment (NR IQA) algorithm, which estimates quality based on changes in these regularities. However, DIIVINE operates based on real-valued wavelet coefficients, whereas the visual appearance of an image can be strongly determined by both the magnitude and phase information.In this paper, we present a complex extension of the DIIVINE algorithm (called C-DIIVINE), which blindly assesses image quality based on the complex Gaussian scale mixture model corresponding to the complex version of the steerable pyramid wavelet transform. Specifically, we applied three commonly used distribution models to fit the statistics of the wavelet coefficients: (1) the complex generalized Gaussian distribution is used to model the wavelet coefficient magnitudes, (2) the generalized Gaussian distribution is used to model the coefficients׳ relative magnitudes, and (3) the wrapped Cauchy distribution is used to model the coefficients׳ relative phases. All these distributions have characteristic shapes that are consistent across different natural images but change significantly in the presence of distortions. We also employ the complex wavelet structural similarity index to measure degradation of the correlations across image scales, which serves as an important indicator of the subbands׳ energy distribution and the loss of alignment of local spectral components contributing to image structure. Experimental results show that these complex extensions allow C-DIIVINE to yield a substantial improvement in predictive performance as compared to its predecessor, and highly competitive performance relative to other recent no-reference algorithms.

Microarray image denoising using complex Gaussian scale mixtures of complex wavelets.

Microarray images when contaminated with noise may severely affect the detection and quantification of gene expression. In this paper, we propose to use the complex Gaussian scale mixture (CGSM) model in complex wavelet domain for noise reduction in complementary DNA microarray images. Based on the joint information in the red and green channel microarray images, we model the complex wavelet coefficients of the channel images jointly using the CGSM, and subsequently perform image denoising using Bayes least square estimator in complex wavelet domain. The experimental results show that using the CGSM of complex wavelet coefficients provides better noise reduction of microarray images compared to other complex wavelet-based models.