Scale Mixtures Of Normals Research Articles

Conditions for posterior contraction in the sparse normal means problem

The first Bayesian results for the sparse normal means problem were proven for spike-and-slab priors. However, these priors are less convenient from a computational point of view. In the meanwhile, a large number of continuous shrinkage priors has been proposed. Many of these shrinkage priors can be written as a scale mixture of normals, which makes them particularly easy to implement. We propose general conditions on the prior on the local variance in scale mixtures of normals, such that posterior contraction at the minimax rate is assured. The conditions require tails at least as heavy as Laplace, but not too heavy, and a large amount of mass around zero relative to the tails, more so as the sparsity increases. These conditions give some general guidelines for choosing a shrinkage prior for estimation under a nearly black sparsity assumption. We verify these conditions for the class of priors considered by Ghosh and Chakrabarti (2015), which includes the horseshoe and the normal-exponential gamma priors, and for the horseshoe+, the inverse-Gaussian prior, the normal-gamma prior, and the spike-and-slab Lasso, and thus extend the number of shrinkage priors which are known to lead to posterior contraction at the minimax estimation rate.

On predictive density estimation for location families under integrated squared error loss

Our investigation concerns the estimation of predictive densities and a study of efficiency as measured by the frequentist risk of such predictive densities with integrated squared error loss. Our findings relate to a d -variate spherically symmetric observable X ∼ p X ( ‖ x − μ ‖ 2 ) and the objective of estimating the density of Y ∼ q Y ( ‖ y − μ ‖ 2 ) based on X . We describe Bayes estimation, minimum risk equivariant estimation (MRE), and minimax estimation. We focus on the risk performance of the benchmark minimum risk equivariant estimator, plug-in estimators, and plug-in type estimators with expanded scale. For the multivariate normal case, we make use of a duality result with a point estimation problem bringing into play reflected normal loss. In three or more dimensions (i.e., d ≥ 3 ), we show that the MRE predictive density estimator is inadmissible and provide dominating estimators. This brings into play Stein-type results for estimating a multivariate normal mean with a loss which is a concave and increasing function of ‖ μ ˆ − μ ‖ 2 . We also study the phenomenon of improvement on the plug-in density estimator of the form q Y ( ‖ y − a X ‖ 2 ) , 0 < a ≤ 1 , by a subclass of scale expansions 1 c d q Y ( ‖ ( y − a X ) / c ‖ 2 ) with c > 1 , showing in some cases, inevitably for large enough d , that all choices c > 1 are dominating estimators. Extensions are obtained for scale mixture of normals including a general inadmissibility result of the MRE estimator for d ≥ 3 .

Bayesian analysis of censored linear regression models with scale mixtures of normal distributions

As is the case of many studies, the data collected are limited and an exact value is recorded only if it falls within an interval range. Hence, the responses can be either left, interval or right censored. Linear (and nonlinear) regression models are routinely used to analyze these types of data and are based on normality assumptions for the errors terms. However, those analyzes might not provide robust inference when the normality assumptions are questionable. In this article, we develop a Bayesian framework for censored linear regression models by replacing the Gaussian assumptions for the random errors with scale mixtures of normal (SMN) distributions. The SMN is an attractive class of symmetric heavy-tailed densities that includes the normal, Student-t, Pearson type VII, slash and the contaminated normal distributions, as special cases. Using a Bayesian paradigm, an efficient Markov chain Monte Carlo algorithm is introduced to carry out posterior inference. A new hierarchical prior distribution is suggested for the degrees of freedom parameter in the Student-t distribution. The likelihood function is utilized to compute not only some Bayesian model selection measures but also to develop Bayesian case-deletion influence diagnostics based on the q-divergence measure. The proposed Bayesian methods are implemented in the R package BayesCR. The newly developed procedures are illustrated with applications using real and simulated data.

Linear censored regression models with scale mixtures of normal distributions

In the framework of censored regression models the random errors are routinely assumed to have a normal distribution, mainly for mathematical convenience. However, this method has been criticized in the literature because of its sensitivity to deviations from the normality assumption. Here, we first establish a new link between the censored regression model and a recently studied class of symmetric distributions, which extend the normal one by the inclusion of kurtosis, called scale mixtures of normal (SMN) distributions. The Student-t, Pearson type VII, slash, contaminated normal, among others distributions, are contained in this class. A member of this class can be a good alternative to model this kind of data, because they have been shown its flexibility in several applications. In this work, we develop an analytically simple and efficient EM-type algorithm for iteratively computing maximum likelihood estimates of the parameters, with standard errors as a by-product. The algorithm has closed-form expressions at the E-step, that rely on formulas for the mean and variance of certain truncated SMN distributions. The proposed algorithm is implemented in the R package SMNCensReg. Applications with simulated and a real data set are reported, illustrating the usefulness of the new methodology.

A Bayesian non-parametric approach to asymmetric dynamic conditional correlation model with application to portfolio selection

A Bayesian non-parametric approach for modeling the distribution of multiple returns is proposed. More specifically, an asymmetric dynamic conditional correlation (ADCC) model is considered to estimate the time-varying correlations of financial returns where the individual volatilities are driven by GJR-GARCH models. This composite model takes into consideration the asymmetries in individual assets’ volatilities, as well as in the correlations. The errors are modeled using a Dirichlet location–scale mixture of multivariate Normals allowing for a flexible return distribution in terms of skewness and kurtosis. This gives rise to a Bayesian non-parametric ADCC (BNP-ADCC) model, as opposed to a symmetric specification, called BNP-DCC. Then these two models are compared using a sample of Apple Inc. and NASDAQ Industrial index daily returns. The obtained results reveal that for this particular data set the BNP-ADCC outperforms the BNP-DCC model. Finally, an illustrative asset allocation exercise is presented.

Bayesian Analysis of Least Absolute Relative Error Regression

In an important article by Chen et al. (2010) introduced a new distribution compatible with maximum likelihood estimation in a Least Absolute Relative Error (LARE) setting. In this article, we show first that the posterior of the model is log – concave and thus specialized and highly efficient techniques can be used to perform Bayesian inference without the use of MCMC since they provide independent draws from the posterior. Second, we approximate the distribution using a finite mixture of normals. Surprisingly, the log-LARE distribution can be approximated using a finite scale mixture of normals with few components.

A generalized class of skew distributions and associated robust quantile regression models

AbstractThis article proposes a generalized class of univariate skew distributions that are constructed through partitioning two scaled mixture of normal (Gaussian) distributions. The proposed distributions have a skewness parameter defined in the interval (0,1), allowing direct application to parametric quantile regression. Employing scale mixture of normals facilitates efficient estimation via Markov chain Monte Carlo methods. Two simulation studies, one on estimation with skew error regression models, the other on parametric quantile regression models reveal favourable estimation properties. Two corresponding empirical studies, one analysing U.S. market returns, the other on infant birthweight data further illustrate the proposed distributions and their estimation. The Canadian Journal of Statistics 42: 579–596; 2014 © 2014 Statistical Society of Canada

Comment on Article by Finegold and Drton

Scale mixtures of normals have been discussed extensively in the literature as heavy-tailed alternatives to the normal distribution for robust modeling. They have been used either as error models to handle outliers or as prior distributions to provide more reasonable shrinkage of model parameters. The proposed method by Finegold and Drton goes beyond the existing literature both in terms of application (graphical models) and methodology (Dirichlet t) for outlier handling. While this approach can be applied to many other problems, in this discussion I will focus on its application in Bayesian modeling of high throughput biological data.

Inference in Two-Piece Location-Scale Models with Jeffreys Priors

This paper addresses the use of Jeffreys priors in the context of univariate three-parameter location-scale models, where skewness is introduced by differing scale parameters either side of the location. We focus on various commonly used parameterizations for these models. Jeffreys priors are shown to lead to improper posteriors in the wide and practically relevant class of distributions obtained by skewing scale mixtures of normals. Easily checked conditions under which independence Jeffreys priors can be used for valid inference are derived. We also investigate two alternative priors, one of which is shown to lead to valid Bayesian inference for all practically interesting parameterizations of these models and is our recommendation to practitioners. We illustrate some of these models using real data.

Multivariate generalized Poisson geometric process model with scale mixtures of normal distributions

Abstract This paper proposes a new model named as the multivariate generalized Poisson log- t geometric process (MGPLTGP) model to study multivariate time-series of counts with overdispersion or underdispersion, non-monotone trends within each time-series and positive or negative correlation between pairs of time-series. This model assumes that the multivariate counts follow independent generalized Poisson distributions with an additional parameter to adjust for different degrees of dispersion including overdispersion and underdispersion. Their means after discounting the trend effect geometrically by ratio functions form latent stochastic processes and follow a multivariate log- t distribution with a flexible correlation structure to capture both positive correlation and negative correlation. By expressing the multivariate Student’s t -distribution in scale mixtures of normals, the model can be implemented through Markov chain Monte Carlo algorithms via the user-friendly WinBUGS software. The applicability of the MGPLTGP model is illustrated through an analysis of the possession and/or use of two illicit drugs, amphetamines and narcotics in New South Wales, Australia.

A robust extension of the bivariate Birnbaum–Saunders distribution and associated inference

We propose here a robust extension of the bivariate Birnbaum–Saunders (BS) distribution derived recently by Kundu et al. (2010). This extension is based on scale mixtures of normal (SMN) distributions that are used for modeling symmetric data. This type of bivariate Birnbaum–Saunders distribution based on SMN models is an absolutely continuous distribution whose marginals are of univariate Birnbaum–Saunders type. We then develop the EM-algorithm for the maximum likelihood (ML) estimation of the model parameters, and illustrate the obtained results with a real data and display the robustness feature of the estimation procedure developed here.

A new algorithm for maximum likelihood estimation in normal scale-mixture generalized autoregressive conditional heteroskedastic models

In this paper, we propose a new generalized autoregressive conditional heteroskedastic (GARCH) model using infinite normal scale-mixtures which can suitably avoid order selection problems in the application of finite normal scale-mixtures. We discuss its theoretical properties and develop a two-stage algorithm for the maximum likelihood estimator to estimate the mixing distribution non-parametric maximum likelihood estimator (NPMLE) as well as GARCH parameters (two-stage MLE). For the estimation of a mixing distribution, we employ a fast computational algorithm proposed by Wang [On fast computation of the non-parametric maximum likelihood estimate of a mixing distribution. J R Stat Soc Ser B. 2007;69:185–198] under the gradient characterization of the non-parametric mixture likelihood. The GARCH parameters are then estimated either using the expectation-mazimization algorithm or general optimization scheme. In addition, we propose a new forecasting algorithm of value-at-risk (VaR) using the two-stage MLE and the NPMLE. Through a simulation study and real data analysis, we compare the performance of the two-stage MLE with the existing ones including quasi-maximum likelihood estimator based on the standard normal density and the finite normal mixture quasi maximum estimated-likelihood estimator (cf. Lee S, Lee T. Inference for Box–Cox transformed threshold GARCH models with nuisance parameters. Scand J Stat. 2012;39:568–589) in terms of the relative efficiency and accuracy of VaR forecasting.

Simplified pair copula constructions—Limitations and extensions

So-called pair copula constructions (PCCs), specifying multivariate distributions only in terms of bivariate building blocks (pair copulas), constitute a flexible class of dependence models. To keep them tractable for inference and model selection, the simplifying assumption, that copulas of conditional distributions do not depend on the values of the variables which they are conditioned on, is popular.We show that the only Archimedean copulas in dimension d≥3 which are of the simplified type are those based on the Gamma Laplace transform or its extension, while the Student-t copulas are the only one arising from a scale mixture of Normals. Further, we illustrate how PCCs can be adapted for situations where conditional copulas depend on values which are conditioned on, and demonstrate a technique to assess the distance of a multivariate distribution from a nearby distribution that satisfies the simplifying assumption.

Bayesian Inference for P(X&lt;Y) Using Asymmetric Dependent Distributions

This paper studies Bayesian inference for θ=P(X<Y) in the case where the marginal distributions of X and Y belong to classes of distributions obtained by skewing scale mixtures of normals. We separately address the cases where X and Y are independent or dependent random variables. Dependencies between X and Y are modelled using a Gaussian copula. Noninformative benchmark and vague priors are provided for these scenarios and conditions for the existence of the posterior distribution of θ are presented. We show that the use of the Bayesian models proposed here is also valid in the presence of set observations. Examples using simulated and real data sets are presented.

State space mixed models for binary responses with scale mixture of normal distributions links

A state space mixed models for binary time series where the inverse link function is modeled to be a cumulative distribution function of the scale mixture of normal (SMN) distributions. Specific inverse links examined include the normal, Student-t, slash and the variance gamma links. The threshold latent approach to represent the binary system as a linear state space model is considered. Using a Bayesian paradigm, an efficient Markov chain Monte Carlo (MCMC) algorithm is introduced for parameter estimation. The proposed methods are illustrated with real data sets. Empirical results showed that the slash inverse link fits better over the usual inverse probit link.

Pairwise Marginal Likelihood for the Bradley–Terry Model

The statistical inference for the Bradley–Terry model with logit link and random effects is often made cumbersome by the high-dimensional intractable integrals involved in the marginal likelihood. An inferential methodology based on the marginal pairwise likelihood approach is proposed. This method belongs to the broad class of composite likelihood and involves marginal pairs probabilities of the responses. These probabilities are derived from the approximation of the logistic distribution function by a normal scale mixture. The sample performance of this approach is evaluated by a simulation study. The main motivation of this work is the lizards data, on which the the proposed approach is illustrated.

Simulation-based Regularized Logistic Regression

In this paper, we develop a simulation-based framework for regularized logistic regression, exploiting two novel results for scale mixtures of normals. By carefully choosing a hierarchical model for the likelihood by one type of mixture, and implementing regularization with another, we obtain new MCMC schemes with varying efficiency depending on the data type (binary v. binomial, say) and the desired estimator (maximum likelihood, maximum a posteriori, posterior mean). Advantages of our omnibus approach include flexibility, computational efficiency, applicability in p≫n settings, uncertainty estimates, variable selection, and assessing the optimal degree of regularization. We compare our methodology to modern alternatives on both synthetic and real data. An R package called reglogit is available on CRAN.

Efficient Estimation of Some Elliptical Copula Regression Models through Scale Mixtures of Normal

We simplify the implementation of some elliptical copula regression models through the normal representation. Both copula and marginal probability density functions are expressed as the scale mixtures of normals to facilitate the estimation procedure. With the fact that all elliptical distributions have the same correlation structure and some of them can be expressed as a scale mixture of multivariate normal, we can then estimate their correlation matrix with ease. Two simulation studies are illustrated to assess the performance of our proposed models and methods. We also conduct two empirical studies on U.S. excess return and Thai wage earnings to show the applicability to the multivariate Capital Asset Pricing Model and the bivariate seemingly unrelated Tobit model, respectively.

On estimation of measurement error models with replication under heavy-tailed distributions

Measurement error (errors-in-variables) models are frequently used in various scientific fields, such as engineering, medicine, chemistry, etc. In this work, we consider a new replicated structural measurement error model in which the replicated observations jointly follow scale mixtures of normal (SMN) distributions. Maximum likelihood estimates are computed via an EM type algorithm method. A closed expression is presented for the asymptotic covariance matrix of those estimators. The SMN measurement error model provides an appealing robust alternative to the usual model based on normal distributions. The results of simulation studies and a real data set analysis confirm the robustness of SMN measurement error model.

On a GARCH Model with Normal Scale Mixture Innovations

Recently, there has been a lot of interest in modeling real data with a heavy tailed distribution. A popular candidate is the so-called generalized autoregressive conditional heteroscedastic (GARCH) model. Unfortunately, the tails of normal GARCH models are not thick enough in some applications. In this paper, we propose a GARCH model with normal scale mixture innovations, the parameters estimation procedure using EM algorithm is also provided. It is shown that GARCH models with normal scale mixture innovations have tails thicker than those of normal GARCH models. Therefore, the GARCH models with normal scale mixture innovations are more capable of capturing the heavy-tailed features in real data. Shanghai Stock Market Index as a real example illustrates the results.