Shrinkage Prior Distribution Research Articles

Bayesian prediction and estimation based on a shrinkage prior for a Poisson regression model

We introduce a Poisson regression model with a natural structure, as viewed from the perspective of information geometry. A shrinkage prior distribution can be utilized similarly to the multi-dimensional Poisson distribution. In the Poisson regression model, we demonstrate that the Bayes extended estimator and Bayesian predictive density based on the shrinkage prior dominate those based on the Jeffreys prior under the Kullback–Leibler loss. By considering a multi-dimensional Poisson process, we show that the loss of the Bayesian predictive density is expressed as the integral of the loss of the Bayes extended estimators. Additionally, we discuss on the numerical evaluation of the Bayes extended estimates

Bayesian Tensor Response Regression with an Application to Brain Activation Studies

This article proposes a novel Bayesian implementation of regression with multi-dimensional array (tensor) response on scalar covariates. The recent emergence of complex datasets in various disciplines presents a pressing need to devise regression models with a tensor valued response. This article considers one such application of detecting neuronal activation in fMRI experiments in presence of tensor valued brain images and scalar predictors. The overarching goal in this application is to identify spatial regions (voxels) of a brain activated by an external stimulus. In such and related applications, we propose to regress responses from all cells (or voxels in brain activation studies) together as a tensor response on scalar predictors, accounting for the structural information inherent in the tensor response. To estimate model parameters with proper cell specific shrinkage, we propose a novel multiway stick breaking shrinkage prior distribution on tensor structured regression coefficients, enabling identification of cells which are related to the predictors. The major novelty of this article lies in the theoretical study of the contraction properties for the proposed shrinkage prior in the tensor response regression when the number of cells grows faster than the sample size. Specifically, estimates of tensor regression coefficients are shown to be asymptotically concentrated around the true sparse tensor in L2-sense under mild assumptions. Various simulation studies and analysis of a brain activation data empirically verify desirable performance of the proposed model in terms of estimation and inference on cell-level parameters.

A longitudinal Bayesian mixed effects model with hurdle Conway-Maxwell-Poisson distribution.

Dental caries (i.e., cavities) is one of the most common chronic childhood diseases and may continue to progress throughout a person's lifetime. The Iowa Fluoride Study (IFS) was designed to investigate the effects of various fluoride, dietary and nondietary factors on the progression of dental caries among a cohort of Iowa school children. We develop a mixed effects model to perform a comprehensive analysis of the longitudinal clustered data of IFS at ages 5, 9, 13, and 17. We combine a Bayesian hurdle framework with the Conway-Maxwell-Poisson regression model, which can account for both excessive zeros and various levels of dispersion. A hierarchical shrinkage prior distribution is used to share the temporal information for predictors in the fixed-effects model. The dependence among teeth of each individual child is modeled through a sparse covariance structure of the random effects across time. Moreover, we obtain the parameter estimates and credible intervals from a Gibbs sampler. Simulation studies are conducted to assess the accuracy and effectiveness of our statistical methodology. The results of this article provide novel tools to statistical practitioners and offer fresh insights to dental researchers on effects of various risk and protective factors on caries progression.

Contraction properties of shrinkage priors in logistic regression

Bayesian shrinkage priors have received a lot of attention recently because of their efficiency in computation and accuracy in estimation and variable selection. In this paper, we study the contraction properties of shrinkage priors in a logistic regression model where the number of covariates is high. For a shrinkage prior distribution that is heavy-tailed and concentrated around zero with high probability such as the horseshoe prior, the Dirichlet–Laplace prior, and the normal-gamma prior with appropriate choices of hyper-parameters, estimates of the logistic regression coefficient are shown to asymptotically concentrate around the true sparse vector in the L2-sense. It is shown that the proposed contraction rate is comparable with the point mass prior that is studied in Atchadé (2017). The simulation study under the logistic regression model verifies the theoretical results by showing that the horseshoe prior and the Dirichlet–Laplace prior perform like the point mass prior for the estimation, variable selection and prediction, and yield much better results than Bayesian lasso and the non-informative normal prior.

Relevant parameter changes in structural break models

Structural break time series models, which are commonly used in macroeconomics and finance, capture unknown structural changes by allowing for abrupt changes to model parameters. However, many specifications suffer from an over-parametrization issue, since typically all parameters have to change when a break occurs. We introduce a sparse change-point model to detect which parameters change over time. We propose a shrinkage prior distribution, which controls model parsimony by limiting the number of parameters that change from one structural break to another. We develop a Bayesian sampler for inference on the sparse change-point model. An extensive simulation study based on AR, ARMA and GARCH processes highlights the excellent performance of the sampler. We provide several empirical applications including an out-of-sample forecasting exercise showing that the Sparse change-point framework compares favorably with other recent time-varying parameter processes.

A Bayesian approach to Mendelian randomization with multiple pleiotropic variants.

SummaryWe propose a Bayesian approach to Mendelian randomization (MR), where instruments are allowed to exert pleiotropic (i.e. not mediated by the exposure) effects on the outcome. By having these effects represented in the model by unknown parameters, and by imposing a shrinkage prior distribution that assumes an unspecified subset of the effects to be zero, we obtain a proper posterior distribution for the causal effect of interest. This posterior can be sampled via Markov chain Monte Carlo methods of inference to obtain point and interval estimates. The model priors require a minimal input from the user. We explore the performance of our method by means of a simulation experiment. Our results show that the method is reasonably robust to the presence of directional pleiotropy and moderate correlation between the instruments. One section of the article elaborates the model to deal with two exposures, and illustrates the possibility of using MR to estimate direct and indirect effects in this situation. A main objective of the article is to create a basis for developments in MR that exploit the potential offered by a Bayesian approach to the problem, in relation with the possibility of incorporating external information in the prior, handling multiple sources of uncertainty, and flexibly elaborating the basic model.

Sparse Change-point HAR Models for Realized Variance

ABSTRACTChange-point time series specifications constitute flexible models that capture unknown structural changes by allowing for switches in the model parameters. Nevertheless most models suffer from an over-parametrization issue since typically only one latent state variable drives the switches in all parameters. This implies that all parameters have to change when a break happens. To gauge whether and where there are structural breaks in realized variance, we introduce the sparse change-point HAR model. The approach controls for model parsimony by limiting the number of parameters which evolve from one regime to another. Sparsity is achieved thanks to employing a nonstandard shrinkage prior distribution. We derive a Gibbs sampler for inferring the parameters of this process. Simulation studies illustrate the excellent performance of the sampler. Relying on this new framework, we study the stability of the HAR model using realized variance series of several major international indices between January 2000 and August 2015.

Frequency coverage properties of a uniform shrinkage prior distribution

ABSTRACTA uniform shrinkage prior (USP) distribution on the unknown variance component of a random-effects model is known to produce good frequency properties. The USP has a parameter that determines the shape of its density function, but it has been neglected whether the USP can maintain such good frequency properties regardless of the choice for the shape parameter. We investigate which choice for the shape parameter of the USP produces Bayesian interval estimates of random effects that meet their nominal confidence levels better than several existent choices in the literature. Using univariate and multivariate Gaussian hierarchical models, we show that the USP can achieve its best frequency properties when its shape parameter makes the USP behave similarly to an improper flat prior distribution on the unknown variance component.

Convergence rate of Bayesian supervised tensor modeling with multiway shrinkage priors

This article studies the convergence rate of the posterior for Bayesian low rank supervised tensor modeling with multiway shrinkage priors. Multiway shrinkage priors constitute a new class of shrinkage prior distributions for tensor parameters in Bayesian low rank supervised tensor modeling to regress a scalar response on a tensor predictor with the primary aim to identify cells in the tensor predictor which are predictive of the scalar response. This novel and computationally efficient framework stems from pressing needs in many applications, including functional magnetic resonance imaging (fMRI) and diffusion tensor imaging (DTI). This article shows that the convergence rate is nearly optimal in terms of in-sample predictive accuracy of the Bayesian supervised low rank tensor model with a multiway shrinkage prior distribution when the number of observations grows. The conditions under which this nearly optimal convergence rate is achieved are seen to be very mild. More importantly, the rate is achieved for an easily computable method, even when the true CP/PARAFAC rank of the tensor coefficient corresponding to the tensor predictor is unknown.

Variable selection for high-dimensional genomic data with censored outcomes using group lasso prior

The variable selection problem is discussed in the context of high-dimensional failure time data arising from the accelerated failure time model. A data augmentation approach is employed in order to deal with censored survival times and to facilitate prior-posterior conjugacy. To identify a set of grouped relevant covariates, a shrinkage prior distribution is specified for regression coefficients mimicking the effect of group lasso penalty. It is noted that unlike the corresponding frequentist method, a Bayesian penalized regression approach cannot shrink the estimates of coefficients to exact zeros in general. Towards resolving the issue, a two-stage thresholding method that exploits the scaled neighborhood criterion and the Bayesian information criterion is devised. Simulation studies are performed to assess the robustness and performance of the proposed method in terms of variable selection accuracy and predictive power. The method is successfully applied to a set of microarray data on the individuals diagnosed with diffuse large B-cell lymphoma. In addition, an R package called psbcGroup, which can be downloaded freely from CRAN, is developed for the implementation of the methods.

Sparse Change-Point Har Models for Realized Variance

Change-point time series specifications constitute flexible models that capture unknown structural changes by allowing for switches in the model parameters. Nevertheless most models suffer from an over-parametrization issue since typically only one latent state variable drives the switches in all parameters. This implies that all parameters have to change when a break happens. To gauge whether and where there are structural breaks in realized variance, we introduce the sparse change-point HAR model. The approach controls for model parsimony by limiting the number of parameters which evolve from one regime to another. Sparsity is achieved thanks to employing a nonstandard shrinkage prior distribution. We derive a Gibbs sampler for inferring the parameters of this process. Simulation studies illustrate the excellent performance of the sampler. Relying on this new framework, we study the stability of the HAR model using realized variance series of several major international indices between January 2000 and August 2015.

Multiple-Shrinkage Multinomial Probit Models with Applications to Simulating Geographies in Public Use Data

Multinomial outcomes with many levels can be challenging to model. Information typically accrues slowly with increasing sample size, yet the parameter space expands rapidly with additional covariates. Shrinking all regression parameters towards zero, as often done in models of continuous or binary response variables, is unsatisfactory, since setting parameters equal to zero in multinomial models does not necessarily imply "no effect." We propose an approach to modeling multinomial outcomes with many levels based on a Bayesian multinomial probit (MNP) model and a multiple shrinkage prior distribution for the regression parameters. The prior distribution encourages the MNP regression parameters to shrink toward a number of learned locations, thereby substantially reducing the dimension of the parameter space. Using simulated data, we compare the predictive performance of this model against two other recently-proposed methods for big multinomial models. The results suggest that the fully Bayesian, multiple shrinkage approach can outperform these other methods. We apply the multiple shrinkage MNP to simulating replacement values for areal identifiers, e.g., census tract indicators, in order to protect data confidentiality in public use datasets.