Global-local Shrinkage Priors Research Articles

Trend and Variance Adaptive Bayesian Changepoint Analysis and Local Outlier Scoring

We adaptively estimate both changepoints and local outlier processes in a Bayesian dynamic linear model with global-local shrinkage priors in a novel model we call Adaptive Bayesian Changepoints with Outliers (ABCO). We use a state-space approach to identify a dynamic signal in the presence of outliers and measurement error with stochastic volatility. We find that global state equation parameters are inadequate for most real applications and we include local parameters to track noise at each time-step. This setup provides a flexible framework to detect unspecified changepoints in complex series, such as those with large interruptions in local trends, with robustness to outliers and heteroscedastic noise. Finally, we compare our algorithm against several alternatives to demonstrate its efficacy in diverse simulation scenarios and two empirical examples on the U.S. economy.

Sparse Bayesian Inference on Gamma-Distributed Observations Using Shape-Scale Inverse-Gamma Mixtures

In various applications, we deal with high-dimensional positive-valued data that often exhibits sparsity. This paper develops a new class of continuous global-local shrinkage priors tailored to analyzing gamma-distributed observations where most of the underlying means are concentrated around a certain value. Unlike existing shrinkage priors, our new prior is a shape-scale mixture of inverse-gamma distributions, which has a desirable interpretation of the form of posterior mean and admits flexible shrinkage. We show that the proposed prior has two desirable theoretical properties; Kullback-Leibler super-efficiency under sparsity and robust shrinkage rules for large observations. We propose an efficient sampling algorithm for posterior inference. The performance of the proposed method is illustrated through simulation and two real data examples, the average length of hospital stay for COVID-19 in South Korea and adaptive variance estimation of gene expression data.

Bayesian variable selection and estimation in binary quantile regression using global-local shrinkage priors

In this paper, we construct a Bayesian hierarchical model with global-local shrinkage priors for the regression coefficients, which includes the horseshoe prior and normal-gamma prior. This model is used for high-dimensional quantile regression models with dichotomous response data. We have developed an efficient sampling algorithm to generate posterior samplings for making posterior inference. We use a location-scale mixture representation of the asymmetric Laplace distribution. We assess the performance of the proposed methods through Monte Carlo simulations and two real-data applications in terms of parameter estimation and variable selection. Numerical results demonstrate that the proposed methods perform comparably with existing Bayesian methods under a variety of scenarios.

High-Dimensional Bayesian Network Classification with Network Global-Local Shrinkage Priors

This article proposes a novel Bayesian binary classification framework for networks with labeled nodes. Our approach is motivated by applications in brain connectome studies, where the overarching goal is to identify both regions of interest (ROIs) in the brain and connections between ROIs that influence how study subjects are classified. We propose a novel binary logistic regression framework with the network as the predictor, and model the associated network coefficient using a novel class of global-local network shrinkage priors. We perform a theoretical analysis of a member of this class of priors (which we call the Network Lasso Prior) and show asymptotically correct classification of networks even when the number of network edges grows faster than the sample size. Two representative members from this class of priors, the Network Lasso prior and the Network Horseshoe prior, are implemented using an efficient Markov Chain Monte Carlo algorithm, and empirically evaluated through simulation studies and the analysis of a real brain connectome dataset.

Bayesian boundary trend filtering

Estimating boundary curves has many applications such as economics, climate science, and medicine. Bayesian trend filtering has been developed as one of locally adaptive smoothing methods to estimate the non-stationary trend of data. This paper develops a Bayesian trend filtering for estimating the boundary trend. To this end, the truncated multivariate normal working likelihood and global-local shrinkage priors based on the scale mixtures of normal distribution are introduced. In particular, well-known horseshoe prior for difference leads to locally adaptive shrinkage estimation for boundary trend. However, the full conditional distributions of the Gibbs sampler involve high-dimensional truncated multivariate normal distribution. To overcome the difficulty of sampling, an approximation of truncated multivariate normal distribution is employed. Using the approximation, the proposed models lead to an efficient Gibbs sampling algorithm via the Pólya-Gamma data augmentation. The proposed method is also extended by considering a nearly isotonic constraint. The performance of the proposed method is illustrated through some numerical experiments and real data examples.

Global-Local Shrinkage Priors for Asymptotic Point and Interval Estimation of Normal Means under Sparsity

Global-Local Shrinkage Priors for Asymptotic Point and Interval Estimation of Normal Means under Sparsity

Bayesian penalty methods for evaluating measurement invariance in moderated nonlinear factor analysis.

Measurement invariance (MI) is one of the main psychometric requirements for analyses that focus on potentially heterogeneous populations. MI allows researchers to compare latent factor scores across persons from different subgroups, whereas if a measure is not invariant across all items and persons then such comparisons may be misleading. If full MI does not hold further testing may identify problematic items showing differential item functioning (DIF). Most methods developed to test DIF focused on simple scenarios often with comparisons across two groups. In practical applications, this is an oversimplification if many grouping variables (e.g., gender, race) or continuous covariates (e.g., age) exist that might influence the measurement properties of items; these variables are often correlated, making traditional tests that consider each variable separately less useful. Here, we propose the application of Bayesian Moderated Nonlinear Factor Analysis to overcome limitations of traditional approaches to detect DIF. We investigate how modern Bayesian shrinkage priors can be used to identify DIF items in situations with many groups and continuous covariates. We compare the performance of lasso-type, spike-and-slab, and global-local shrinkage priors (e.g., horseshoe) to standard normal and small variance priors. Results indicate that spike-and-slab and lasso priors outperform the other priors. Horseshoe priors provide slightly lower power compared to lasso and spike-and-slab priors. Small variance priors result in very low power to detect DIF with sample sizes below 800, and normal priors may produce severely inflated type I error rates. We illustrate the approach with data from the PISA 2018 study. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Shrinkage with shrunken shoulders: Gibbs sampling shrinkage model posteriors with guaranteed convergence rates.

Use of continuous shrinkage priors - with a "spike" near zero and heavy-tails towards infinity - is an increasingly popular approach to induce sparsity in parameter estimates. When the parameters are only weakly identified by the likelihood, however, the posterior may end up with tails as heavy as the prior, jeopardizing robustness of inference. A natural solution is to "shrink the shoulders" of a shrinkage prior by lightening up its tails beyond a reasonable parameter range, yielding a regularized version of the prior. We develop a regularization approach which, unlike previous proposals, preserves computationally attractive structures of original shrinkage priors. We study theoretical properties of the Gibbs sampler on resulting posterior distributions, with emphasis on convergence rates of the Pólya-Gamma Gibbs sampler for sparse logistic regression. Our analysis shows that the proposed regularization leads to geometric ergodicity under a broad range of global-local shrinkage priors. Essentially, the only requirement is for the prior on the local scale to satisfy . If further satisfies for , as in the case of Bayesian bridge priors, we show the sampler to be uniformly ergodic.

Introducing shrinkage in heavy-tailed state space models to predict equity excess returns

AbstractWe forecast excess returns of the S &P 500 index using a flexible Bayesian econometric state space model with non-Gaussian features at several levels. More precisely, we control for overparameterization via global–local shrinkage priors on the state innovation variances as well as the time-invariant part of the state space model. The shrinkage priors are complemented by heavy tailed state innovations that cater for potential large breaks in the latent states, even if the degree of shrinkage introduced is high. Moreover, we allow for leptokurtic stochastic volatility in the observation equation. The empirical findings indicate that several variants of the proposed approach outperform typical competitors frequently used in the literature, both in terms of point and density forecasts.

Multilevel time-series models for small area estimation at different frequencies and domain levels

A small area estimation method is developed for repeatedly conducted multipurpose surveys. A multilevel time-series model is proposed that uses direct estimates for the most detailed domains observed at the highest frequency of the repeated survey. A consistent set of estimates at different aggregation levels is then derived by aggregation of the model-based predictions obtained for the most detailed domains observed at the highest frequency. The model borrows strength over time and space via smooth and local level trends at different aggregation levels. The model also borrows information from auxiliary series available from registers with coefficients that can vary over both domains and time. Regional domain random effects are allowed to vary smoothly over space according to a spatial autoregressive process. To account for the diversity of domains and for more volatile time-dependence, nonnormally distributed random effects and trend innovations are used via so-called global-local shrinkage priors. A Bayesian approach is taken, and the model is estimated by MCMC simulation. The method is illustrated with an application to the Dutch Labour Force Survey to produce monthly provincial and quarterly municipal unemployment figures.

On Global-Local Shrinkage Priors for Count Data

Global-local shrinkage priors have been recognized as a useful class of priors that can strongly shrink small signals toward prior means while keeping large signals unshrunk. Although such priors have been extensively discussed under Gaussian responses, in practice, we often encounter count responses. Previous contributions on global-local shrinkage priors cannot be readily applied to count data. In this paper, we discuss global-local shrinkage priors for analyzing a sequence of counts. We provide sufficient conditions under which the posterior mean is unshrunk for very large signals, known as the tail robustness property. Then, we propose tractable priors to satisfy those conditions approximately or exactly and develop a custom posterior computation algorithm for Bayesian inference without tuning parameters. We demonstrate the proposed methods through simulation studies and an application to a real dataset.

Bayesian sparse convex clustering via global-local shrinkage priors

Sparse convex clustering is to group observations and conduct variable selection simultaneously in the framework of convex clustering. Although a weighted L_1 norm is usually employed for the regularization term in sparse convex clustering, its use increases the dependence on the data and reduces the estimation accuracy if the sample size is not sufficient. To tackle these problems, this paper proposes a Bayesian sparse convex clustering method based on the ideas of Bayesian lasso and global-local shrinkage priors. We introduce Gibbs sampling algorithms for our method using scale mixtures of normal distributions. The effectiveness of the proposed methods is shown in simulation studies and a real data analysis.

Shrinkage in the Time-Varying Parameter Model Framework Using the R Package shrinkTVP

Time-varying parameter (TVP) models are widely used in time series analysis to flexibly deal with processes which gradually change over time. However, the risk of overfitting in TVP models is well known. This issue can be dealt with using appropriate global-local shrinkage priors, which pull time-varying parameters towards static ones. In this paper, we introduce the R package shrinkTVP (Knaus, Bitto-Nemling, Cadonna, and FrühwirthSchnatter 2021), which provides a fully Bayesian implementation of shrinkage priors for TVP models, taking advantage of recent developments in the literature, in particular those of Bitto and Frühwirth-Schnatter (2019) and Cadonna, Frühwirth-Schnatter, and Knaus (2020). The package shrinkTVP allows for posterior simulation of the parameters through an efficient Markov Chain Monte Carlo scheme. Moreover, summary and visualization methods, as well as the possibility of assessing predictive performance through log-predictive density scores, are provided. The computationally intensive tasks have been implemented in C++ and interfaced with R. The paper includes a brief overview of the models and shrinkage priors implemented in the package. Furthermore, core functionalities are illustrated, both with simulated and real data.

Threshold cointegration in international exchange rates:A Bayesian approach

This paper considers Bayesian estimation of the threshold vector error correction (TVECM) model in moderate to large dimensions. Using the lagged cointegrating error as a threshold variable gives rise to additional difficulties that typically are solved by utilizing large sample approximations. By relying on Markov chain Monte Carlo methods, we are enabled to circumvent these issues and avoid computationally-prohibitive estimation strategies like the grid search. Due to the proliferation of parameters, we use novel global-local shrinkage priors in the spirit of Griffin and Brown (2010). We illustrate the merits of our approach in an application to five exchange rates vis-á-vis the US dollar by means of a forecasting comparison. Our findings indicate that adopting a non-linear modeling approach improves the predictive accuracy for most currencies relative to a set of simpler benchmark models and the random walk.

Flexible shrinkage in high-dimensional Bayesian spatial autoregressive models

Several recent empirical studies, particularly in the regional economic growth literature, emphasize the importance of explicitly accounting for uncertainty surrounding model specification. Standard approaches to deal with the problem of model uncertainty involve the use of Bayesian model-averaging techniques. However, Bayesian model-averaging for spatial autoregressive models suffers from severe drawbacks both in terms of computational time and possible extensions to more flexible econometric frameworks. To alleviate these problems, this paper presents two global–local shrinkage priors in the context of high-dimensional matrix exponential spatial specifications. A simulation study is conducted to evaluate the performance of the shrinkage priors. Results suggest that they perform particularly well in high-dimensional environments, especially when the number of parameters to estimate exceeds the number of observations. Moreover, we use pan-European regional economic growth data to illustrate the performance of the proposed shrinkage priors.

Uncertainty Quantification for Modern High-Dimensional Regression via Scalable Bayesian Methods

ABSTRACTTremendous progress has been made in the last two decades in the area of high-dimensional regression, especially in the “large p, small n” setting. Such sample starved settings inevitably lead to models which are potentially very unstable and hence quite unreliable. To this end, Bayesian shrinkage methods have generated a lot of recent interest in the modern high-dimensional regression and model selection context. Such methods span the wide spectrum of modern regression approaches and include among others, spike-and-slab priors, the Bayesian lasso, ridge regression, and global-local shrinkage priors such as the Horseshoe prior and the Dirichlet–Laplace prior. These methods naturally facilitate tractable uncertainty quantification and have thus been used extensively across diverse applications. A common unifying feature of these models is that the corresponding priors on the regression coefficients can be expressed as a scale mixture of normals. This property has been leveraged extensively to develop various three-step Gibbs samplers to explore the corresponding intractable posteriors. The convergence of such samplers however is very slow in high dimensions settings, making them disconnected to the very setting that they are intended to work in. To address this challenge, we propose a comprehensive and unifying framework to draw from the same family of posteriors via a class of tractable and scalable two-step blocked Gibbs samplers. We demonstrate that our proposed class of two-step blocked samplers exhibits vastly superior convergence behavior compared to the original three-step sampler in high-dimensional regimes on simulated data as well as data from a variety of applications including gene expression data, infrared spectroscopy data, and socio-economic/law enforcement data. We also provide a detailed theoretical underpinning to the new method by deriving explicit upper bounds for the (geometric) rate of convergence, and by proving that the proposed two-step sampler has superior spectral properties. Supplementary material for this article is available online.

Bayesian Variable Selection and Estimation Based on Global-Local Shrinkage Priors

We consider in this paper simultaneous Bayesian variable selection and estimation for linear regression models with global-local shrinkage priors on the regression coefficients. We propose a variable selection procedure that selects a variable if the ratio of the posterior mean of its regression coefficient to the corresponding ordinary least square estimate is greater than a half. The regression coefficient is estimated by the posterior mean or zero depending on whether the corresponding variable is selected or not. Under the assumption of orthogonal designs, we prove that if the local parameters have polynomial-tailed priors, the proposed method enjoys the oracle property in the sense that it can achieve variable selection consistency and optimal estimation rate at the same time. However, if, instead, an exponential-tailed prior is used for the local parameters, the proposed method has variable selection consistency but not the optimal estimation rate. We show via simulation and real data examples that our proposed selection mechanism works for nonorthogonal designs as well.

Variable Selection via Penalized Credible Regions with Dirichlet–Laplace Global-Local Shrinkage Priors

The method of Bayesian variable selection via penalized credible regions separates model fitting and variable selection. The idea is to search for the sparsest solution within the joint posterior credible regions. Although the approach was successful, it depended on the use of conjugate normal priors. More recently, improvements in the use of global-local shrinkage priors have been made for high-dimensional Bayesian variable selection. In this paper, we incorporate global-local priors into the credible region selection framework. The Dirichlet–Laplace (DL) prior is adapted to linear regression. Posterior consistency for the normal and DL priors are shown, along with variable selection consistency. We further introduce a new method to tune hyperparameters in prior distributions for linear regression. We propose to choose the hyperparameters to minimize a discrepancy between the induced distribution on R-square and a prespecified target distribution. Prior elicitation on R-square is more natural, particularly when there are a large number of predictor variables in which elicitation on that scale is not feasible. For a normal prior, these hyperparameters are available in closed form to minimize the Kullback–Leibler divergence between the distributions.

Gene Network Reconstruction using Global-Local Shrinkage Priors.

Reconstructing a gene network from high-throughput molecular data is an important but challenging task, as the number of parameters to estimate easily is much larger than the sample size. A conventional remedy is to regularize or penalize the model likelihood. In network models, this is often done locally in the neighbourhood of each node or gene. However, estimation of the many regularization parameters is often difficult and can result in large statistical uncertainties. In this paper we propose to combine local regularization with global shrinkage of the regularization parameters to borrow strength between genes and improve inference. We employ a simple Bayesian model with non-sparse, conjugate priors to facilitate the use of fast variational approximations to posteriors. We discuss empirical Bayes estimation of hyper-parameters of the priors, and propose a novel approach to rank-based posterior thresholding. Using extensive model- and data-based simulations, we demonstrate that the proposed inference strategy outperforms popular (sparse) methods, yields more stable edges, and is more reproducible. The proposed method, termed ShrinkNet, is then applied to Glioblastoma to investigate the interactions between genes associated with patient survival.

Compressive dynamic range imaging via Bayesian shrinkage dictionary learning

We apply the Bayesian shrinkage dictionary learning into compressive dynamic-range imaging. By attenuating the luminous intensity impinging upon the detector at the pixel level, we demonstrate a conceptual design of an 8-bit camera to sample high-dynamic-range scenes with a single snapshot. Coding strategies for both monochrome and color cameras are proposed. A Bayesian reconstruction algorithm is developed to learn a dictionary in situ on the sampled image, for joint reconstruction and demosaicking. We use global-local shrinkage priors to learn the dictionary and dictionary coefficients representing the data. Simulation results demonstrate the feasibility of the proposed camera and the superior performance of the Bayesian shrinkage dictionary learning algorithm.