Shrinkage Priors Research Articles

Variable selection in macroeconomic stress test: a Bayesian quantile regression approach

AbstractThe key assumption in stress test scenarios is that selected risk factors are useful in predicting banks’ tail risks under severe economic conditions. We argue that high-dimensional Bayesian quantile regression models with shrinkage priors are ideal for identifying those factors. We illustrate our methods by identifying key drivers for banks with different asset sizes from a high-dimensional database. We found that leverage indicators, asset prices, and labor market measures are the best predictors of banks’ performance. The usefulness of our methods is further demonstrated by a forecast comparison between the selected variables and those used in the regulatory stress tests.

Concentration of a Sparse Bayesian Model With Horseshoe Prior in Estimating High‐Dimensional Precision Matrix

ABSTRACTPrecision matrices are crucial in many fields such as social networks, neuroscience and economics, representing the edge structure of Gaussian graphical models (GGMs), where a zero in an off‐diagonal position of the precision matrix indicates conditional independence between nodes. In high‐dimensional settings where the dimension of the precision matrix exceeds the sample size and the matrix is sparse, methods like graphical Lasso, graphical SCAD and CLIME are popular for estimating GGMs. While frequentist methods are well‐studied, Bayesian approaches for (unstructured) sparse precision matrices are less explored. The graphical horseshoe estimate, applying the global‐local horseshoe prior, shows superior empirical performance, but theoretical work for sparse precision matrix estimations using shrinkage priors is limited. This paper addresses these gaps by providing concentration results for the tempered posterior with the fully specified horseshoe prior in high‐dimensional settings. Moreover, we also provide novel theoretical results for model misspecification, offering a general oracle inequality for the posterior. A concise set of simulations is performed to validate our theoretical findings.

Locally Adaptive Shrinkage Priors for Trends and Breaks in Count Time Series

Non-stationary count time series characterized by features such as abrupt changes and fluctuations about the trend arise in many scientific domains including biophysics, ecology, energy, epidemiology, and social science domains. Current approaches for integer-valued time series lack the flexibility to capture local transient features while more flexible models for continuous data types are inadequate for universal applications to integer-valued responses such as settings with small counts. We present a modeling framework, the negative binomial Bayesian trend filter (NB-BTF), that offers an adaptive model-based solution to capturing multiscale features with valid integer-valued inference for trend filtering. The framework is a hierarchical Bayesian model with a dynamic global-local shrinkage process. The flexibility of the global-local process allows for the necessary local regularization while the temporal dependence induces a locally smooth trend. In simulation, the NB-BTF outperforms a number of alternative trend filtering methods. Then, we demonstrate the method on weekly power outage frequency in Massachusetts townships. Power outage frequency is characterized by a persistent low level trend with occasional spikes. These illustrations show the estimation of a smooth, non-stationary trend with adequate uncertainty quantification.

Bayesian Shrinkage Estimation of Credible Subgroups for Count Data with Excess Zeros

Heterogeneity of treatment effects due to heterogeneous patient characteristics often arises in clinical trials. Subgroup analysis and the analysis of interactions are the most common approaches for evaluating such heterogeneous effects but do not explicitly address multiplicity issues. Another common challenge of analyzing treatment effect heterogeneity is the large number of possible covariates which inevitably causes problems related to multiplicity and lack of power. In this article, we develop a Bayesian credible subgroups method using continuous shrinkage priors to assess heterogeneity in treatment effects and multiplicity-adjusted benefiting subgroup identification for zero-inflated count data, which are often encountered in medical and public health studies. Our proposed method provides two bounding subgroups for the true benefiting subgroup: one that is probably contained by the true benefiting subgroup and one that probably contains the true benefiting subgroup. A simulation study has been conducted to compare the performance of the proposed method with other methods through frequentist properties. We apply our method to a clinical bladder tumor trial studying the effect of thiotepa treatment on the reduction of the recurrence of bladder tumor.

Group Inverse-Gamma Gamma Shrinkage for Sparse Linear Models with Block-Correlated Regressors

Heavy-tailed continuous shrinkage priors, such as the horseshoe prior, are widely used for sparse estimation problems. However, there is limited work extending these priors to predictors with grouping structures. Of particular interest in this article, is regression coefficient estimation where pockets of high collinearity in the covariate space are contained within known covariate groupings. To assuage variance inflation due to multicollinearity we propose the group inverse-gamma gamma (GIGG) prior, a heavy-tailed prior that can trade-off between local and group shrinkage in a data adaptive fashion. A special case of the GIGG prior is the group horseshoe prior, whose shrinkage profile is correlated within-group such that the regression coefficients marginally have exact horseshoe regularization. We show posterior consistency for regression coefficients in linear regression models and posterior concentration results for mean parameters in sparse normal means models. The full conditional distributions corresponding to GIGG regression can be derived in closed form, leading to straightforward posterior computation. We show that GIGG regression results in low mean-squared error across a wide range of correlation structures and within-group signal densities via simulation. We apply GIGG regression to data from the National Health and Nutrition Examination Survey for associating environmental exposures with liver functionality.

Discrepancy Between Genetically Predicted and Observed BMI Predicts Incident Type 2 Diabetes.

Obesity is a key predictor of type 2 diabetes (T2D). However, metabolic complications are not solely due to increased BMI. We hypothesized that differences between genetically predicted BMI and observed BMI (BMI-diff) could reflect deviation from individual set point and may predict incident T2D. From the UK Biobank cohort, we selected participants of European ancestry without T2D (n = 332,154). The polygenic risk score for BMI was calculated via Bayesian regression and continuous shrinkage priors (PRS-CS). According to the BMI-diff, the 10-year risk of T2D was assessed using multivariable Cox proportional hazards model. Independent data from the Korean Genome and Epidemiology Study (KoGES) cohort from South Korea (n = 7,430) were used for replication. Participants from the UK Biobank were divided into train (n = 268,041) and test set (n = 115,119) to establish genetically predicted BMI. In the test set, the genetically predicted BMI explained 7.1% of the variance of BMI, and there were 3,599 T2D cases (3.1%) during a 10-year follow-up. Participants in the higher quintiles of BMI-diff (more obese than genetically predicted) had significantly higher risk of T2D than those in the lowest quintile after adjusting for observed BMI: the adjusted hazard ratio of the 1st quintile (vs. 5th quintile) was 1.61 (95% CI, 1.26-2.05, P < 0.001). Results were consistent among individuals in the KoGES study. Moreover, higher BMI than predicted was associated with impaired insulin sensitivity. Having a higher BMI than genetically predicted is associated with an increased risk of T2D. These findings underscore the potential to reassess T2D risk based on individual levels of obesity using genetic thresholds for BMI.

Bayesian regression modeling and inference of energy efficiency data: the effect of collinearity and sensitivity analysis

The majority of research predicted heating demand using linear regression models, but they did not give current building features enough context. Model problems such as Multicollinearity need to be checked and appropriate features must be chosen based on their significance to produce accurate load predictions and inferences. Numerous building energy efficiency features correlate with each other and with heating load in the energy efficiency dataset. The standard Ordinary Least Square regression has a problem when the dataset shows Multicollinearity. Bayesian supervised machine learning is a popular method for parameter estimation and inference when frequentist statistical assumptions fail. The prediction of the heating load as the energy efficiency output with Bayesian inference in multiple regression with a collinearity problem needs careful data analysis. The parameter estimates and hypothesis tests were significantly impacted by the Multicollinearity problem that occurred among the features in the building energy efficiency dataset. This study demonstrated several shrinkage and informative priors on likelihood in the Bayesian framework as alternative solutions or remedies to reduce the collinearity problem in multiple regression analysis. This manuscript tried to model the standard Ordinary Least Square regression and four distinct Bayesian regression models with several prior distributions using the Hamiltonian Monte Carlo algorithm in Bayesian Regression Modeling using Stan and the package used to fit linear models. Several model comparison and assessment methods were used to select the best-fit regression model for the dataset. The Bayesian regression model with weakly informative prior is the best-fitted model compared to the standard Ordinary Least Squares regression and other Bayesian regression models with shrinkage priors for collinear energy efficiency data. The numerical findings of collinearity were checked using variance inflation factor, estimates of regression coefficient and standard errors, and sensitivity of priors and likelihoods. It is suggested that applied research in science, engineering, agriculture, health, and other disciplines needs to check the Multicollinearity effect for regression modeling for better estimation and inference.

Drift versus Shift: Decoupling Trends and Changepoint Analysis

We introduce a new approach for decoupling trends (drift) and changepoints (shifts) in time series. Our locally adaptive model-based approach for robustly decoupling combines Bayesian trend filtering and machine learning based regularization. An over-parameterized Bayesian dynamic linear model (DLM) is first applied to characterize drift. Then a weighted penalized likelihood estimator is paired with the estimated DLM posterior distribution to identify shifts. We show how Bayesian DLMs specified with so-called shrinkage priors can provide smooth estimates of underlying trends in the presence of complex noise components. However, their inability to shrink exactly to zero inhibits direct changepoint detection. In contrast, penalized likelihood methods are highly effective in locating changepoints. However, they require data with simple patterns in both signal and noise. The proposed decoupling approach combines the strengths of both, that is, the flexibility of Bayesian DLMs with the hard thresholding property of penalized likelihood estimators, to provide changepoint analysis in complex, modern settings. The proposed framework is outlier robust and can identify a variety of changes, including in mean and slope. It is also easily extended for analysis of parameter shifts in time-varying parameter models like dynamic regressions. We illustrate the flexibility and contrast the performance and robustness of our approach with several alternative methods across a wide range of simulations and application examples.

Trend and Variance Adaptive Bayesian Changepoint Analysis and Local Outlier Scoring

We adaptively estimate both changepoints and local outlier processes in a Bayesian dynamic linear model with global-local shrinkage priors in a novel model we call Adaptive Bayesian Changepoints with Outliers (ABCO). We use a state-space approach to identify a dynamic signal in the presence of outliers and measurement error with stochastic volatility. We find that global state equation parameters are inadequate for most real applications and we include local parameters to track noise at each time-step. This setup provides a flexible framework to detect unspecified changepoints in complex series, such as those with large interruptions in local trends, with robustness to outliers and heteroscedastic noise. Finally, we compare our algorithm against several alternatives to demonstrate its efficacy in diverse simulation scenarios and two empirical examples on the U.S. economy.

Genome-wide polygenic risk scores predict risk of glioma and molecular subtypes.

Polygenic risk scores (PRS) aggregate the contribution of many risk variants to provide a personalized genetic susceptibility profile. Since sample sizes of glioma genome-wide association studies (GWAS) remain modest, there is a need to efficiently capture genetic risk using available data. We applied a method based on continuous shrinkage priors (PRS-CS) to model the joint effects of over 1 million common variants on disease risk and compared this to an approach (PRS-CT) that only selects a limited set of independent variants that reach genome-wide significance (P < 5 × 10-8). PRS models were trained using GWAS stratified by histological (10 346 cases and 14 687 controls) and molecular subtype (2632 cases and 2445 controls), and validated in 2 independent cohorts. PRS-CS was generally more predictive than PRS-CT with a median increase in explained variance (R2) of 24% (interquartile range = 11-30%) across glioma subtypes. Improvements were pronounced for glioblastoma (GBM), with PRS-CS yielding larger odds ratios (OR) per standard deviation (SD) (OR = 1.93, P = 2.0 × 10-54 vs. OR = 1.83, P = 9.4 × 10-50) and higher explained variance (R2 = 2.82% vs. R2 = 2.56%). Individuals in the 80th percentile of the PRS-CS distribution had a significantly higher risk of GBM (0.107%) at age 60 compared to those with average PRS (0.046%, P = 2.4 × 10-12). Lifetime absolute risk reached 1.18% for glioma and 0.76% for IDH wildtype tumors for individuals in the 95th PRS percentile. PRS-CS augmented the classification of IDH mutation status in cases when added to demographic factors (AUC = 0.839 vs. AUC = 0.895, PΔAUC = 6.8 × 10-9). Genome-wide PRS has the potential to enhance the detection of high-risk individuals and help distinguish between prognostic glioma subtypes.

Effect of global shrinkage parameter of horseshoe prior in compressed sensing

This study investigates the effect of the global shrinkage parameter τ of a horseshoe prior, one of the global–local shrinkage priors, on linear regression in sparse signal processing. Statistical mechanics methods are employed to examine the accuracy of signal estimation. The phase diagram of the success and failure of signal recovery in noiseless compressed sensing with varying τ is discussed from the viewpoint of dynamic characterization of approximate message passing (AMP) as a solving algorithm and static characterization of the free-energy landscape. It is found that there exists a parameter region where the AMP algorithm can hardly recover the true signal, even though the true signal is locally stable. The analysis of the free-energy landscape also provides important insight into the optimal choice of τ.

Bayesian Variable Shrinkage and Selection in Compositional Data Regression: Application to Oral Microbiome

Microbiome studies generate multivariate compositional responses, such as taxa counts, which are strictly non-negative, bounded, residing within a simplex, and subject to unit-sum constraint. In presence of covariates (which can be moderate to high dimensional), they are popularly modeled via the Dirichlet-Multinomial (D-M) regression framework. In this paper, we consider a Bayesian approach for estimation and inference under a D-M compositional framework, and present a comparative evaluation of some state-of-the-art continuous shrinkage priors for efficient variable selection to identify the most significant associations between available covariates, and taxonomic abundance. Specifically, we compare the performances of the horseshoe and horseshoe+ priors (with the benchmark Bayesian lasso), utilizing Hamiltonian Monte Carlo techniques for posterior sampling, and generating posterior credible intervals. Our simulation studies using synthetic data demonstrate excellent recovery and estimation accuracy of sparse parameter regime by the continuous shrinkage priors. We further illustrate our method via application to a motivating oral microbiome data generated from the NYC-Hanes study. RStan implementation of our method is made available at the GitHub link: (https://github.com/dattahub/compshrink).

A predictor-informed multi-subject bayesian approach for dynamic functional connectivity.

Dynamic functional connectivity investigates how the interactions among brain regions vary over the course of an fMRI experiment. Such transitions between different individual connectivity states can be modulated by changes in underlying physiological mechanisms that drive functional network dynamics, e.g., changes in attention or cognitive effort. In this paper, we develop a multi-subject Bayesian framework where the estimation of dynamic functional networks is informed by time-varying exogenous physiological covariates that are simultaneously recorded in each subject during the fMRI experiment. More specifically, we consider a dynamic Gaussian graphical model approach where a non-homogeneous hidden Markov model is employed to classify the fMRI time series into latent neurological states. We assume the state-transition probabilities to vary over time and across subjects as a function of the underlying covariates, allowing for the estimation of recurrent connectivity patterns and the sharing of networks among the subjects. We further assume sparsity in the network structures via shrinkage priors, and achieve edge selection in the estimated graph structures by introducing a multi-comparison procedure for shrinkage-based inferences with Bayesian false discovery rate control. We evaluate the performances of our method vs alternative approaches on synthetic data. We apply our modeling framework on a resting-state experiment where fMRI data have been collected concurrently with pupillometry measurements, as a proxy of cognitive processing, and assess the heterogeneity of the effects of changes in pupil dilation on the subjects' propensity to change connectivity states. The heterogeneity of state occupancy across subjects provides an understanding of the relationship between increased pupil dilation and transitions toward different cognitive states.

Estimating Posterior Sensitivities with Application to Structural Analysis of Bayesian Vector Autoregressions

The inherent feature of Bayesian empirical analysis is the dependence of posterior inference on prior parameters, which researchers typically specify. However, quantifying the magnitude of this dependence remains difficult. This article extends Infinitesimal Perturbation Analysis, widely used in classical simulation, to compute asymptotically unbiased and consistent sensitivities of posterior statistics with respect to prior parameters from Markov chain Monte Carlo inference via Gibbs sampling. The method demonstrates the possibility of efficiently computing the complete set of prior sensitivities for a wide range of posterior statistics, alongside the estimation algorithm using Automatic Differentiation. The method’s application is exemplified in Bayesian Vector Autoregression analysis of fiscal policy in U.S. macroeconomic time series data. The analysis assesses the sensitivities of posterior estimates, including the Impulse response functions and Forecast error variance decompositions, to prior parameters under common Minnesota shrinkage priors. The findings illuminate the significant and intricate influence of prior specification on the posterior distribution. This effect is particularly notable in crucial posterior statistics, such as the substantial absolute eigenvalue of the companion matrix, ultimately shaping the structural analysis.

Bayesian Markov switching model for BRICS currencies' exchange rates

AbstractExchange rate modeling has always fascinated researchers because of its complex macroeconomic dynamics. This study documents the exchange rate dynamics of major emerging economies after accounting for their macroeconomic cycles and explores the Bayesian Vector Error Correction Model (VECM) Markov Regime switching model, which uses time‐varying transition probabilities. The main objective is to study the exchange rate dynamics of Brazil, Russia, India, China, and South Africa (BRICS) vis‐à‐vis the US dollar. The Bayesian setup uses two hierarchal shrinkage priors, the normal‐gamma (NG) prior and the Litterman prior, for parameters' estimation. These shrinkage priors allow for a more comprehensive assessment of the regime‐specific coefficients. The model performed well in differentiating between the two regimes for all currencies. The Russian ruble was identified to be the most depreciated currency, whereas the African Rand was the most appreciated. The evaluation of model features revealed that many regime‐specific coefficients differed significantly from their common mean. A forecasting exercise was then performed for the out‐of‐sample period to assess the model's performance. A significant improvement was observed over the basic random walk (RW) model and the linear Bayesian vector autoregression (BVAR) model.

Sophisticated and small versus simple and sizeable: When does it pay off to introduce drifting coefficients in Bayesian vector autoregressions?

AbstractWe assess the relationship between model size and complexity in the time‐varying parameter vector autoregression (VAR) framework via thorough predictive exercises for the euro area, the United Kingdom, and the United States. It turns out that sophisticated dynamics through drifting coefficients are important in small data sets, while simpler models tend to perform better in sizeable data sets. To combine the best of both worlds, novel shrinkage priors help to mitigate the curse of dimensionality, resulting in competitive forecasts for all scenarios considered. Furthermore, we discuss dynamic model selection to improve upon the best performing individual model for each point in time.

Sparse Bayesian Inference on Gamma-Distributed Observations Using Shape-Scale Inverse-Gamma Mixtures

In various applications, we deal with high-dimensional positive-valued data that often exhibits sparsity. This paper develops a new class of continuous global-local shrinkage priors tailored to analyzing gamma-distributed observations where most of the underlying means are concentrated around a certain value. Unlike existing shrinkage priors, our new prior is a shape-scale mixture of inverse-gamma distributions, which has a desirable interpretation of the form of posterior mean and admits flexible shrinkage. We show that the proposed prior has two desirable theoretical properties; Kullback-Leibler super-efficiency under sparsity and robust shrinkage rules for large observations. We propose an efficient sampling algorithm for posterior inference. The performance of the proposed method is illustrated through simulation and two real data examples, the average length of hospital stay for COVID-19 in South Korea and adaptive variance estimation of gene expression data.

Shrinkage priors for isotonic probability vectors and binary data modeling, with applications to dose-response modeling.

Motivated by the need to model dose-response or dose-toxicity curves in clinical trials, we develop a new horseshoe-based prior for Bayesian isotonic regression modeling a binary outcome against an ordered categorical predictor, where the probability of the outcome is assumed to be monotonically non-decreasing with the predictor. The set of differences between outcome probabilities in consecutive categories of the predictor is equipped with a multivariate prior having support over simplex. The Dirichlet distribution, which can be derived from a normalized sum of independent gamma-distributed random variables, is a natural choice of prior, but using mathematical and simulation-based arguments, we show that the resulting posterior is prone to underflow and other numerical instabilities, even under simple data configurations. We propose an alternative prior based on horseshoe-type shrinkage that is numerically more stable. We show that this horseshoe-based prior is not subject to the numerical instability seen in the Dirichlet/gamma-based prior and that the horseshoe-based posterior can estimate the underlying true curve more efficiently than the Dirichlet-based one. We demonstrate the use of this prior in a model predicting the occurrence of radiation-induced lung toxicity in lung cancer patients as a function of dose delivered to normal lung tissue. Our methodology is implemented in the R package isotonicBayes and therefore suitable for use in the design of dose-finding studies or other dose-response modeling contexts.

Multi-Ancestry Polygenic Risk Score for Coronary Heart Disease Based on an Ancestrally Diverse Genome-Wide Association Study and Population-Specific Optimization.

Predictive performance of polygenic risk scores (PRS) varies across populations. To facilitate equitable clinical use, we developed PRS for coronary heart disease (CHD; PRSCHD) for 5 genetic ancestry groups. We derived ancestry-specific and multi-ancestry PRSCHD based on pruning and thresholding (PRSPT) and ancestry-based continuous shrinkage priors (PRSCSx) applied to summary statistics from the largest multi-ancestry genome-wide association study meta-analysis for CHD to date, including 1.1 million participants from 5 major genetic ancestry groups. Following training and optimization in the Million Veteran Program, we evaluated the best-performing PRSCHD in 176,988 individuals across 9 diverse cohorts. Multi-ancestry PRSPT and PRSCSx outperformed ancestry-specific PRSPT and PRSCSx across a range of tuning values. Two best-performing multi-ancestry PRSCHD (ie, PRSPTmult and PRSCSxmult) and 1 ancestry-specific (PRSCSxEUR) were taken forward for validation. PRSPTmult demonstrated the strongest association with CHD in individuals of South Asian ancestry and European ancestry (odds ratio per 1 SD [95% CI, 2.75 [2.41-3.14], 1.65 [1.59-1.72]), followed by East Asian ancestry (1.56 [1.50-1.61]), Hispanic/Latino ancestry (1.38 [1.24-1.54]), and African ancestry (1.16 [1.11-1.21]). PRSCSxmult showed the strongest associations in South Asian ancestry (2.67 [2.38-3.00]) and European ancestry (1.65 [1.59-1.71]), lower in East Asian ancestry (1.59 [1.54-1.64]), Hispanic/Latino ancestry (1.51 [1.35-1.69]), and the lowest in African ancestry (1.20 [1.15-1.26]). The use of summary statistics from a large multi-ancestry genome-wide meta-analysis improved the performance of PRSCHD in most ancestry groups compared with single-ancestry methods. Despite the use of one of the largest and most diverse sets of training and validation cohorts to date, improvement of predictive performance was limited in African ancestry. This highlights the need for larger genome-wide association study datasets of underrepresented populations to enhance the performance of PRSCHD.

Accelerated Structured Matrix Factorization

Matrix factorization exploits the idea that, in complex high-dimensional data, the actual signal typically lies in lower-dimensional structures. These lower dimensional objects provide useful insights, with interpretation favored by sparse structures. Sparsity, in addition, is beneficial in terms of regularization and, thus, to avoid over-fitting. By exploiting Bayesian shrinkage priors, we devise a computationally convenient approach for high-dimensional matrix factorization. The dependence between row and column entities is modeled by inducing flexible sparse patterns within factors. The availability of external information is accounted for in such a way that structures are allowed while not imposed. Inspired by boosting algorithms, we pair the proposed approach with a numerical strategy relying on a sequential inclusion and estimation of low-rank contributions, with a data-driven stopping rule. Practical advantages of the proposed approach are demonstrated by means of a simulation study and the analysis of soccer heatmaps obtained from new generation tracking data. Supplementary materials for this article are available online.