A Robust Bayesian Analysis of Variable Selection under Prior Ignorance

Abstract

We propose a cautious Bayesian variable selection routine by investigating the sensitivity of a hierarchical model, where the regression coefficients are specified by spike and slab priors. We exploit the use of latent variables to understand the importance of the co-variates. These latent variables also allow us to obtain the size of the model space which is an important aspect of high dimensional problems. In our approach, instead of fixing a single prior, we adopt a specific type of robust Bayesian analysis, where we consider a set of priors within the same parametric family to specify the selection probabilities of these latent variables. We achieve that by considering a set of expected prior selection probabilities, which allows us to perform a sensitivity analysis to understand the effect of prior elicitation on the variable selection. The sensitivity analysis provides us sets of posteriors for the regression coefficients as well as the selection indicators and we show that the posterior odds of the model selection probabilities are monotone with respect to the prior expectations of the selection probabilities. We also analyse synthetic and real life datasets to illustrate our cautious variable selection method and compare it with other well known methods.