On Model Selection Consistency of Bayesian Method for Normal Linear Models

Abstract

We consider the problem of variable selection using Bayesian method in high-dimensional linear models, where the number of explanatory variables K n is possibly much larger than the sample size n. In particular, the true regression coefficients vector β* is required sparsity in the sense of , which describes a general situation when all explanatory variables are relevant, but most of them have very small effects. According to Bayesian inference of Wasserman (1998), given a proper prior to propose joint densities f(y, x) of response y and explanatory variables vector x, the posterior-proposed densities are often close to the true density f 0(y, x), for large n. It's the so-called “density consistency”. We aim to prove that in linear models, density consistency can induce regression function consistency, which ensures good performance of variable selection. Especially in the special case of , when some regression coefficients are bounded away from zero while the rest are exactly zero, our method would identify the underlying true model by giving consistent estimate of true regression coefficients vector. We also provide simulation studies and a real data example to demonstrate satisfactory finite-sample performance of our method.