Abstract
When model uncertainty is handled by Bayesian model averaging (BMA) or Bayesian model selection (BMS), the posterior distribution possesses a desirable “oracle property” for parametric inference, if for large enough data it is nearly as good as the oracle posterior, obtained by assuming unrealistically that the true model is known and only the true model is used. We study the oracle properties in a very general context of quasi-posterior, which can accommodate non-regular models with cubic root asymptotics and partial identification. Our approach for proving the oracle properties is based on a unified treatment that bounds the posterior probability of model mis-selection. This theoretical framework can be of interest to Bayesian statisticians who would like to theoretically justify their new model selection or model averaging methods in addition to empirical results. Furthermore, for non-regular models, we obtain nontrivial conclusions on the choice of prior penalty on model complexity, the temperature parameter of the quasi-posterior, and the advantage of BMA over BMS.
Highlights
The terminology of frequentist oracle property was first introduced in Fan and Li (2001) for a frequentist penalization method in model selection, by which statistical inferences “work as well as if the correct submodel were known.” Thereafter the oracle property has become a popular concept in the statistics literature
We are interested in the interplay between several different subjects: Bayesian model averaging (BMA), Bayesian model selection (BMS) based on the Maximum-A-Posteriori (MAP) model, and Bayesian posterior inference based on the unknown true model
We have established a fundamental relation between three different topics: Bayesian model averaging, model selection consistency, and oracle performance in posterior distribution
Summary
The terminology of frequentist oracle property was first introduced in Fan and Li (2001) for a frequentist penalization method in model selection, by which statistical inferences “work as well as if the correct submodel were known.” Thereafter the oracle property has become a popular concept in the statistics literature. We define different versions of Bayesian oracle properties in a general framework with quasi-posteriors and present a systematic way to study them by bounding the probability of model mis-selection. We are interested in the interplay between several different subjects: Bayesian model averaging (BMA), Bayesian model selection (BMS) based on the Maximum-A-Posteriori (MAP) model, and Bayesian posterior inference based on the unknown true model (i.e. the oracle model). The posterior density of θ through Bayesian model averaging (BMA) is given by π(θ| D) ∝ p(D |θ, Mj)π(θ|Mj)π(Mj), for θ ∈ Θ. We explain why the Bayesian version of oracle properties is desirable for dimension reduction in standard regular models, why the more general quasi-Bayesian framework is useful, and why our work will be of interest to the community of Bayesian statisticians
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
