Model-guided adaptive sampling for Bayesian model selection

Abstract

We propose an adaptive design for variable selection in Bayesian modeling process. First randomly select some models to evaluate (e.g. by posterior model probability). Using these models, we predict the performance of all models in the candidate pool, based on which more models are selected and evaluated, in which models with good predicted performance or large prediction variances have high probabilities of being selected. Newly sampled models are used to update the performance predictions of candidate models. Repeat the process until informative models are not likely to be left unsampled in terms of the preset model selection criterion. When there are high-dimensional variables, we propose the use of highest-resolution-minimum-aberration-fractional-factorial design to select candidate-model sets to enable inferences on main effects and low-level interactions of variables. Simulations and a real data example have shown that the proposed adaptive design is efficient in finding informative models compared with other variable selection procedures.