Multi-model approach to model selection*1

Abstract

The single-model approach to model selection based on information criteria, such as AIC or BIC, is omnipresent in the signal processing literature. However, any single-model approach picks up only one model and hence misses the potentially significant information associated with the other models fitted to the data. In our opinion this is a drawback: indeed, depending on the application, even the true model structure (assuming that there was one) may not be the best choice for the intended use of the model. The multi-model approach does not suffer from such a problem: using nothing more than the values of AIC or BIC it estimates the a posteriori probabilities of each model under consideration and then it goes on to use all fitted models in a weighted manner according to their posterior likelihoods. We show via a numerical study that the multi-model approach can outperform the single-model approach in terms of statistical accuracy, without unduly increasing the computational burden. The first goal of this paper is to advocate the multi-model approach. A second goal is to introduce some guidelines for numerically studying the performance of a model selection rule.