Abstract
Model selection criteria proposed over the years have become common procedures in applied research. This article examines the true model selection rates of any model selection criteria; with true model meaning the data generating model. The rate at which model selection criteria select the true model is important because the decision of model selection criteria affects both interpretation and prediction. This article provides a general functional form for the mean function of the true model selection rates process, for any model selection criteria. Until now, no other article has provided a general form for the mean function of true model selection rate processes. As an illustration of the general form, this article provides the mean function for the true model selection rates of two commonly used model selection criteria, Akaike’s Information Criterion (AIC) and Bayesian Information Criterion (BIC). The simulations reveal deeper insight into properties of consistency and efficiency of AIC and BIC. Furthermore, the methodology proposed here for tracking the mean function of model selection procedures, which is based on accuracy of selection, lends itself for determining sufficient sample size in linear models for reliable inference in model selection.
Highlights
The task of model selection is common in most disciplines where the objective is to select a simple model that “best” explains or predicts the data
The existing literature on Akaike’s Information Criterion (AIC) and Bayesian Information Criterion (BIC), though extensive in discussing properties of consistency and efficiency, does not address the important question of their true model selection rates, when the true model is contained in the class of model under consideration
The graphs corresponding to configuration with p∗ = 3, 4 are not included since the findings about AIC and BIC were the same across all data configurations considered in the simulation study
Summary
The task of model selection is common in most disciplines where the objective is to select a simple model that “best” explains or predicts the data. The most commonly used penalized model selection criteria are the Akaike Information Criterion (Akaike, 1974) and Bayesian Information Criterion (Schwarz, 1978). These model selection procedures are well known and documented in statistical literature. The existing literature on AIC and BIC, though extensive in discussing properties of consistency and efficiency, does not address the important question of their true model selection rates, when the true model is contained in the class of model under consideration. The major conclusion in these two papers implies that the model selected by AIC will contain the true model with probability 1.
Sign-in/Register to view highlights & summary Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
