Model Averaging Based on Kullback-Leibler Distance

Abstract

Model averaging is an alternative of model selection for dealing with model uncertainty. This paper proposes a model averaging procedure based on an unbiased estimator of the expected Kullback-Leibler distance. The resulting model average estimator is proved to be asymptotically optimal. When combining the least squares estimators, the model average estimator has the same large sample property as Mallows model average (MMA) estimator developed by Hansen (2007). A modified version of the model average estimator is also suggested for the case of heteroscedastic random errors. Simulation study shows that the proposed model average estimators perform better than some other existing model average estimators in literature and selected estimator by the corrected Akaike information criterion. Our new methods are further applied to analyzing two real-world datasets.