The Consistency of MDL for Linear Regression Models With Increasing Signal-to-Noise Ratio

Abstract

Recent work by Ding and Kay has demonstrated that the Bayesian information criterion (BIC) is an inconsistent estimator of model order in nested model selection as the noise variance τ*→ 0. Unfortunately, Ding and Kay have erroneously concluded that the minimum description length (MDL) principle also leads to inconsistent estimates of model order in this setting by equating BIC with MDL. This correspondence shows that only the earlier MDL criterion based on asymptotic assumptions has this problem, and proves that the new MDL linear regression criteria based on normalized maximum likelihood and Bayesian mixture codes satisfy the notion of consistency as τ*→ 0. The main result may be used as a basis to easily establish similar consistency results for other closely related information theoretic regression criteria.