On improvability of model averaging by penalized model selection

Abstract

We propose MMAe, a modified MMA based on penalized Mallow's Cp. A weighted elastic net penalty is added to handle the inevitable collinearity among models, which is beneficial to high‐dimensional data modelling. We proved the sparsity, the asymptotic optimality of its weight solution and also proved that its candidate model set can be exponentially enlarged under Gaussian noises. We further proved that an MMAe adjusted by generalized cross validation (GCV) has an asymptotically lower risk than MMA under more relaxed conditions. Our approach can be implemented efficiently by convex optimization algorithms. In simulation and real‐life analysis, MMAe achieves higher prediction accuracy compared with other methods.