Relative Cost Based Model Selection for Sparse High-Dimensional Linear Regression Models

Abstract

In this paper, we propose a novel model selection method named multi-beta-test (MBT) for the sparse high-dimensional linear regression model. The estimation of the correct subset in the linear regression problem is formulated as a series of hypothesis tests where the test statistic is based on the relative least-squares cost of successive parameter models. The performance of MBT is compared to existing model selection methods for high-dimensional parameter space such as extended Bayesian information criterion (EBIC), extended Fisher Information criterion (EFIC), residual ratio thresholding (RRT) and orthogonal matching pursuit (OMP) with a priori knowledge of the sparsity. Simulation results indicate that the performance of MBT in identifying the true support set surpasses that of EBIC, EFIC and RRT in certain regions of the considered parameter settings.