Sparse Boosting with Correlation Based Penalty

Abstract

In high dimensional setting, componentwise L 2boosting method has been used to construct sparse model of high prediction, but it tends to select many ineffective variables. Several sparse boosting methods, such as, SparseL 2Boosting and Twin Boosting, have been proposed to improve the variable selection of L 2boosting algorithm. In this paper, we propose a new general sparse boosting method (GSBoosting). The relations are established between GSBoosting and other well known regularized variable selection methods in orthogonal linear model, such as adaptive Lasso, hard thresholds etc. Simulations results show that GSBoosting has good performance in both prediction and variable selection.