Sparse covariance thresholding for high-dimensional variable selection

Abstract

In high dimensions, variable selection methods such as the lasso are of- ten limited by excessive variability and rank deficiency of the sample covariance matrix. Covariance sparsity is a natural phenomenon in such high-dimensional ap- plications as microarray analysis, image processing, etc., in which a large number of predictors are independent or weakly correlated. In this paper, we propose the covariance-thresholded lasso, a new class of regression methods that can utilize covariance sparsity to improve variable selection. We establish theoretical results, under the random design setting, that relate covariance sparsity to variable selec- tion. Data and simulations indicate that our method can be useful in improving variable selection performances.