Thresholding least-squares inference in high-dimensional regression models

Abstract

We propose a thresholding least-squares method of inference for high-dimensional regression models when the number of parameters, $p$, tends to infinity with the sample size, $n$. Extending the asymptotic behavior of the F-test in high dimensions, we establish the oracle property of the thresholding least-squares estimator when $p=o(n)$. We propose two automatic selection procedures for the thresholding parameter using Scheffe and Bonferroni methods. We show that, under additional regularity conditions, the results continue to hold even if $p=\exp(o(n))$. Lastly, we show that, if properly centered, the residual-bootstrap estimator of the distribution of thresholding least-squares estimator is consistent, while a naive bootstrap estimator is inconsistent. In an intensive simulation study, we assess the finite sample properties of the proposed methods for various sample sizes and model parameters. The analysis of a real world data set illustrates an application of the methods in practice.