Sparse semiparametric efficient estimation in high-dimensional linear regression models

Abstract

Linear regression estimation 问题 with high-dimensional covariates has been extensively studied in the literature. However, how to integrate the efficiency consideration into the high-dimensional estimation with the unknown error density is still an unsolved but challenging 问题. Parametric estimators such as ordinary least squares estimators will suffer efficiency loss with non-Gaussian error density, while the maximum likelihood estimation cannot be directly applied due to the unknown error density. In this paper, we propose a novel sparse semiparametric efficient estimation method for the high-dimensional linear regression with the unknown error density via penalized estimating equations. We prove that the new estimator is asymptotically as efficient as the oracle MLE (maximum likelihood estimator) in the ultra-high-dimensional setting with the unknown error density and thus is more efficient than the traditional penalized least squares estimator for non-Gaussian error densities. In addition, we demonstrate that several popularly used high-dimensional regression estimators are special cases of ours. Extensive simulation studies and the empirical analysis of a real data set are conducted which demonstrate the effectiveness of the proposed procedure and its superior performance compared with least squares based methods.