How Many Iterations are Sufficient for Efficient Semiparametric Estimation?

Abstract

Abstract. A common practice in obtaining an efficient semiparametric estimate is through iteratively maximizing the (penalized) full log‐likelihood w.r.t. its Euclidean parameter and functional nuisance parameter. A rigorous theoretical study of this semiparametric iterative estimation approach is the main purpose of this study. We first show that the grid search algorithm produces an initial estimate with the proper convergence rate. Our second contribution is to provide a formula in calculating the minimal number of iterationsk*needed to produce an efficient estimate. We discover that (i)k*depends on the convergence rates of the initial estimate and the nuisance functional estimate, and (ii)k*iterations are also sufficient for recovering the estimation sparsity in high dimensional data. The last contribution is the novel construction ofwhich does not require knowing the explicit expression of the efficient score function. The above general conclusions apply to semiparametric models estimated under various regularizations, for example, kernel or penalized estimation. As far as we are aware, this study provides a first general theoretical justification for the ‘one‐/two‐step iteration’ phenomena observed in the semiparametric literature.