Conditional regression for single-index models

Abstract

The single-index model is a statistical model for intrinsic regression where responses are assumed to depend on a single yet unknown linear combination of the predictors, allowing to express the regression function as E[Y|X]=f(⟨v,X⟩) for some unknown index vector v and link function f. Conditional methods provide a simple and effective approach to estimate v by averaging moments of X conditioned on Y, but depend on parameters whose optimal choice is unknown and do not provide generalization bounds on f. In this paper we propose a new conditional method converging at n rate under an explicit parameter characterization. Moreover, we prove that polynomial partitioning estimates achieve the 1-dimensional min-max rate for regression of Hölder functions when combined to any n-convergent index estimator. Overall this yields an estimator for dimension reduction and regression of single-index models that attains statistical optimality in quasilinear time.