Orthogonal systems based stable estimator for non parametric regression problems

Abstract

. In this work, we study a univariate as well as a multivariate Jacobi polynomial-based estimator for solving the fundamental problem of non parametric (NP) regression. The proposed NP regression estimator aims to approximate a possibly very low regular regression function from the knowledge of its noised values given at some n random sampling points. We first study the univariate version of the proposed estimator, then extend this study to the general multivariate case with random sampling vectors drawn according to an unknown distribution. Our proposed estimator has the advantage of being stable with reasonable time complexity. Moreover, it has an optimal convergence rate under the weak condition that the regression function has a certain low Sobolev smoothness property. Although the proposed estimators are studied under the hypothesis that the random sampling vectors are supported on the d dimensional unit hypercube, by applying trivial transformations, the results of this work are still valid in the more general setting of any d dimensional hyperrectangle. Finally, the theoretical results of this work are illustrated by some numerical simulations on synthetic as well as real datasets.