Nonlinear orthogonal series estimates for random design regression

Abstract

Let ( X, Y) be a pair of random variables with supp( X)⊆[0,1] and E Y 2<∞. Let m be the corresponding regression function. Estimation of m from i.i.d. data is considered. The L 2 error with integration with respect to the design measure μ (i.e., the distribution of X) is used as an error criterion. Estimates are constructed by estimating the coefficients of an orthonormal expansion of the regression function. This orthonormal expansion is done with respect to a family of piecewise polynomials, which are orthonormal in L 2( μ n ), where μ n denotes the empirical design measure. It is shown that the estimates are weakly and strongly consistent for every distribution of ( X, Y). Furthermore, the estimates behave nearly as well as an ideal (but not applicable) estimate constructed by fitting a piecewise polynomial to the data, where the partition of the piecewise polynomial is chosen optimally for the underlying distribution. This implies e.g., that the estimates achieve up to a logarithmic factor the rate n −2 p/(2 p+1) , if the underlying regression function is piecewise p-smooth, although their definition depends neither on the smoothness nor on the location of the discontinuities of the regression function.