Smoothing Spline Nonlinear Nonparametric Regression Models

Abstract

Almost all of the current nonparametric regression methods, such as smoothing splines, generalized additive models, and varyingcoefficients models, assume a linear relationship when nonparametric functions are regarded as parameters. In this article we propose a general class of smoothing spline nonlinear nonparametric models that allow nonparametric functions to act nonlinearly. They arise in many fields as either theoretical or empirical models. Our new estimation methods are based on an extension of the Gauss–Newton method to infinite-dimensional spaces and the backfitting procedure. We extend the generalized cross-validation and generalized maximum likelihood methods to estimate smoothing parameters. We establish connections between some nonlinear nonparametric models and nonlinear mixed-effects models. We derive approximate Bayesian confidence intervals for inference. We illustrate the methods with an application to term structure of interest rates and conduct simulations to evaluate the finite-sample performance of our methods.