Nonlinear partial least squares

Abstract

We propose a new nonparametric regression method for high-dimensional data, nonlinear partial least squares (NLPLS), which is motivated by projection-based regression methods, e.g. PLS, projection pursuit regression and feedforward neural networks. The model takes the form of a composition of two functions. The first function in the composition projects the predictor variables onto a lower-dimensional curve or surface yielding scores, and the second predicts the response variable from the scores. We implement NLPLS with feedforward neural networks. NLPLS often will produce a more parsimonious model (fewer score vectors) than projection-based methods. We extend the model to multiple response variables and discuss situations when multiple response variables should be modeled simultaneously and when they should be modeled with separate regressions. We provide empirical results that evaluate the performances of NLPLS, projection pursuit, and neural networks on response variable predictions and robustness to starting values.