Moderate projection pursuit regression for multivariate response data

Abstract

Consider a regression problem with a multivariate response that we expect depends on a set of predictor variables in a non-linear way. A method designed for such problems is projection pursuit regression (PPR). PPR allows for very flexible modelling of the relationship between the response and the predictor variables, but it can have severe problems due to overfitting when there are few or noisy data. In this paper, I present a modified version of PPR called moderate PPR which is close to linear reduced rank regression. Substantial numerical evidence is presented to show that moderate PPR outperforms the ordinary PPR when the non-linearity is moderate and the data are few or noisy. Further, moderate PPR is robust in the sense that it rarely performs much worse than the linear reduced rank regression.