Computationally Intensive Nonlinear Regression Methods

Abstract

This article covers some nonlinear regression methods including classification and regression trees (CART), projection pursuit regression (PPR), multivariate adaptive regression splines (MARS), random forest (RF) and boosting regression trees (BRT). These methods can provide attractive alternatives to more common methods and provide a useful insight into the nature of the data and relationship between the variables. CART and MARS are partition-based methods since they choose the most relevant variables. PPR is a projection-based method since it projects all the input variables onto a unit vector before application of the nonlinear activation function. RF and BRT are ensemble methods using trees as the base models. The article provides theoretical and methodological insights into these methods, discusses their pros and cons from a practical point of view, and illustrates their features with the help of simulated examples.