How projection-pursuit learning works in high dimensions

Abstract

This paper addresses an important question in machine learning: What kinds of network architectures work better on what kinds of problems? A projection pursuit learning network has a very similar structure to a one hidden layer sigmoidal neural network. A general method based on a continuous version of projection pursuit regression is developed to show that projection pursuit regression works better on angular smooth functions than on Laplacian smooth functions. There exists a ridge function approximation scheme to avoid the curse of dimensionality for approximating some class of underlying functions.