Regularization Theory, Radial Basis Functions and Networks

Abstract

This paper consists of two parts. In the first part we consider the problem of learning from examples in the setting of the theory of the approximation of multivariate functions from sparse data. We first will show how an approach based on regularization theory leads to develop a family of approximation techniques, including Radial Basis Functions, and some tensor product and additive splines. Then we will show how this fairly classical approach has to be extended in order to cope with special features of the problem of learning of examples, such as high dimensionality and strong anisotropics. Furthermore, the same extension that leads from Radial Basis Functions (RBF) to Hyper Basis Functions (HBF) also leads from additive models to ridge approximation models, such as some forms of Projection Pursuit Regression.