Function approximation with zonal function networks with activation functions analogous to the rectified linear unit functions

Abstract

A zonal function (ZF) network on the q dimensional sphere Sq is a network of the form x↦∑k=1nakϕ(x⋅xk) where ϕ:[−1,1]→R is the activation function, xk∈Sq are the centers, and ak∈R. While the approximation properties of such networks are well studied in the context of positive definite activation functions, recent interest in deep and shallow networks motivate the study of activation functions of the form ϕ(t)=|t|, which are not positive definite. In this paper, we define an appropriate smoothness class and establish approximation properties of such networks for functions in this class. The centers can be chosen independently of the target function, and the coefficients are linear combinations of the training data. The constructions preserve rotational symmetries.