Constructive function approximation by neural networks with optimized activation functions and fixed weights

Abstract

Our purpose in this paper is to construct three types of single-hidden layer feed-forward neural networks (FNNs) with optimized piecewise linear activation functions and fixed weights and to present the ideal upper and lower bound estimations on the approximation accuracy of the FNNs, for continuous function defined on bounded intervals. We also prove these three types of single-hidden layer FNNs can interpolate any bounded and measurable functions. Our approach compared with existing methods does not require training. Our conclusions not only uncover the inherent properties of approximation of the FNNs, but also reveal the latent relationship among the precision of approximation, the number of hidden units and the smoothness of the target function. Finally, we demonstrate some numerical results that show good agreement with theory.