Construction of feedforward neural networks with simple architectures and approximation abilities

Abstract

The universal approximation property of feedforward neural networks (FNNs) is the basis for all FNNs applications. In almost all existing FNN approximation studies, it is always assumed that the activation function of the network satisfies certain conditions, such as sigmoid or bounded continuity. This paper focuses on building simple architecture FNNs with approximation ability by constructing an activation function. First, for any continuous function defined on [0, 1] and an arbitrary accuracy ϵ > 0, an FNN is constructed, which has only one neuron and a fixed weight 1 and can approximate the function with an accuracy of ϵ. Thereafter, we study the representation of FNNs for polynomial functions by constructing a proper activation function. It is proved that any algebraic polynomial with the degree n can be represented by an FNN. Further, a piecewise function is constructed as an activation function of an FNN such that the FNN represents the famous Bernstein polynomials.