Low-error digital hardware implementation of artificial neuron activation functions and their derivative

Abstract

In this paper we propose a low-error approximation of the sigmoid function and hyperbolic tangent, which are mainly used to activate the artificial neuron, based on the piecewise linear method. Here, the hyperbolic tangent is alternatively approximated by exploiting its mathematical relationship with the sigmoid function, showing better results. Special attention has been paid to study the minimum number of precision bits to achieve the convergence of a multi-layer perceptron network in finite arithmetic machine. All the approximation results show lower mean relative and absolute error than those reported in the state-of-the-art. Finally, the sigmoid digital implementation is discussed and assessed in terms of work frequency, complexity and error in comparison with the state-of-the-art.