An Efficient Hardware Implementation of Feed-Forward Neural Networks*

Abstract

This paper proposes a new way of digital hardware imple- mentation of nonlinear activation functions in feed-forward neural net- works. The basic idea of this new realization is that the nonlinear functions can be implemented using a matrix-vector multiplication. Recently a new approach was proposed for the realization of matrix-vector mul- tiplications which approach can also be applied for implementing the nonlinear functions if the nonlinear functions are approximated by sim- ple basis functions. The paper proposes to use B-spline basis functions to the approximate nonlinear sigmoidal functions, it shows that this ap proximation fulfills the general requirements on the activation functions, presents the details of the proposed hardware implementation, and gives a summary of an extensive study about the effects of B-spline nonlin- ear function realization on the size and the trainability of feed-forward neural networks.KeywordsMean Square ErrorActivation FunctionHide NeuronOrder SplineNonlinear Activation FunctionThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.