P-SFA: Probability based Sigmoid Function Approximation for Low-complexity Hardware Implementation

Abstract

A probability-based sigmoid function approximation (P-SFA), which is based on piecewise linear function and neuron's values statistical probability distribution in each layer, is proposed to lower the complexity of neural network hardware implementation with only addition circuit. The sigmoid function is divided into three fixed regions, and the number of sub-regions with different sizes in each fixed region is adapted to neuron's values distribution in each layer to reduce the error between the sigmoid function and P-SFA function. The experimental results on FPGA show that the P-SFA function is efficient in terms of power and speed, and the recognition accuracies in DNN and CNN for MNIST with P-SFA are the highest among the state-of-the-art methods, up to 97.46% and 99.02% respectively.