FPGA Implementation and Comparison of Sigmoid and Hyperbolic Tangent Activation Functions in an Artificial Neural Network

Abstract

An artificial neural network(ANN) is a type of computing system that mimics the functioning of the neural networks in a biological brain. In this paper, an ANN has been simulated on an Field Programmable Gate Array (FPGA) for the application of handwritten digit recognition using the sigmoid and hyperbolic tangent(tanh) activation functions. Both software and hardware simulations have been carried out, using python and Verilog HDL respectively. The two activation functions were implemented on an Artix-7 FPGA. A comparison has been made between the sigmoid and tanh activation functions based on speed, accuracy, and hardware required, and it has been inferred that the tanh function is best for the application of handwritten digit recognition as it has 3% higher accuracy and uses 5 less LUTs than the sigmoid activation function.