VLSI implementation of transcendental function hyperbolic tangent for deep neural network accelerators

Abstract

Extensive use of neural network applications prompted researchers to customize a design to speed up their computation based on ASIC implementation. The choice of activation function (AF) in a neural network is an essential requirement. Accurate design architecture of an AF in a digital network faces various challenges as these AF require more hardware resources because of its non-linear nature. This paper proposed an efficient approximation scheme for hyperbolic tangent (tanh) function which purely based on combinational design architecture. The approximation is based on mathematical analysis by considering maximum allowable error in a neural network. The results prove that the proposed combinational design of an AF is efficient in terms of area, power and delay with negligible accuracy loss on MNIST and CIFAR-10 benchmark datasets. Post synthesis results show that the proposed design area is reduced by 66% and delay is reduced by nearly 16% compared to state-of-the-art.