Analyzing different high speed adder architecture for Neural Networks

Abstract

The first neural network model which was developed for image recognition application consisted of simple perceptrons. It had input, processing unit, and a single output. Neural networks which are used in today’s world consist of many complex MAC (Multiply and Accumulate) units. Be it the simple pattern recognition neural network model or complex models used for autonomous driving applications; adders are used for computing the activation point of neurons. Some adders offer better performance at the cost of area and power while some offer better power at the cost of performance. So, choosing the right type of adder architecture based upon the application becomes a very important criterion when we are trying to develop an inference engine for the neural network in hardware. To determine weight or activation point of a neuron, typically, float32 or float64 number representation is used. Float64 offers better accuracy than float32 but the drawback of using float64 is that it requires huge computation power. So, in this manuscript we compare different high-speed adder topologies, then discuss the implementation of an optimized 64-bit conditional sum and carry select adder that can be used to implement Deep Neural Network with float64 number representation. Analysis between different adder architecture is performed using Synopsys Design Compiler with 45nm Toshiba library for three different metrics: Timing, Area, and Power.