Abstract
Despite the success of Deep Neural Networks—a type of Artificial Neural Network (ANN)—in problem domains such as image recognition and speech processing, the energy and processing demands during both training and deployment are growing at an unsustainable rate in the push for greater accuracy. There is a temptation to look for radical new approaches to these applications, and one such approach is the notion that replacing the abstract neuron used in most deep networks with a more biologically-plausible spiking neuron might lead to savings in both energy and resource cost. The most common spiking networks use rate-coded neurons for which a simple translation from a pre-trained ANN to an equivalent spike-based network (SNN) is readily achievable. But does the spike-based network offer an improvement of energy efficiency over the original deep network? In this work, we consider the digital implementations of the core steps in an ANN and the equivalent steps in a rate-coded spiking neural network. We establish a simple method of assessing the relative advantages of rate-based spike encoding over a conventional ANN model. Assuming identical underlying silicon technology we show that most rate-coded spiking network implementations will not be more energy or resource efficient than the original ANN, concluding that more imaginative uses of spikes are required to displace conventional ANNs as the dominant computing framework for neural computation.
Highlights
By comparing the digital implementations of the core computations for both the conventional neural network and the spike-based equivalent, under the assumption of identical silicon substrates, we show that most rate-coded spiking network implementations will not compete with the Artificial Neural Network (ANN)
We present a brief overview of the key processes in a deep neural networks, present two implementations of these processes: one as a conventional ANN and the other as a spiking neural network
The argument presented above focuses on the fundamental computation in the ANN and SNN cases
Summary
Within the broad field of Artificial Neural Networks (ANNs) the development of Deep Neural Networks (DNNs) over the last decade has made a number of significant applications possible (Graves et al, 2013; Barsoum et al, 2016; Howard et al, 2017; Vinyals et al, 2019), elevating the neural network from a laboratory-bound curiosity to a dependable tool for real world applications in the areas of image and speech recognition (Graves et al, 2013; Kepuska and Bohouta, 2018; Brown et al, 2020). A digital implementation of neural networks it is normal to map a group of neurons to a single processing module or pipeline, the neural state being updated sequentially using shared resources This is appropriate when the speed of execution of the module (of the order of GigaHertz in a modern process) greatly exceeds the arrival rate of incoming activations (of the order of one every one tenth of a millisecond), allowing the efficiency gains that can be obtained from the time-domain multiplexing of shared hardware. Each incoming spike must trigger the reading of the same associative memory defined for the ANN, using the ID Xi as the key, retrieving the list of synaptic weights Wij and their corresponding target neuron IDs, Yj. Each entry in the list will trigger the reading of the neural state UJ of a target neuron, given its identifier YJ. If the neuron state has surpassed the predefined threshold an output spike is generated and the neuron state value is reset (either to an absolute value or by a fixed amount) before it is written back (Rueckauer et al, 2017)
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
