Neural Coding in Spiking Neural Networks: A Comparative Study for Robust Neuromorphic Systems.

Mohammed E Fouda,Wenzhe Guo,Ahmed M Eltawil,Khaled Nabil Salama

doi:10.3389/fnins.2021.638474

Abstract

Various hypotheses of information representation in brain, referred to as neural codes, have been proposed to explain the information transmission between neurons. Neural coding plays an essential role in enabling the brain-inspired spiking neural networks (SNNs) to perform different tasks. To search for the best coding scheme, we performed an extensive comparative study on the impact and performance of four important neural coding schemes, namely, rate coding, time-to-first spike (TTFS) coding, phase coding, and burst coding. The comparative study was carried out using a biological 2-layer SNN trained with an unsupervised spike-timing-dependent plasticity (STDP) algorithm. Various aspects of network performance were considered, including classification accuracy, processing latency, synaptic operations (SOPs), hardware implementation, network compression efficacy, input and synaptic noise resilience, and synaptic fault tolerance. The classification tasks on Modified National Institute of Standards and Technology (MNIST) and Fashion-MNIST datasets were applied in our study. For hardware implementation, area and power consumption were estimated for these coding schemes, and the network compression efficacy was analyzed using pruning and quantization techniques. Different types of input noise and noise variations in the datasets were considered and applied. Furthermore, the robustness of each coding scheme to the non-ideality-induced synaptic noise and fault in analog neuromorphic systems was studied and compared. Our results show that TTFS coding is the best choice in achieving the highest computational performance with very low hardware implementation overhead. TTFS coding requires 4x/7.5x lower processing latency and 3.5x/6.5x fewer SOPs than rate coding during the training/inference process. Phase coding is the most resilient scheme to input noise. Burst coding offers the highest network compression efficacy and the best overall robustness to hardware non-idealities for both training and inference processes. The study presented in this paper reveals the design space created by the choice of each coding scheme, allowing designers to frame each scheme in terms of its strength and weakness given a designs’ constraints and considerations in neuromorphic systems.

Highlights

Artificial neural networks (ANNs) have achieved state-of-theart results in various applications ranging from computer vision (Krizhevsky et al, 2017), speech recognition (Graves and Schmidhuber, 2005), to natural language processing (Collobert et al, 2011)
We investigated different neural coding schemes in an unsupervised Spiking neural networks (SNNs) from various aspects during training and inference phases, including classification performance, computational performance, hardware implementation, network compression efficacy, noise resilience, and fault tolerance
The classification and computational performance were analyzed in terms of accuracy, latency, and synaptic operations (SOPs)

Summary

INTRODUCTION

Artificial neural networks (ANNs) have achieved state-of-theart results in various applications ranging from computer vision (Krizhevsky et al, 2017), speech recognition (Graves and Schmidhuber, 2005), to natural language processing (Collobert et al, 2011). Various neuromorphic computing systems built on SNNs have been proposed to solve the bottleneck posed by the traditional Von Neumann computing systems (Furber et al, 2014; Davies et al, 2018; Frenkel et al, 2019) Their massive parallelism, asynchronous event-driven operations, and distributed memory provide huge potential in accelerating information processing and reducing energy consumption in many applications. In this work, we present a comparative study of the impact and performance of different neural coding schemes from various aspects of design during both the training and inference processes. Rate coding, TTFS coding, phase coding, and burst coding, are chosen for their importance in understanding the underlying information encoding mechanism and their important roles in many parts of nervous systems.

AND METHODS

RESULTS AND DISCUSSION

CONCLUSION