Temporal Dependent Local Learning for Deep Spiking Neural Networks

Abstract

Spiking neural networks (SNNs) are promising to replicate the efficiency of the brain by utilizing a paradigm of spike-based computation. Training a deep SNN is of great importance for solving practical tasks as well as discovering the fascinating capability of spike-based computation. The biologically plausible scheme of local learning motivates many approaches that enable training deep networks in an efficient parallel way. However, most of the existing spike-based local learning approaches show relatively low performances on challenging tasks. In this paper, we propose a new spike-based temporal dependent local learning (TDLL) algorithm, where each hidden layer of a deep SNN is independently trained with an auxiliary trainable spiking projection layer, and temporal dependency is fully employed to construct local errors for adjusting parameters. We examine the performance of the proposed TDLL with various networks on the MNIST, Fashion-MNIST, SVHN and CIFAR-10 datasets. Experimental results highlight that our method can scale up to larger networks, and more importantly, achieves relatively high accuracies on all benchmarks, which are even competitive with the ones obtained by global backpropagation-based methods. This work therefore contributes to providing an effective and efficient local learning method for deep SNNs, which could greatly benefit the developments of distributed neuromorphic computing.