Efficient learning in spiking neural networks

Abstract

Spiking neural networks (SNNs) are a large class of neural model distinct from ‘classical’ continuous-valued networks such as multilayer perceptrons (MLPs). With event-driven dynamics and a continuous-time model in contrast to the discrete-time model of their classical counterparts, they offer interesting advantages in representational capacity and energy consumption. Spiking networks may also be more biologically plausible, offering more insights into neuroscience. However, developing models of learning for SNNs has historically proven challenging: as discrete-time systems, their dynamics are much more complex and they cannot benefit from the strong theoretical developments in MLPs such as convergence proofs and optimal gradient descent. Nor do they gain automatically from algorithmic improvements that have produced efficient matrix inversion and batch training methods. Most of the existing research has focused on the most well-studied learning mechanism in SNNs, spike-timing-dependent plasticity (STDP), and although there has been progress, there are also notable pathologies that have often been solved with a variety of ad-hoc techniques. While efforts have been made to map SNNs to classical convolutional neural networks (CNNs), these have not yet shown any decisive efficiency advantage over conventional CNNs. More promising research directions lie in the realm of pure spiking learning models that exploit the inherent temporal dynamics (and often leverage recurrency). Metrics are needed; one possibility would be a measure of total energy cost per unit reduction in error. This tutorial overview looks at existing techniques for learning in SNNs and offers some thoughts for future directions.