Ultra-low latency spiking neural networks with spatio-temporal compression and synaptic convolutional block

Abstract

Spiking neural networks (SNNs), as one of the brain-inspired models, has spatio-temporal information processing capability, low power feature, and high biological plausibility. The effective spatio-temporal feature makes it suitable for event streams classification. However, neuromorphic datasets, such as N-MNIST, CIFAR10-DVS, DVS128-gesture, need to aggregate individual events into frames with a new higher temporal resolution for event stream classification, which causes high training and inference latency. In this work, we proposed a spatio-temporal compression method to aggregate individual events into a few time steps of synaptic current to reduce the training and inference latency. To keep the accuracy of SNNs under high compression ratios, we also proposed a synaptic convolutional block to balance the dramatic changes between adjacent time steps. And multi-threshold Leaky Integrate-and-Fire (LIF) models with learnable membrane time constants are introduced to increase their information processing capability. We evaluate the proposed method for event stream classification tasks on neuromorphic N-MNIST, CIFAR10-DVS, and DVS128 gesture datasets. The experiment results show that our proposed method outperforms the state-of-the-art accuracy on nearly all datasets, using fewer time steps.