Real-Time Target Tracking System With Spiking Neural Networks Implemented on Neuromorphic Chips

Abstract

Real-time target tracking is a usual task for humans despite the neural delays during the nervous system’s axonal transfer and neural processing. A plausible explanation is that the human brain employs predictive mechanisms to compensate for the delay. Inspired by the brain, this paper adopts a prediction network based on spiking neural networks (SNNs) to implement a real-time tracking task on a neuromorphic chip with low power consumption. The SNN-based prediction network outperforms the long short-term memory (LSTM) network on a small dataset and reduces 90% to 98% computations compared with LSTM. The quantized SNN-based network is deployed on a neuromorphic chip, and it takes 25ms and only 442 626nJ for a single prediction. The tracking performance of the system is also verified in real-life scenarios. Furthermore, the proposed real-time target tracking system can be easily ported to other neuromorphic platforms.