Signal Denoising with Recurrent Spiking Neural Networks and Active Tuning

Abstract

Active Tuning is an optimization paradigm specifically designed to increase the robustness and generalization ability of temporal forward models like recurrent neural networks (RNNs). This work explores how the Active Tuning method can be used to optimize the internal dynamics of recurrent spiking neural networks (RSNNs). Active Tuning decouples the network from direct influence of the data stream and instead tunes its internal dynamics. This is based on the temporal gradient signals from propagating the error between outputs and observations backwards through time. Meanwhile, the network is running in a closed-loop prediction cycle, where the own output is used as the next input. As modern ANNs often demand excessive amounts of computational resources, spiking neural networks (SNNs) aim for the energy efficiency demonstrated by the human brain. This is accomplished by using an event-driven model inspired by the spiking behavior of biological neurons. Target of the Active Tuning optimization in RSNNs is the membrane potential of the neurons in the hidden layer. We show in two scenarios how RSNNs handle noisy inputs and that Active Tuning is a reliable method to increase their robustness as well as general prediction performance.