Abstract
Event-driven neuromorphic spiking sensors such as the silicon retina and the silicon cochlea encode the external sensory stimuli as asynchronous streams of spikes across different channels or pixels. Combining state-of-art deep neural networks with the asynchronous outputs of these sensors has produced encouraging results on some datasets but remains challenging. While the lack of effective spiking networks to process the spike streams is one reason, the other reason is that the pre-processing methods required to convert the spike streams to frame-based features needed for the deep networks still require further investigation. This work investigates the effectiveness of synchronous and asynchronous frame-based features generated using spike count and constant event binning in combination with the use of a recurrent neural network for solving a classification task using N-TIDIGITS18 dataset. This spike-based dataset consists of recordings from the Dynamic Audio Sensor, a spiking silicon cochlea sensor, in response to the TIDIGITS audio dataset. We also propose a new pre-processing method which applies an exponential kernel on the output cochlea spikes so that the interspike timing information is better preserved. The results from the N-TIDIGITS18 dataset show that the exponential features perform better than the spike count features, with over 91% accuracy on the digit classification task. This accuracy corresponds to an improvement of at least 2.5% over the use of spike count features, establishing a new state of the art for this dataset.
Highlights
The event processing methods for the asynchronous spikes of event-based sensors such as the Dynamic Vision Sensor (DVS) (Lichtsteiner et al, 2008; Berner et al, 2013; Posch et al, 2014; Yang et al, 2015) and the Dynamic Audio Sensor (DAS) (Liu et al, 2014; Yang et al, 2016) fall roughly into two categories: either by the use of neural network methods or machine learning algorithms
We present the network accuracy results of the different pre– processing methods on the audio classification tasks based on the N-TIDIGITS18 dataset when these features are presented to the different recurrent models
We performed a comparative study of the performance accuracy of a gated recurrent neural network that processes either the raw audio spikes or framed features extracted by different spike processing methods
Summary
The event processing methods for the asynchronous spikes of event-based sensors such as the Dynamic Vision Sensor (DVS) (Lichtsteiner et al, 2008; Berner et al, 2013; Posch et al, 2014; Yang et al, 2015) and the Dynamic Audio Sensor (DAS) (Liu et al, 2014; Yang et al, 2016) fall roughly into two categories: either by the use of neural network methods or machine learning algorithms. By using conversion methods that convert pre-trained standard deep networks into equivalent-accurate spiking networks (Diehl et al, 2015; Rueckauer et al, 2017) or by using the training methods from deep learning on networks that capture the underlying parameters of the spiking neuron (O’Connor et al, 2013; Stromatias et al, 2015), we are starting to see spiking deep networks that can be competitive with the standard deep networks
Sign-in/Register to view highlights & summary Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
