Event-based Real-time Hand Gesture Recognition using Spiking Neural Network

Abstract

Deep learning represents the state of the art in many machine learning and computer vision problem. The core of this technology is the analog neural network (ANN) composed of multiple convolution and pooling layers. Unfortunately, such system demands massive computational power thus consuming a lot of energy and therefore causing negative effect to the environment. On one hand, human brain is known to be much more energy efficient, so the spiking neural network (SNN) was created to replicate the brain activity in order to improve the energy efficiency of current deep learning model. On the other hand, the event-based domain based on neuromorphic sensor like event camera made huge progress since last few years and become more and more popular. The data signal flow in spiking neural network is a perfect fit for the output of event camera. Therefore, in this article we built a system based on the combination of event camera and SNN for the real-time hand gesture recognition. We also give an analysis to prove the energy efficiency of this technology compared to the ANN counterpart.