A bio-inspired approach modeling spiking neural networks of visual cortex for human action recognition

Abstract

Human visual system is an effective recognition one. Based on information processing mechanism of visual cortex, a bio-inspired approach for the human action recognition from video sequences is proposed in this paper. The approach gives a hierarchical architecture of the feedforward spiking neural network modeling two visual cortical areas: primary visual cortex (VI) and middle temporal area (MT), neurobiologically dedicated to motion processing. We augment the operator of motion information processing with center surround interaction to model the nonclassical receptive field inhibitory effect based on horizontal connection of spiking neurons in each cortical area. The weight function of lateral connection between VI and MT areas is built based on a previous study that explained direction selectivity in MT area by a linear combination of normalized VI direction-tuned signals. Moreover, we propose a three-dimensional (3D) Gabor filter to model the spatiotemporal direction and speed tuning properties of time-dependent receptive fields of the VI cells. The conductance-driven integrate-and-fire (IF) neuron model is used to obtain spike trains generated by the spiking neurons in two cortical areas. Finally, in order to analyze spike trains, we consider a characteristic of the neural code: mean motion map based on the mean firing rates of neurons in MT, called action code, as feature vector representing human actions. The approach is carried out on the Weizmann and KTH action database. Experimental results show that our approach has higher recognition performance and computational efficiency than other bio-inspired ones.