Real-time gesture recognition using bio inspired 3D vision sensor

Abstract

This paper presents a sensor system for spatiotemporal motion analysis in a 4D representation and real-time pattern analysis based on dynamic stereo vision. The system comprises a dynamic stereo vision sensor, which is sensitive to temporal contrast and therefore reacts to motion such that pixels affected by the scene changes autonomously generate events. The data are completely asynchronous and therefore have ultra-high temporal resolution (1μs@1000 lux) and wide dynamic range (over 120dB). Using this sensor body motion analysis and tracking can be efficiently performed because of the continuous stream of data, which accurately capture changes even in difficult illumination conditions. For dance pattern recognition we use a machine learning method based on the Hidden Markov Model, for a realtime recognition of activities. The whole system has been evaluated on a dance choreography consisting of eight different activities and a training set of 430 recorded activities performed by 15 different persons. Preliminary results show that the proposed system reaches an average recognition rate of 94%.