Histogram of 3D Facets: A characteristic descriptor for hand gesture recognition

Abstract

The availability of 3D sensors has recently made it possible to capture depth maps in real time, which facilitates a variety of visual recognition tasks including hand gesture recognition. However, most existing methods simply treat depth information as intensities of gray images and ignore the strong 3D shape information. In this paper, we propose a novel characteristic descriptor, i.e., Histogram of 3D Facets (H3DF), to explicitly encode the 3D shape information from depth maps. We define a 3D facet as a 3D local support surface associated with each 3D cloud point. By robust coding and pooling 3D facets from a depth map, the proposed H3DF descriptor can effectively represent the 3D shapes and structures of various hand gestures. We evaluate the proposed descriptor on two challenging 3D datasets of hand gesture recognition. The recognition results in the context of both decimal digits and letters in American Sign Language (ASL) demonstrate that our approach significantly outperforms the state-of-the-art methods.