Hand gesture recognition of Arabic numbers using leap motion via deterministic learning

Abstract

The aim of this study is to develop a new method for hand gesture recognition using Leap Motion via deterministic learning. Efficient and accurate extraction and representation of gesture features are achieved. The recognition approach consists of two stages: a training stage and a recognition stage. In the training stage, hand gesture features representing hand motion dynamics, including spatial position and direction of fingers, are derived from Leap Motion. Hand motion dynamics underlying motion patterns of different gestures which represent Arabic numbers (0–9) are locally accurately modeled and approximated by radial basis function (RBF) neural networks. The obtained knowledge of approximated hand motion dynamics is stored in constant RBF networks. In the recognition stage, a bank of dynamical estimators is constructed for all the training patterns. Prior knowledge of hand motion dynamics represented by the constant RBF networks is embedded in the estimators. By comparing the set of estimators with a test gesture pattern to be recognized, a set of recognition errors are generated. The average Li norms of the errors are taken as the recognition measure between the dynamics of the training gesture patterns and the dynamics of the test gesture pattern according to the smallest error principle. Finally, experiments are carried out to demonstrate the high recognition performance of the proposed method. By using the 2-fold and 10-fold cross-validation styles, the correct recognition rates are reported to be 94.2% and 95.1%, respectively.