Human hand gesture recognition with convolutional neural networks for K-12 double-teachers instruction mode classroom

Abstract

Infrared hand gesture images can capture the human accurate gesture information in the weak illumination environment. In this article, we propose a novel hand gesture recognition model with convolutional neural networks for the human behavior analysis in “double teachers” classroom instruction and learning scenario. The recognized instructors’ hand gestures can be utilized to analyze teachers’ non-verbal behaviors which attract learners’ attention as well as enhance their learning outcomes. To extract the features of the infrared hand gesture images, a non-linear neural networks is constructed with four convolution layers. It aims to learn teachers’ pointing gestures and provides with the accurate gestures information. To achieve robust hand gesture recognition, the supervised convolutional neural network is constructed with three convolution layers to extract the infrared hand images features. Experiment results demonstrate that the proposed method outperforms the state-of-the-art methods in the term of the recognition accuracy and classification error. From the comparing experiments, the results prove that the developed approach can estimate the hand gesture with more than 92% recognition ratio.