Hand Gesture Recognition Using 3D CNN and Computer Interfacing

Abstract

AbstractSince the 1970s, the field of gesture recognition and its applications has been at the centre of considerable research in human–computer interaction. Researchers have been able to construct strong models that can recognize gestures in real time thanks to recent advances in deep learning and computer vision, but they face hurdles when it comes to classifying gestures in variable lighting conditions. In this paper, we train a 3D convolutional neural network to recognize dynamic hand gestures in real time. Our focus is on ensuring that gesture recognition systems can perform well under varying light conditions. We use a huge training set consisting of numerous clips of people performing specific gestures in varying lighting conditions to train the model. We were able to attain an accuracy of 76.40% on the training set and 66.56% on the validation set with minimal pre-processing applied to the data set. The trained model was able to successfully recognize hand gestures recorded from a Webcam in real time. We were then able to use the model’s predictions to control video playback on the VLC media player such as increasing/decreasing volume and pausing video. These experimental results show the effectiveness and efficiency of the proposed framework to recognize gestures in both bright and dim lighting conditions.KeywordsDeep learningConvolutional neural networksPyTorch3D CNN