A deep convolutional neural network model for hand gesture recognition in 2D near-infrared images

Abstract

The hand gesture recognition (HGR) process is one of the most vital components in human-computer interaction systems. Especially, these systems facilitate hearing-impaired people to communicate with society. This study aims to design a deep learning CNN model that can classify hand gestures effectively from the analysis of near-infrared and colored natural images. This paper proposes a new deep learning model based on CNN to recognize hand gestures improving recognition rate, training, and test time. The proposed approach includes data augmentation to boost training. Furthermore, five popular deep learning models are used for transfer learning, namely VGG16, VGG19, ResNet50, DenseNet121, and InceptionV3 and compared their results. These models are applied to recognize 10 different hand gestures for near-infrared images and 24 ASL hand gestures for colored natural images. The proposed CNN model, VGG16, VGG19, Resnet50, DenseNet121, and InceptionV3 models achieve recognition rates of 99.98%, 100%, 99.99%, 91.63%, 82.42% and 81.84%, respectively on near-infrared images. For colored natural ASL images, the models achieve recognition rates of 99.91%, 99.31%, 98.67%, 91.97%, 93.37%, and 93.21%, respectively. The proposed model achieves promising results spending the least amount of time.