Vision-based hand gesture recognition using deep learning for the interpretation of sign language

Abstract

Hand gestures have been the key component of communication since the beginning of an era. The hand gestures are the foundation of sign language, which is a visual form of communication. In this paper, a deep learning based convolutional neural network (CNN) model is specifically designed for the recognition of gesture-based sign language. This model has a compact representation that achieves better classification accuracy with a fewer number of model parameters over the other existing architectures of CNN. In order to evaluate the efficacy of this model, VGG-11 and VGG-16 have also been trained and tested in this work. To evaluate the performance, 2 datasets have been considered. First, in this work, a large collection of Indian sign language (ISL) gestures consisting of 2150 images is collected using RGB camera, and second, a publicly available American sign language (ASL) dataset is used. The highest accuracy of 99.96% and 100% is obtained by the proposed model for ISL and ASL datasets respectively. The performance of the proposed system, VGG-11, and VGG-16 are experimentally evaluated and compared with the existing state-of-art approaches. In addition to accuracy, other efficiency indices have been also used to ascertain the robustness of the proposed work. The findings indicate that the proposed model outperforms the existing techniques as it has the potential to classify maximum gestures with a minimal rate of error. The model is also tested with the augmented data and is found as invariant to rotation and scaling transformation.