Indonesian Sign Language Fingerspelling Recognition using Vision-based Machine Learning

Abstract

Sign language is a method of communication using hand gestures that are usually used by Deaf people. In Indonesia, there are 2 types of sign language, namely SIBI and BISINDO. However, in everyday life, BISINDO is more often used. Communication gaps often occur between Deaf people and hearing people. So that we need media that can bridge their communication. one of the technologies that can be used is SLR (Sign Language Recognition). SLR itself has various kinds of approaches, one of which is a vision-based SLR. Vision-based SLR has an advantage, such as not requiring a special device attached to the hand, but simply making gestures with bare hands in front of the camera. In this study, we created a machine learning model with a vision-based SLR approach. The model we created was using the CNN (Convolutional Neural Network) architecture. The CNN model was trained and tested on the BISINDO alphabet (A-Z) dataset that we created on our own. This model achieves an accuracy of 99.28% on validation accuracy, 98.57% on testing accuracy, and 98.07% on real-time testing accuracy.