SIBI (Sistem Bahasa Isyarat Indonesia) berbasis Machine Learning dan Computer Vision untuk Membantu Komunikasi Tuna Rungu dan Tuna Wicara

Abstract

The Indonesian Sign Language System (SIBI) is used to translate sign language into text or speech. SIBI helps improve communication between people using sign language and those who do not understand it. Unlike commonly used languages, SIBI sign language is less known to most people due to a lack of interest. To address this, an artificial intelligence-based application was developed, focusing on deep learning to recognize SIBI sign language hand movements in real-time. The model was created with 20 epochs, a batch size of 16, and a learning rate of 0.001. It consists of 13 layers, with the ReLU activation function used for the input layer, while the output layer uses Sigmoid. The ADAM optimizer was used to expedite the model creation process. The image dataset used had a size of 300x300 pixels. In the classification testing of the SIBI alphabet results in this study, it was tested using distance tests. The distance between the webcam and the SIBI language speaker was divided into two categories: 40 cm and 60 cm. For a 40-cm distance, an accuracy of 87.50% was obtained, and for a 60-cm distance, an accuracy of 79.17% was achieved. One limitation of this study is that two alphabets, J and Z, were not included in the dataset. This is because recognition of these two alphabets requires not only finger pattern recognition but also recognition of their gesture patterns.