Indonesian Sign Research Articles

Recognizing Indonesian sign language (Bisindo) gesture in complex backgrounds

Sign language, particularly Indonesian sign language (Bisindo), is vital for deaf individuals, but learning it is challenging. This study aims to develop an automated Bisindo recognition system suitable for diverse backgrounds. Previous research focused on greenscreen backgrounds and struggled with natural or complex backgrounds. To address this problem, the study proposes using Faster region-based convolutional neural networks (RCNN) and YOLOv5 for hand and face detection, MobileNetV2 for feature extraction, and long short-term memory (LSTM) for classification. The system is also designed to focus on computational efficiency. YOLOv5 model achieves the best result with a sentence accuracy (SAcc) of 49.29% and a word error rate (WER) of 16.42%, with a computational time of 0.0188 seconds, surpassing the baseline model. Additionally, the system achieved a SacreBLEU score of 67.77%, demonstrating its effectiveness in Bisindo recognition across various backgrounds. This research improves accessibility for deaf individuals by advancing automated sign language recognition technology.

Implementasi Algoritma You Only Look Once (YOLO) untuk Mendeteksi Bahasa Isyarat SIBI

The Indonesian Sign Language System (SIBI) is a translator of sign language into text or speech. This research aims to bridge communication between ordinary people and speech impaired people through the introduction of SIBI sign language using the YOLO algorithm. This research uses 24 alphabets which are divided into 4 groups, where each alphabet has 20 image data which is divided into 70% train data, 25% valid data, and 5% test data. The train data was then added with augmented data from Roboflow which was then carried out using a training process using a batch number of 16 and epochs of 100. The results of the research show that the YOLO algorithm can detect SIBI sign language alphabet gestures using confusion matrix testing and achieve quite good performance, as shown by the results F1 Score: Group 1 was 90.90%, Group 2 was 97.1%, Group 3 was 90.90%, and Group 4 was 83.8%. Other factors such as hand size, lighting conditions, and variations in data position also affect detection accuracy. A limitation in this research is that the alphabets J and Z were not included because these two alphabets not only use shape patterns, but also gesture patterns.

Implementation of Sibi Sign Language Realtime Detection Program (Case Studi At Sekolah Luar Biasa Negeri 1 Tabanan)

Indonesian deaf people utilize SIBI to communicate using spoken words, gestures, facial expressions, and body language. SIBI, certified for Special Schools (SLB), helps deaf pupils communicate. This project implements SIBI (Indonesian Sign Language System) a real-time detection algorithm at Sekolah Luar Biasa Negeri 1 Tabanan using image processing and YoloV8 ultralytics deep learning. The program trains a sign language gesture detection model on Google Colab's GPU. The SIBI sign language images were used to train a YoloV8 object detection model. The camera captures movements, which the YoloV8 algorithm trained on SIBI gesture data processes. It can recognize gestures in real time and generate text to non-sign language users. The dataset has 107 class vocabulary and 7 class affix prefixes for complete gesture recognition. Shirt color, room brightness, and webcam quality affect detection rates. Optimal detection accuracy is 87.74% and subpar 58.02%. Despite these limitations, the strategy helps deaf students communicate more effectively with non-sign language speakers. This program improves inclusivity and communication in schools, making learning easier for hearing-impaired pupils. This work provides a reliable and quick sign language identification system to help deaf educators and caregivers with daily interactions and education.

Identification of Indonesian Sign Language System Using Deep Learning in Yolo-based

Identification of Indonesian Sign Language System Using Deep Learning in Yolo-based

Temporal Action Segmentation in Sign Language System for Bahasa Indonesia (SIBI) Videos Using Optical Flow-Based Approach

Sign language (SL) is vital in fostering communication for the deaf and hard-of-hearing communities. Continuous Sign Language Translation (CSLT) is a work that translates sign language into spoken language. CSLT translation is done by changing continuous forms into isolated signs. Segmenting morpheme signs from phrase signs has several challenges, such as the availability of annotated datasets and the complexity of continuous gesture movements. The Indonesian Sign Language (SIBI) system follows Indonesian grammatical norms, including word formation, in contrast to other sign languages with rules derived from their spoken language. In SIBI, a word can consist of a root word and an affix word. Therefore, temporal action segmentation in SIBI is important to reconstruct the results of translating each sign into spoken Indonesian sentences. This research uses an optical flow approach to segment temporal actions in SIBI videos. Optical flow methods that calculate changes in intensity between adjacent frames can be used to determine the occurrence of sign movement or vice versa to determine the delay between sign movements. The absence of intensity differences between the two frames indicates the boundary between sign gestures. This study tested the use of dense optical flow on videos containing SIBI sentences taken from 3 signers. Evaluation is done on several parameters in the dense optical flow algorithm, such as threshold size, PyrScale, and WinSize, to obtain the best accuracy. This paper shows that the optical flow algorithm successfully performs segmentation, as measured by Perf and F1r. The experimental results showed that the highest Perf and F1r yields were 0.8298 and 0.8524, respectively.

Android Application Prototype of Basic BISINDO Introduction and Practice for General People

One of the sign languages used in Indonesia is BISINDO (Indonesian Sign Language) created by the deaf. BISINDO is like the mother tongue for the deaf in Indonesia, although it is not as popular as SIBI (Indonesian Sign System). Therefore, the researchers propose to develop an Android-based prototype to promote BISINDO to others and help the deaf in daily communication. This prototype was built using coding in React Native framework and code editor of Visual Studio Code, then it was compiled to an Android Smartphone. The features include BISINDO's basic dictionary, translation from hand gesture to text, voice to sign language, text to sign language, and voice to text. This prototype is expected to introduce basic BISINDO vocabulary and practice using BISINDO. This prototype ia expected to be developed further into a more useful application, especially to assist communication between deaf people and the general public, but not as a substitution of BISINDO teachers.

Comparative Analysis of CNN Architectures for SIBI Image Classification

The classification of images from the Indonesian Sign Language System (SIBI) using VGG16, ResNet50, Inception, Xception, and MobileNetV2 Convolutional Neural Network (CNN) architectures is evaluated in this paper. With Google Colab Pro, a 224 × 224-pixel picture dataset was used for the study. A five-stage technique consisting of Dataset Collection, Dataset Preprocessing, Model Design, Model Training, and Model Testing was applied. Performance evaluation focused on accuracy, precision, recall, and F1-Score. The results identified VGG16 as the top-performing model with an accuracy of 99.60% and an equivalent F1-Score, followed closely by ResNet50 with nearly similar performance. Inception, XCeption, and MobileNetV2 demonstrated balanced performance but with lower accuracy. This study sheds light on the best CNN models to choose for SIBI image classification, and it makes recommendations for further research that include using sophisticated data augmentation methods, investigating novel CNN architectures, and putting the models to practical use.

Object Detection of BISINDO Sign Language Letters Using Residual Network

Indonesian Sign Language or BISINDO is an alternative language used by people who suffer from disabilities, especially those who have hearing impairments. This language grew and developed from the deaf community, so its use is based on the visual aspect. This research aims to apply Residual Networks to detect objects in the context of Bisindo Letter Sign Language, with the hope of increasing accuracy and efficiency in letter recognition. Object detection goes through 2 stages, namely feature extraction and model training. ResNet is a type of Convolutional Neural Network (CNN) architecture that utilizes models that have been previously trained, so it can save the time required in the model development process. In this research, Residual Network (ResNet) was used for feature extraction to recognize important aspects in the Bisindo letter sign image, such as hand position, finger shape characteristics, and direction of movement. The research results show that the new dataset used as training data and test data has a fairly good ability to detect with a division of 70% train set, 20% valid set and 10% test set with size 640x640 with 300 epochs for the training model.

Penerapan Metode Random Forest dalam Klasifikasi Huruf BISINDO dengan Menggunakan Ekstraksi Fitur Warna dan Bentuk

Digital image processing is a field of study that focuses on how an image can be formed, processed, and analyzed to generate useful information for humans. In this research, the utilization of digital images is implemented to classify BISINDO (Indonesian Sign Language) letters from A to Z using the Random Forest classification method. The initial stage in the classification of BISINDO letter images involves pre-processing, which includes converting RGB images to grayscale and performing segmentation through three stages: thresholding, morphology, and edge detection using the Prewitt operator. Subsequently, features such as HSV color extraction and metric shape features, as well as eccentricity, are extracted. These extracted feature values are then utilized in the classification stage of BISINDO letter images from A to Z using the Random Forest method. In this study, three data comparison scenarios were employed for testing purposes. The first scenario involved an 80:20 data ratio, which achieved a testing accuracy of 94.2%. The second scenario with a 70:30 data ratio achieved a testing accuracy of 93.6%, while the third scenario with a 60:40 data ratio had a lower accuracy of only 77.9%. Based on the results of our testing, the system developed is capable of effectively classifying BISINDO letters from A to Z using color and shape feature extraction, along with the Random Forest classification method. The best results were obtained in the data comparison scenario of 80:20, achieving an accuracy rate of 94.2%. Keywords – BISINDO, HSV, Metric, Eccentricity, Random Forest.

Applying LSTM and GRU Methods to Recognize and Interpret Hand Gestures, Poses, and Face-Based Sign Language in Real Time

This research uses a real-time, human-computer interaction application to examine sign language recognition. This work develops a rule-based hand gesture approach for Indonesian sign language in order to interpret some words using a combination of hand movements, mimics, and poses. The main objective in this study is the recognition of sign language that is based on hand movements made in front of the body with one or two hands, movements which may involve switching between the left and right hand or may be combined with mimics and poses. To overcome this problem, a research framework is developed by coordinating hand gestures with poses and mimics to create features by using holistic MediaPipe. To train and test data in real time, the long short time memory (LSTM) and gated recurrent unit (GRU) approaches are used. The research findings presented in this paper show that hand gestures in real-time interactions are reliably recognized, and some words are interpreted with the high accuracy rates of 94% and 96% for the LSTM and GRU methods, respectively.

Empowering deaf communication: a novel LSTM model for recognizing Indonesian sign language

Empowering deaf communication: a novel LSTM model for recognizing Indonesian sign language

The Comparison of K-Nearest Neighbors and Random Forest Algorithm to Recognize Indonesian Sign Language in a Real-Time

Purpose: Comparing 2 models or prototype programs which can recognize Indonesian Sign Language System or Sistem Isyarat Bahasa Indonesia (SIBI) fonts from hand gesture and translate it’s into writing Messages in real-time.Methods: After selecting datasets and reprocessed by the researcher into 1 dataset, which are a combination of several sign image datasets of the SIBI letters images available on the Kaggle website, the dataset is converted into landmarks. The landmarks are divided into 26 sign classes and preprocessed to a total of 19,826 rows of data, and then divided into 67% training data and 33% test data. Next, both K-NN and Random Forest algorithm are implemented into different program and get tested into 2 different tests, model evaluation and real-time. At the end, the result is compared to see the increase of accuracy level of both K-Nearest Neighbors (K-NN) and Random Forest algorithm.Result: The constructed and trained model is then evaluated and the results of Precision, Recall, Accuracy, and F1-Score are 99.88% using the Random Forest algorithm. The results of real-time program testing with the K-Nearest Neighbors algorithm get higher results, where the average accuracy value reaches 99%.Novelty: From the result shows that the model built with the Random Forest algorithm is superior, but the K-Nearest Neighbors algorithm is better in real-time testing. Therefore, image data and its diversity should be increased, in order to improve recognition accuracy. The program could be enhanced by adding a function where the program can recognize hand gesture, not only one or two hands but also can recognize a hand gesture with movements so the program can recognize static and dynamic letter (required hands movement).

Analysis of Sign Language Skills in Improving Communication and Learning for Deaf Children

This study uses qualitative methods that aim to identify and describe how sign language skills improve communication and learning for deaf children. Data collection in this study is literature research, namely searching for research data and information through academic articles, books, references, and published materials contained in the library. The results of this study show that one form of communication used by deaf children is nonverbal sign language, and sign language is very important for deaf children who need special support because they cannot communicate like others. Deaf children usually encounter obstacles in the learning process such as when the interaction between the teacher and students is disrupted and students may not be able to understand the content delivered by the teacher. To achieve optimal learning, appropriate teaching methods and sign language are needed, namely SIBI (Indonesian Sign Language System) and Bisindo. Such methods help deaf students acquire language skills, improve communication skills, and facilitate the exchange of materials between teachers and deaf students. Therefore, in communicating with healthy people, it is necessary to communicate using sign language that can be understood by deaf people.

Real-time recognition of Indonesian sign language using recurrent neural network

Hand gestures serve as a vital means of communication for deaf individuals. They often face communication challenges in their daily interactions due to the language barrier. This underscores the necessity of sign language interpreters. However, prevailing methods primarily rely on the Indonesian Sign Language System (SIBI), despite the widespread use of Indonesian Sign Language (BISINDO) for communication. Additionally, the effectiveness of these methods hinges greatly on the accuracy of feature extraction. To address this limitation, this study introduces a Recurrent Neural Network (RNN) approach for BISINDO interpretation. Data acquisition involved the use of a webcam to capture video data, subsequently transformed into frames andarrays. Collected from three respondents, the dataset comprises 3,240 videos and 97,200 array data points, encompassing letters and numbers. Among the tested parameters, training results indicate that utilizing the Adam optimizer with a learning rate of 0.0001 and 500 epochs yields the highest accuracy and minimal loss compared to other configurations. Subsequently, this modelunderwent real-time testing, conducted five times for 36 classes, achieving an accuracy of 81.67%. It is important to note that errors may arise due to similarities within hand signal language, particularly involving characters such as I, J, D, P, M, and N.

Penerapan Convolutional Neural Network dengan Arsitektur Mobilenetv2 Pada Aplikasi Penerjemah dan Pembelajaran Bahasa Isyarat

Sign Language is the main communication medium for deaf people, in Indonesia itself there are several types of sign language, one of which is SIBI or Indonesian Sign Language System. This type of SIBI sign language is one type of sign language used in special school (SLB) environments. Deaf children are not yet able to use sign language and others have difficulty communicating with people around them. This research aims to help deaf children learn sign language, as well as help them communicate through sign language interpreters. The methodology applied to the application is by using a Convolutional Neural Network with the MobileNetV2 architecture. CNN is an algorithm that is included in the artificial neural network category which has the advantage of having a very high level of accuracy in classification, and MobileNetV2 is a form of Convolutional Neural Network (CNN) architecture that is able to overcome the need for excess computing resources created by Google researchers. so that it can be used on mobile devices or cell phones. The model training stage obtained an accuracy value of 98.99% with a total of 100 epochs. The model trained did not experience overfitting or underfitting so the model can be used. The testing stage uses black box testing to ensure the application is running correctly. The final stage is conducting research using TAM (Technology Acceptance Model) to measure the level of approval of the application using a Likert scale. Testing using black box testing was successful and met expectations, and the implementation of the application received an approval rate of 78.4% on average.

PENGARUH VERBAL ENCOURAGEMENT TERHADAP SELF-EFFICACY MAHASISWA DALAM MENGERJAKAN KUIS BAHASA ISYARAT INDONESIA DASAR (BISINDO)

Self-efficacy is one of the essential things that students need to have, especially in academic performance. Self-efficacy has several sources, one of which is verbal persuasion in the form of verbal encouragement. Verbal encouragement can increase individual beliefs about their ability to complete tasks. This experimental study aims to determine the effect of verbal encouragement on self-efficacy. The design used is a between-participant post-test only and conducted online. The sample of this study was 80 students of the Faculty of Psychology, Padjadjaran University, Bandung. The sample was randomly divided into two groups: The experimental group with verbal encouragement and the control group without intervention. Measurement of self-efficacy using two measuring instruments, General Self-efficacy Scale, which measures self-efficacy in general, and New General Self-efficacy Scale to measure self-efficacy more specifically, namely according to the research context. Data analysis using SPSS 26 and the results obtained based on the independent t-test showed that there was a significant effect of providing verbal encouragement on the level of self-efficacy of participants at p-value=0.0015 and p-value=0.000. This shows that giving verbal encouragement can increase the self-efficacy of participants who take tests, especially the Basic Indonesian Sign Language Quiz (BISINDO).

SIBI (Sistem Bahasa Isyarat Indonesia) berbasis Machine Learning dan Computer Vision untuk Membantu Komunikasi Tuna Rungu dan Tuna Wicara

The Indonesian Sign Language System (SIBI) is used to translate sign language into text or speech. SIBI helps improve communication between people using sign language and those who do not understand it. Unlike commonly used languages, SIBI sign language is less known to most people due to a lack of interest. To address this, an artificial intelligence-based application was developed, focusing on deep learning to recognize SIBI sign language hand movements in real-time. The model was created with 20 epochs, a batch size of 16, and a learning rate of 0.001. It consists of 13 layers, with the ReLU activation function used for the input layer, while the output layer uses Sigmoid. The ADAM optimizer was used to expedite the model creation process. The image dataset used had a size of 300x300 pixels. In the classification testing of the SIBI alphabet results in this study, it was tested using distance tests. The distance between the webcam and the SIBI language speaker was divided into two categories: 40 cm and 60 cm. For a 40-cm distance, an accuracy of 87.50% was obtained, and for a 60-cm distance, an accuracy of 79.17% was achieved. One limitation of this study is that two alphabets, J and Z, were not included in the dataset. This is because recognition of these two alphabets requires not only finger pattern recognition but also recognition of their gesture patterns.

Design a Sign Language Translator Using Flexible Sensors

In terms of communication skills, many of us have limitations and shortcomings or what we are more familiar with is speech impairment Speaking is the ability to pronounce articulated sounds or words to express, express and convey thoughts, ideas and feelings. Communication skills can include many ways, including using verbal skills, namely verbally and non-verbally. In Indonesia, there are two sign languages used, namely Indonesian sign language (BISINDO) and Indonesian sign system (SIBI). BISINDO is a sign language that appears naturally in Indonesian culture and is practical for use in everyday life so that BISINDO has several variations in each region. The flex sensor has a thin and densely curved shape so that the flex sensor can be used as a motion detection and finger curve. Flex sensor application for human movement detection, patient monitoring. Therefore, hand-to-letter/text sign language translators using flexible sensors is a very important problem today. The method carried out is system design using tools and components used in research. This tool using the working principle in this system is to translate the sign language of alphabetic letters using flexible sensors. Design sign language translation of alphabetic letters using flexible sensors. The test to display the letters of the alphabet A-Z has a total minimum and maximum resistance at a flexible session of 1000 ohms with a voltage of 5V each. The result of this data is that there is no error in the play because the range value does not violate each other with other range values

Pengembangan Website ‘BALASAN’ Sebagai Upaya Peningkatan Minat Belajar Bahasa Isyarat Indonesia

This website development uses fun and interactive methods. This website collaborates with GERKATIN which provides Bisindo classes which discuss understanding the world of the Deaf, how to communicate with the Deaf, and learning the basics of Indonesian Sign Language to build inclusive communication relationships. In this web development, there are 3 Bisindo service options that can be chosen according to your needs. One of the services is the Bisindo class, where we can take direct (face to face) classes accompanied by experienced Deaf teachers. To be able to take this class you can choose a level, namely level 1 & 2. The class schedule starts when the class quota has been met. Each class is held 10 times according to the schedule determined by GERKATIN.

Klasifikasi Abjad SIBI (Sistem Bahasa Isyarat Indonesia) menggunakan Mediapipe dengan Metode Deep Learning

General public knowledge in Indonesia regarding the Indonesian Sign Language System (SIBI) is quite low. This can prevent deaf and mute people from doing activities in public facilities. In this paper, we propose an alphabetical classification of SIBI sign language using the mediapipe and the Deep Learning method to help deaf and mute people communicate with the public. The methodology of this paper collects a dataset in the form of image data from the hand patterns of each SIBI sign language alphabet by combining a webcam with the help of the open-cv library to retrieve image data. With the mediapipe, the data is extracted from the coordinates of the landmarks in his hand and then normalized for each coordinate. Then the model is trained with a fully connected layer deep learning algorithm. The experimental results obtained an accuracy of 94.32% and a loss of 15.17% on the training data, an accuracy of 93.52% and a loss of 18.91% on the validation data, and an accuracy of 93.94% on the test data. These results show that the mediapipe and the deep learning fully connected layer algorithm successfully detect the right and left hands, and coordinates of landmarks according to the SIBI sign language alphabet correctly