Indonesian Sign Research Articles

Indonesian Sign Language (SIBI) Learning Media Application Based on Deep Learning Technology for Deaf Children

One of the main methods in teaching deaf children effectively is through visualization of learning materials. Previous research has developed augmented reality-based interactive learning media for deaf children. However, these learning media have limitations related to smartphone memory, dependence on certain flashcards, and limited amount of material due to flashcard limitations. Here, researchers innovate to develop SIBI learning media applications based on deep learning technology that material does not depend on a specific flashcard, but the application is able to detect real objects and various images. TensorFlow Lite is considered lighter but still allows the system to run deep learning on mobile devices with low latency. This application is designed to capture a real object or image through the camera, then the object or image will be processed and recognized using TensorFlow Lite. The object recognition process is carried out at the classification layer using switch-case syntax and global class synchronization with the YouTube API. After the image or object is recognized, the system will issue an output in the form of an Indonesian sign language video link from the object.

Implementasi Metode Dynamic Time Warping Pada Aplikasi Kamus Bahasa Isyarat Indonesia Sebagai Media Komunikasi

Communication is something vital in human existence. Communication is generally divided into two, namely verbal and non-verbal. For deaf and speech impaired people, the comic used is sign language and the intermediary used is a web-based application based on sign language from SIBI, with the implementation of the Dynamic Time Warping (DTW) method to analyze its usability and use. The use of this method in sign language dictionaries is to change the time coefficient into distance, because similarities in images can produce patterns from the object that is the example. In this test there are 24 letters and for each letter 10 image models taken via camera are used, and training is carried out with 50 datasets. The average test result collected reaches 90% because the condition of the hand can be said to be like the condition of other hands. This application is used to learn SIBI language, translate and read hand movements via real-time video. And during testing, accuracy results affect certain situations such as camera capacity, model coloring in the background and hand movements, and light brightness in the training data.
 
 Keyword: Metode Dynamic Time Warping, Indonesian Sign Language System, Deaf and Speech Impaired

Closer Look at Image Classification for Indonesian Sign Language with Few-Shot Learning Using Matching Network Approach

Huge datasets are important to build powerful pipelines and ground well to new images. In Computer Vision, the most basic problem is image classification. The classification of images may be a tedious job, especially when there are a lot of amounts. But CNN is known to be data-hungry while gathering. How can we build some models without much data? For example, in the case of Sign Language Recognition (SLR). One type of Sign Language Recognition system is vision-based. In Indonesian Sign Language dataset has a relatively small sample image. This research aims to classify sign language images using Computer Vision for Sign Language Recognition systems. We used a small dataset, Indonesian Sign Language. Our dataset is listed in 26 classes of alphabet, A-Z. It has loaded 12 images for each class. The methodology in this research is few-shot learning. Based on our experiment, the best accuracy for few-shot learning is Mnasnet1_0 (85.75%) convolutional network model for Matching Networks, and loss estimation is about 0,43. And the experiment indicates that the accuracy will be increased by increasing the number of shots. We can inform you that this model's matching network framework is unsuitable for the Inception V3 model because the kernel size cannot be greater than the actual input size. We can choose the best algorithm based on this research for the Indonesian Sign Language application we will develop further.

Development of Indonesian Sign Language System (SIBI) Dictionary Application for Students with Special Needs

The process of communication for deaf persons needs to be facilitated by a particular language. Friends who are deaf want language that is appropriate for everyday requirements in order to communicate and comprehend communications. The objectives of developing assistive technology for the Indonesian Sign Language System (SIBI) dictionary application are to optimize services, create friendly assistive technology, improve the language skills of students with hearing impairments by utilizing technology, and create an inclusive campus that is friendly for students with special needs. The model used in research and development (RD) is Borg Gall. The development stage of the SIBI dictionary application goes through 10 stages, from collecting vocabulary information to the resulting product in the form of an application embedded on a smartphone. Product testing was carried out in two stages to identify constraints in the application and use of the available features. The population was 17 students with special needs at various semester levels, including eight students with hearing impairments (deaf), students with visual impairment (blind/low vision) as many as eight students, then there is one student with obstacles to the function of mobility. The results of this study are that the SIBI Dictionary Application is a dictionary in the form of sign language videos equipped with sound and text as well as word meanings. The video also has a tutorial. This tool can help hearing-impaired pupils study after utilising a digital dictionary. Increasing hearing-impaired kids' vocabularies (value indicator assessment). Deaf community "GERKATIN" and special education students. Thus, the SIBI Dictionary Application with sign language movies with sound, text, and word definitions can better aid kids with special needs and can be downloaded from smartphone app services. All UNM students receive volunteer instruction from special needs classmates.

Applying Artificial Neural Network Based on Backpropagation Method for Indonesian Sign Language Recognition

Applying Artificial Neural Network Based on Backpropagation Method for Indonesian Sign Language Recognition

Android-Based Application for Real-Time Indonesian Sign Language Recognition Using Convolutional Neural Network

Individuals with hearing or speech impairments face challenges in communicating with others, requiring special techniques to express their thoughts and feelings. Sign language is an alternative way to communicate using a specific pattern of hand gestures to deliver messages instead of verbal speaking (oral communication). Unfortunately, most people do not know how to use and read sign language. Because of its complexity and many types of sign language worldwide, only well-trained personnel could use it as a communication medium. This research provides a solution in the form of a machine-learning Android-based application designed to recognize sign language captured by a smartphone camera and translate it into Latin characters. The recognition accommodates Convolutional Neural Network (CNN), one of the popular deep learning algorithms. This application recognizes 26 characters of Indonesian Sign Language (Bahasa Isyarat Indonesia/BISINDO) alphabets using MobileNetV3 architecture. To build the data model, dataset images were collected from 5 different models demonstrating 26 BISINDO characters in various lighting, background, and hand gesture position. These dataset images were also generated using image augmentation process to achieve the randomness by adjusting the image rotation, noise, and brightness. Based on the testing result using 6,240 dataset images, the application has 75.38% accuracy in recognizing Indonesian Sign Language alphabets.

The development of a wearable device for Sign Language Gesture Recognition (SLGR) by using Conceptual Design Approach (CDA)

In recent years, there has been a significant focus on researching and developing Sign Language Gesture Recognition (SLGR) for people with hearing and speaking impairments. This is especially important in Indonesia, where there are approximately two million people with these disabilities. However, current research and developments on SLGR devices are specifically designed to understand only particular sign language systems, such as Korean Standard Sign Language (KSDSL) and American Sign Language (ASL), each of which has its unique gestures and models. As a result, no device has been developed to recognize the gestures of the Indonesian sign language system, known as Sistem Isyarat Bahasa Indonesia (SIBI). Therefore, this study aims to develop an SLGR device that can recognize and translate SIBI gestures into output images via text and speech. The development of the SLGR device in this study is conducted by using the Conceptual Design Approach (CDA) methodology. Where in this case, previous research on SLGR devices is first observed as a benchmark for comparison. Furthermore, the benchmark is used as the basis for the function, sub-function, and specification of the proposed design. Based on these stages of benchmarking and concept development, the study concluded that the final design of the proposed SLGR device is constructed by using 5 flex and Gyroscope sensors connected wirelessly to the Raspberry microcontroller. The device is equipped with a voice system and LCR RPi as the output system for translation. Based on the combination of these sensors, the device is able to identify any particular gestures that correspond to words and phrases in SIBI and translate them into speech via the designated speaker and text display on the LCD screen. To fully understand the performance of the device, experimental tests are conducted by analyzing the input of 26 alphabets in the SIBI system. As a result, the device demonstrated an average of 92% accuracy to convert sign language into voice and text, which demonstrates the usefulness of the proposed device

Classifying BISINDO Alphabet using TensorFlow Object Detection API

Indonesian Sign Language (BISINDO) is one of the sign languages used in Indonesia. The process of classifying BISINDO can be done by utilizing advances in computer technology such as deep learning. The use of the BISINDO letter classification system with the application of the MobileNet V2 FPNLite SSD model using the TensorFlow object detection API. The purpose of this study is to classify BISINDO letters A-Z and measure the accuracy, precision, recall, and cross-validation performance of the model. The dataset used was 4054 images with a size of consisting of 26 letter classes, which were taken by researchers by applying several research scenarios and limitations. The steps carried out are: dividing the ratio of the simulation dataset 80:20, and applying cross-validation (k-fold = 5). In this study, a real time testing using 2 scenarios was conducted, namely testing with bright light conditions of 500 lux and dim light of 50 lux with an average processing time of 30 frames per second (fps). With a simulation data set ratio of 80:20, 5 iterations were performed, the first iteration yielded a precision result of 0.758 and a recall result of 0.790, and the second iteration yielded a precision result of 0.635 and a recall result of 0.77, then obtained an accuracy score of 0.712, the third iteration provides a recall score of 0.746, the fourth iteration obtains a precision score of 0.713 and a recall score of 0.751, the fifth iteration gives a precision score of 0.742 for a fit score case and the recall score is 0.773. So, the overall average precision score is 0.712 and the overall average recall score is 0.747, indicating that the model built performs very well.

PENGEMBANGAN APLIKASI PENERJEMAH BAHASA ISYARAT INDONESIA (BISINDO) MENGGUNAKAN METODE LONG-SHORT TERM MEMORY

The Indonesian Sign Language Translator (BISINDO) application is intended for individuals who are deaf and mute assist them in communicating with normal people. Due to the limitations of technology to assist individuals who are deaf and mute, the Indonesian Sign Language Translator application was developed. Additionally, it is hoped that this application will enable individuals who are deaf and mute to learn Indonesian Sign Language in real-time. LSTM generally consists of a cell, input, gate, and forget gate. LSTM is highly suitable for classifying, processing, and making predictions based on time series data. In this research, hand landmark recognition is used as a medium to store finger gestures from user inputs, which will then be saved in the form of coordinate points and stored in a file in csv format. Subsequently, this file will be trained and called back to determine the output.

Sign Language Translator Based on Raspberry Pi Camera Using The Haar Cascade Classifier Method

Sign language is the main tool of communication for people with hearing impairments. Communication is very limited and difficult to understand between normal people who do not know sign language, so an interpreter is needed. Where not everyone, even a few normal people, learns sign language, especially the Indonesian Sign Language System (SIBI). Motion Detection is an important subject in the field of computer vision, which is used by many systems. Today's Internet of Things is very helpful and facilitates daily human activities. An internet network allows a device to be controlled from a considerable distance. This study described a sign language translator tool for the deaf and speech impaired using a raspberry-pi camera and displayed it on the other device monitor. This system was built using the Python programming language and the OpenCV Library. The system is using Haar Cascade Classifier algorithm, where there will be data on all hand shapes based on the letters to be translated. This application uses the OpenCV library and Visual Studio Code IDE software connected to the Raspberry Pi Camera. The publisher will send data to other devices using the MQTT Broker to connect and display detection results to other device monitors wirelessly using a local network. The research was conducted at various distances between the hand and the webcam, from 30cm to 150cm. The research results using the Haar Cascade Classifier method to detect sign language obtained an accuracy of 82%.

Indonesian Sign Language Visualization Model (BISINDO) Website-Based Oral Health on Tooth Brushing Behavior in Deaf Children

Indonesian Sign Language Visualization Model (BISINDO) Website-Based Oral Health on Tooth Brushing Behavior in Deaf Children

Application of Random Forest Classifier Algorithm in Indonesian Sign Language System (Sibi) Detection System

Sign language is a language that prioritizes body movements and facial expressions as symbols to communicate with each other. Deaf and speech-impaired groups are the main users of sign language. One type of sign language in Indonesia is SIBI (Indonesian Sign Language System). SIBI (Indonesian Sign Language System) is a form of spoken language that is converted into sign language. Sign language is not only used as a means of communication and interaction between deaf and mute people, but also with normal people. However, until now there is still a communication gap between people who are deaf or mute and normal people. Then made a system design that can detect the sign language symbol SIBI (Indonesian Sign Language System). This system is made using the Random Forest Classifier algorithm with the help of MediaPipe Holistic and OpenCV and is made using the Python programming language. In this study, a dataset was created which was limited to 10 (ten) classes of symbols representing words in the SIBI (Indonesian Sign System) with a total of 8734 data lines which were then pre-processed by dividing the dataset into 70% training data and 30% test data. This study contains stages such as making detection, creating datasets, training classification models, and testing. The test is carried out by calculating the accuracy using the Confusion Matrix and then getting the Accuracy rate of 98.6%, Precision of 98.6%, and Recall of 98.66%. With the creation of the SIBI (Indonesian Language Sign System) detection system, it is hoped that it can reduce the gap between people who are deaf or speech impaired and normal people in communicating. Contribute to knowledge about symbol detection in SIBI (Indonesian Sign System) using the Random Forest Classifier algorithm as well as MediaPipe Holistic and OpenCV. As well as preserving and popularizing SIBI (Indonesian Sign System) as a sign language in Indonesia.

Translator of Indonesian Sign Language Video using Convolutional Neural Network with Transfer Learning


 Sign language is a language used to communicate by utilizing gestures and facial expressions. This study focuses on classification of Bahasa Isyarat Indonesia (BISINDO). There are still many people who have difficulty communicating with the deaf people. This study builds video-based translator system using Convolutional Neural Network (CNN) with transfer learning which is commonly used in computer vision especially in image classification. Transfer learning used in this study are a MobileNetV2, ResNet50V2, and Xception. This study uses 11 different commonly used vocabularies in BISINDO. Predictions will be made in real-time scenario using a webcam. In addition, the system given good results in the experiment with an interaction approach between one pair of deaf and normal people. From all the experiments, it was found that the Xception architectures has the best F1 Score of 98.5%.

TEKNIK KOMUNIKASI DAN TEKNIK PEMBELAJARAN ANAK BERKEBUTUHAN KHUSUS PADA MASA PANDEMI COVID 19

This study aims to describe communication techniques and learning techniques for Children with Special Needs during the Covid19 pandemic at KTK BSS Giwangan Yogyakarta. The research used descriptive qualitative research. The qualitative approach was chosen because this study used social interaction to obtain data from natural data sources. The data source of this research is Children with Special Needs (ABK) or Special Needs at KTK BSS Giwangan Yogyakarta in kindergarten and elementary school levels. From this research, it was found that in KTK BSS Giwangan there are three types of children with special needs, namely speech impaired, autism and learning difficulties. Communication techniques used are: 1) Using private learning methods (one teacher one child); 2) Make eye contact with the child you are talking to; 3) Cultivating a sense of empathy in conversation; 4) Discuss specific and clear topics and 5) Be patient in waiting for answers from children. Different learning techniques from other children. The learning technique used is the learning technique used for the speech impaired using the Indonesian Sign Language System (SIBI) and the Indonesian Sign Language (BISINDO). For children with Autism constraints using 1) Modeling; 2) Latent Learning 3) Give positive praise and 4) Divide all activities step by step. Learning Difficulties using 1) Multisensory Method; 2) Linguistic Method and 3) Glass Analysis

TEKNIK KOMUNIKASI DAN TEKNIK PEMBELAJARAN ANAK BERKEBUTUHAN KHUSUS PADA MASA PANDEMI COVID 19

This study aims to describe communication techniques and learning techniques for Children with Special Needs during the Covid19 pandemic at KTK BSS Giwangan Yogyakarta. The research used descriptive qualitative research. The qualitative approach was chosen because this study used social interaction to obtain data from natural data sources. The data source of this research is Children with Special Needs (ABK) or Special Needs at KTK BSS Giwangan Yogyakarta in kindergarten and elementary school levels. From this research, it was found that in KTK BSS Giwangan there are three types of children with special needs, namely speech impaired, autism and learning difficulties. Communication techniques used are: 1) Using private learning methods (one teacher one child); 2) Make eye contact with the child you are talking to; 3) Cultivating a sense of empathy in conversation; 4) Discuss specific and clear topics and 5) Be patient in waiting for answers from children. Different learning techniques from other children. The learning technique used is the learning technique used for the speech impaired using the Indonesian Sign Language System (SIBI) and the Indonesian Sign Language (BISINDO). For children with Autism constraints using 1) Modeling; 2) Latent Learning 3) Give positive praise and 4) Divide all activities step by step. Learning Difficulties using 1) Multisensory Method; 2) Linguistic Method and 3) Glass Analysis

TEKNIK KOMUNIKASI DAN TEKNIK PEMBELAJARAN ANAK BERKEBUTUHAN KHUSUS PADA MASA PANDEMI COVID 19

This study aims to describe communication techniques and learning techniques for Children with Special Needs during the Covid19 pandemic at KTK BSS Giwangan Yogyakarta. The research used descriptive qualitative research. The qualitative approach was chosen because this study used social interaction to obtain data from natural data sources. The data source of this research is Children with Special Needs (ABK) or Special Needs at KTK BSS Giwangan Yogyakarta in kindergarten and elementary school levels. From this research, it was found that in KTK BSS Giwangan there are three types of children with special needs, namely speech impaired, autism and learning difficulties. Communication techniques used are: 1) Using private learning methods (one teacher one child); 2) Make eye contact with the child you are talking to; 3) Cultivating a sense of empathy in conversation; 4) Discuss specific and clear topics and 5) Be patient in waiting for answers from children. Different learning techniques from other children. The learning technique used is the learning technique used for the speech impaired using the Indonesian Sign Language System (SIBI) and the Indonesian Sign Language (BISINDO). For children with Autism constraints using 1) Modeling; 2) Latent Learning 3) Give positive praise and 4) Divide all activities step by step. Learning Difficulties using 1) Multisensory Method; 2) Linguistic Method and 3) Glass Analysis

TEKNIK KOMUNIKASI DAN TEKNIK PEMBELAJARAN ANAK BERKEBUTUHAN KHUSUS PADA MASA PANDEMI COVID 19

This study aims to describe communication techniques and learning techniques for Children with Special Needs during the Covid19 pandemic at KTK BSS Giwangan Yogyakarta. The research used descriptive qualitative research. The qualitative approach was chosen because this study used social interaction to obtain data from natural data sources. The data source of this research is Children with Special Needs (ABK) or Special Needs at KTK BSS Giwangan Yogyakarta in kindergarten and elementary school levels. From this research, it was found that in KTK BSS Giwangan there are three types of children with special needs, namely speech impaired, autism and learning difficulties. Communication techniques used are: 1) Using private learning methods (one teacher one child); 2) Make eye contact with the child you are talking to; 3) Cultivating a sense of empathy in conversation; 4) Discuss specific and clear topics and 5) Be patient in waiting for answers from children. Different learning techniques from other children. The learning technique used is the learning technique used for the speech impaired using the Indonesian Sign Language System (SIBI) and the Indonesian Sign Language (BISINDO). For children with Autism constraints using 1) Modeling; 2) Latent Learning 3) Give positive praise and 4) Divide all activities step by step. Learning Difficulties using 1) Multisensory Method; 2) Linguistic Method and 3) Glass Analysis

Aplikasi Penerjemah Bahasa Isyarat Indonesia Ke Dalam Huruf Abjad

Sign language is the language used by people with limited speech or hearing. However, not everyone knows how to translate sign language in letters. Therefore, this research is motivated because there are many people who are normal or who do not have disabilities in speaking or listening who do not know how to communicate with people who have these limitations. So from this i raised a topic for the final project, "Application of Indonesian Sign Language Into Alphabet Letters. By using the Convolutional Neural Network classification method using the MobilenetV2 model architecture. The results of the classification will be exported into a model which will then be processed using Tensorflow Lite Android Studio. Based on the results of model testing, the resulting accuracy level reaches 0.9995 with a loss value of 0.6982. So that the performance of the application detection is not optimal. In addition, this application also displays animated letters of the Indonesian sign language alphabet so that people can also learn Indonesian sign language in visual form.

Modelling of basic Indonesian Sign Language translator based on Raspberry Pi technology

Deaf people have hearing loss from mild to very severe. Such people have difficulty processing language information both with and without hearing aids. Deaf people who do not use hearing aids use sign language in their everyday conversations. At the same time, it is difficult for general people to communicate with the deaf, so in order to communicate with the deaf they must know sign language. There are two sign languages in Indonesia, namely SIBI (Indonesian Sign Language System) and BISINDO (Indonesian Sign Language). To help with communication between deaf and normal people, we developed a model using the one-handed SIBI method as an example, and then further developed it using the one-handed and two-handed BISINDO. The main function of the method is the recognition of basic letters, words, sentences and numbers using a Raspberry Pi single-board computer and a camera which are designed to detect the movements of language gestures. With the help of a special program, images are translated into text on the monitor screen. The method used is image processing and machine learning using the Python programming language and Convolutional Neural Network techniques. The device prototype issues a warning to repeat the sign language if the translation fails, and delete the translation if it doesn’t match the database. The prototype of the device requires further development providing its flexibility: to provide reading of dynamic movements, facial expressions, to provide translation of words not included in the existing database. You need to add a database other than SIBI, such as BISINDO, or sign languages from other regions or countries.

Indonesian Sign Language System Analysis for Deaf Students at SLB Negeri 7 Jakarta During Pandemic

This research aimed to determine the kinds and how sign language used by Deaf students in SLB Negeri 7 Jakarta, especially during the current Covid-19 pandemic. It was preceed as descriptive qualitative research which complied with observation and questionnaire method for data collection, continued by data analysis in accordance with Pierce’s Semiotics theory. The sample data was taken from the sign language produced by Deaf students at SLB Negeri 7 Jakarta. It was analyzed with SIBI Dictionary to find the prudeced sign meaning. This research resulted that SLB Negeri 7 Jakarta used SIBI as the main language on the learning activity and communicate between teachers and Deaf students.