Indonesian Sign Research Articles

Real-time BISINDO Hand Gesture Detection and Recognition with Deep Learning CNN

BISINDO is one of Indonesian sign language, which do not have many facilities to implement. Because it can cause deaf people have difficulty to live their daily life. Therefore, this research tries to offer an recognition or translation system of the BISINDO alphabet into a text. The system is expected to help deaf people to communicate in two directions. In this study the problems encountered is small datasets. Therefore this research will do the testing of hand gesture recognition, by comparing two model CNN algorithms, that is LeNet-5 and Alexnet. This test will look for which classification technique is better if the dataset conditions in an amount that does not reach 1000 images in each class. After testing, the results found that the CNN technique on the Alexnet architectural model is better to used, this is because when doing the testing process by using still-image and Alexnet model data which has been released in training process, Alexnet model data gives greater prediction results that is equal to 76%. While the LeNet model is only able to predict with the percentage of 19%. When that Alexnet data model used on the system offered, only able to predict correcly by 60%.
 
 Keywords: Sign language, BISINDO, Computer Vision, Hand Gesture Recognition, Skin Segmentation, CIELab, Deep Learning, CNN.

Pengembangan Media Pembelajaran Berbasis Flash pada Materi Bilangan untuk Anak Tunarungu Kelas V

This study aims to develop flash-based learning media for whole numbers and fractions for deaf children of class V that are valid, practical, and effective in training students to be skilled in solving whole number and fraction problems. The learning media is equipped with a video using SIBI (Indonesian Sign Language System) to make it easier for deaf students to understand the content of the media. This research using 6 stages referring to the Luther-Sutopo development model, namely: concept, design, material, collection, assembly, testing, and distribution. The instruments in this development research were the media validation sheet and the student response questionnaire sheet. The results of the development research are: (1) there are no misconceptions and all media functions can run well and the media validation score is 4.63 so that the media developed is valid, (2) the student response questionnaire is declared valid with a score of 4.73 so that it can be used in the study, (3) the media practicality score after being tested was 4.74 so that the media was declared practical, (4) 75% of students obtained a KKM score, this shows that students are skilled in solving whole number and fraction problems, so the media is said to be effective.

Indonesian Sign Language Interpreter Device Based on ATMega328 Microcontroller for Bali Deaf Community Denpasar

This study aims to design and build a Sign Language Interpreter Device with Voice Output in the form of an ATMega328 Microcontroller-Based Voice Speaker Module so that in its implementation and later in designing this device the writer focuses on the translation of 16 words that have been predetermined in Indonesian Sign Language especially in Denpasar City by using a Flex Sensor and a Gyro Sensor based on the ATMega328 Microcontroller with Arduino IDE programming. This device is also equipped with a 4GB SD card memory for storing voice recordings, using an ATMega328 microcontroller, four analog Flex sensors, a Gyro sensor, a buzzer and an 8 ohm speaker, and using a 7.4 volt Li-Po battery. The application of this device is aimed for thehearing impaired people who fall into the adult category who can understand writing and understand sign language. The output of this device uses an MP3 player module that is already included in the Sign Language Interpreter Device. The flex sensor readings range from 998-1005 ADC (analog digital converter) in open conditions and the sensor ranges from 1006-10018 ADC in closed conditions. The reading for the gyro pitch (Y axis) ranges from -10º to 76º then on the reading of the gyro Roll (X axis) ranges from -100º to 90º. 
 Keywords: ATMega328 microcontroller; Buzzer; Flex Sensor; Gyro Sensor

The Development of Android for Indonesian Sign Language Using Tensorflow Lite and CNN: An Initial Study

Deaf people in the world have different sign language according to their living place. They tend to use simple sign language to communicate with others. Whereas every country has their own sign language. Usually a country has one national sign language, but there are 2 kinds of sign languages (SIBI and BISINDO) in Indonesia. However, BISINDO (Indonesian Sign Language) had not recognized yet as a formal sign language like as SIBI (Indonesian Sign Language System), it is only a daily sign language among them and with normal people. Thus, researcher propose to promote BISINDO by developing BisAndro (BISINDO using Android) application. At the start, researcher prepare to compare BISINDO systems in cyberspace, develope basic BISINDO website by PHP and MySQL, train some gestures in TensorFlow lite, and make CNN (Convolutional Neural Network) plan before developing BisAndro application. BisAndro application will be built by Python programming, Tensorflow as an image recognition technology, and Android mobile phone as the launcher device. The application is expected can be used by deaf people to communicate with others easily because it facilitates two ways communication through mobile phone. So, the preparation is needed to know about initial requirement of the next BISINDO learning application using Android smartphone in order to be more useful for deaf and normal people directly.

The development of sign language learning media using multimedia systems

Language is a means to communicate. The expressions in the world are diverse, one of which is sign language. There are two types of sign languages in Indonesia, namely BISINDO (Indonesian Sign Language) and SIBI (Indonesian Language System). Besides that, few people know the sign language, and the method used still uses conventional or manual methods so that it feels challenging to communicate with people who are deaf speech. This study aims to develop sign language learning applications using multimedia systems that expected to help people with hearing impairment and hearing impairment, the general public, and teachers in communicating with speech deaf people. In this study, using the Multimedia Development Life Cycle research method, Alpha testing with Black Box testing and beta testing as an application testing method to measure user satisfaction. The results of research using beta testing is an android-based sign language learning application suitable for the general public, and the deaf equipped with evaluation features and delivery of material in the form of video.

Developing Indonesia Sign Language (BISINDO) Application with Android Based for Learning Sign Language

<p>Problems that occur in learning Indonesian sign language, especially for the Solo area, are the scarcity of opening sign language classes for the public and the limited use of instructional media. To overcome this problem, an Android-based sign language learning media application was developed with the ADDIE research method. ADDIE method, namely analysis, design, development, implementation and evaluation. This research will focus on the feasibility test of the application of learning media by using the indicators on the ISO 25010, standard that is used to test the feasibility of the software. The aspects tested are functional suitability, usability aspects, portability aspects and validation by material experts and media experts. The results of the feasibility test obtained, for the functional suitability test 100%, usability test 83.2%, 100% portability test, 100% material expert test and 93% media expert and it can be concluded that the application of sign language learning media is feasible to use.</p>

Indonesian Sign Language Recognition using YOLO Method

Sign language is a form of communication commonly used by people with hearing impairment or people with speech impediments. Not all ordinary people understand the language. The translation of sign language into the alphabet/text automatically will facilitate the communication of the deaf with ordinary people. This research aims to develop a sign language recognition system that can process input from video data using You Only Look Once (YOLO) in real-time. YOLO is an object detection method based on Convolutional Neural Network (CNN), which is accurate and fast. Retraining the Yolov3 pre-trained model is performed with adjustments to the number of channels and classes according to the sign language recognition requirement. In this research, we collect datasets independently based on the Indonesian Sign Language (BISINDO). In the experiment using image data, the system achieves 100% precision, recall, accuracy, and F1 score. While using video data, the system’s performance gets precision 77.14%, recall 93.1%, accuracy 72.97%, and F1 score 84.38%, with a speed of 8 fps.

Feature Optimization on Dual Leap Motion Controller for Indonesian Sign Language

Sign language is a language formed by a combination of finger, hand, body movements and facial expressions used by persons with disabilities such as deaf and speech impaired. One of these sign language recognitions is recognition using Leap Motion Controller (LMC) sensor technology. In addition to the sign language that is formed has diversity such as folded fingers, hidden fingers, indonesian sign forms also have characteristics and shapes that are almost similar to one another. The LMC sensor is not always able to recognize all forms of signs properly. In this study, optimization is proposed at the feature level where optimization aims to provide more detailed features and characteristics of each sign language formed. The stages of the process are designing the layout of the sensors, adding features and combining feature data from each sensor. The test of the feature optimization on this dual LMC sensor can provide an increase in the recognition accuracy of the given Indonesian sign language. The Indonesian sign language can be recognized well with an average accuracy of 87.24% and the optimization carried out is able to produce an increase in accuracy of up to 2.88%.

Variation in Indonesian Sign Language: A typological and sociolinguistic analysis

Variation in Indonesian Sign Language: A typological and sociolinguistic analysis

Variation in Indonesian Sign Language: A typological and sociolinguistic analysis

Variation in Indonesian Sign Language: A typological and sociolinguistic analysis

Social meanings of linguistic variation in BISINDO (Indonesian Sign Language)

AbstractAbstract (International Sign)In contrast to sociolinguistic research on spoken languages, little attention has been paid to how signers employ variation as a resource to fashion social meaning. This study focuses on an extremely understudied social practice, that of sign language usage in Indonesia, and asks where one might look to find socially meaningful variables. Using spontaneous data from a corpus of BISINDO (Indonesian Sign Language), it blends methodologies from Labovian variationism and analytic practices from the ‘third wave’ with a discursive approach to investigate how four variable linguistic features are used to express social identities. These features occur at different levels of linguistic organisation, from the phonological to the lexical and the morphosyntactic, and point to identities along regional and ethnic lines, as well as hearing status. In applying third wave practices to sign languages, constructed action and mouthings in particular emerge as potent resources for signers to make social meaning.

HAND GESTURE RECOGNITION FOR INDONESIAN SIGN LANGUAGE INTERPRETER SYSTEM WITH MYO ARMBAND USING SUPPORT VECTOR MACHINE

<p><em>Hand gestures are the communication ways for the deaf people and the other. Each hand gesture has a different meaning. In order to better communicate, we need an automatic translator who can recognize hand movements as a word or sentence in communicating with deaf people. </em><em>This paper proposes a system to recognize hand gestures based on Indonesian Sign Language Standard. This system uses Myo Armband as hand gesture sensors. Myo Armband has 21 sensors to express the hand gesture data. Recognition process uses a Support Vector Machine (SVM) to classify the hand gesture based on the dataset of Indonesian Sign Language Standard. SVM yields the accuracy of 86.59% to recognize hand gestures as sign language.</em></p><p><em><strong>Keywords</strong></em><em>: </em><em>Hand Gesture Recognition, Feature Extraction, Indonesian Sign Language, Myo Armband, Moment Invariant</em></p>

Optimization of Modified Hidden Markov Model for Vision-Based Indonesian Sign Languages Recognition

Sign language is one of the most popular language which is used as a communication bridge that depends on hand movement. As it is used worldwide, the variety of sign languages is very large and most of the people doesn’t understand it which makes the communication between deaf and normal people interrupted. This makes sign language recognition popular as it makes people doesn’t need to understand the sign but still understand the meaning of the sign. But sign language itself has many problems such as the possibility of different dataset has the same movement but different meaning, the method used of each dataset could be ineffective in other dataset, and many other else which makes it difficult to be implemented. Other than that, vision-based recognition is not as popular as sensor-based recognition because of the difference in feature accuracy even though it could give more area of improvement. That’s why the aim of this paper is to presents the combination of methods used to recognize vision-based Indonesian Sign Language and enhance the method using optimization technique. The methodology used in this study follows four steps framework of sign language recognition which is dataset collection, preprocessing, feature extraction, and recognition. Each method is improved in order to be compared and checked what is the method combination and optimization method is the best for sign language recognition. For dataset collection, the dataset that is used is formal Indonesian Sign Language which is called Sistem Isyarat Bahasa Indonesia (SIBI) with some constraint in order to make sure the quality of the dataset is good. The preprocessing methods are differentiated into two categories which is image enhancement and hand detection. Image enhancement methods include no image enhancement, Gaussian blur, Bilateral blur, and Contrast Limited Adaptive Histogram Equalization and hand detection methods include skin detection using YCbCr color space and edge detection using Canny algorithm. Feature that is used in this study is pair of left and right-side movement for each frame that is extracted by calculating the average value of each pixels from left and right-side image that could be a representative value of each frame. Lastly the recognition system that is used is Gaussian Hidden Markov Model and its’ state optimization which includes no optimization, Latin Hypercube optimization, Hill Climbing optimization, and Combination of Latin Hypercube and Hill Climbing optimization. The experiment result of the proposed method could recognize up to 82% accuracy rate. The improvement of this research could be implemented on educational studies or game development that need vision-based sign language recognition system.

Variation in Indonesian Sign Language: A Typological and Sociolinguistic Analysis by Nick Palfreyman

Variation in Indonesian Sign Language: A Typological and Sociolinguistic Analysis by Nick Palfreyman

Android-based Indonesian sign language model using robot hand

Sign language is one of the most effective ways to communicate with deaf people. However, not everyone is able to use sign language. Therefore, in this study, a sign language display system was created using a hand robot that was moved through an Android-based application on a smartphone to overcome this problem. The design and development of the system begin from the manufacture of hardware consisting of one hand robot with 11 servo motors as the actuators, the Arduino Mega 2560 microcontroller as the main controller and a Bluetooth module for wireless communication between hand robot and smartphone. Then software development is carried out on the hand robot controller and android based application on a smartphone. Furthermore, system testing is carried out by validating the results of the letters and numbers shown by hand robot with gestures in the Indonesian Language Signalling System Dictionary (SIBI). The results show that the hand robot has been able to display letters and numbers according to the instructions given by an Android-based application on a smartphone with an 80% accuracy rate.

Indonesian Sign Language API (OpenSIBI API) as The Gateway Services for Myo Armband

We create an API (Application Programming Interface) for Indonesian Sign Language (Sistem Isyarat Bahasa Indonesia/SIBI) which is called OpenSIBI. In this case study, we use the Myo Armband device to capture hand gesture data movement. It uses five sensors: Accelerometer, Gyroscope, Orientation, Orientation-Euler, and EMG. First, we record, convert and save those data into JSON dataset in the server as data learning. Then, every data request (trial data) from the client will compare them using k-NN Normalization process. OpenSIBI API works as the middleware which integrated to RabbitMQ as the queue request arranger. Every service request from the client will automatically spread to the server with the queue process. As the media observation, we create a client data request by SIBI Words and Alphabeth Game, which allows the user to answer several stages of puzzle-game with Indonesian Sign Language hand gesture. Game-player must use the Myo armband as an interactive device that reads the hand movement and its fingers for answering the questions given. Thus, the data will be classified and normalized by the k-NN algorithm, which will be processed on the server. In this process, data will pass OpenAPI SIBI (which connected to RabbitMQ) to queue every incoming data-request. So, the obtained data will be processed one by one and sent it back to the client as the answer.

Indonesian Sign Language Letter Interpreter Application Using Leap Motion Control based on Naïve Bayes Classifier

Most deaf children are born into hearing families and the majority do not begin their Sign Language learning before age 3-6 in school. As children learning Sign Language from infancy show higher language proficiency than those who do not, it underlines the importance of hearing parents learning how to sign. The recent innovation in the technology and introduction of new gesture control based devices, like leap motion controller. Technology that can track finger and hand gestures accurately at very low cost. The research method in Naïve Bayes algorithm, in order to classify 26 alphabet letter of Indonesian sign language including letter J and Z which using gesture for describe it. The Leap Motion controller is a consumer gesture sensor aimed to augment a user’s interactive experience with their computer and it was utilized as an interface for hand motion tracking without the need of wearing any external instruments. It use for derive feature data from the deaf children gesture recording to set data in application. The accuracy level of the test result reached 96%.

PEMBASIINDO (PEMBELAJARAN BAHASA ISYARAT INDONESIA) BESBASIS ANDROID DI SEKOLAH LUAR BIASA NEGERI 02 MAKASSAR

Many inhabitants of Indonesia, including individuals who have physical characteristics, the characters are different traits supplied by heredity and the formation of characteristics related to the physical environment and social environment. Based on these factors, humans are classified as normal people and those with special needs such as deaf. At Sekolah Luar Biasa Negeri 02 Makassar the sign language learning process still considered less effective because of the limited teaching team. In the PKM-M activities, it aims to introduce the features that provide in Androidbased PEMBASIINDO consisting of video tutorials. The benefit of this activity is the teaching team can carry out an effective learning process and the program can be useful as a learning media that can be used by the deaf and the general society. The Implementation method of these activities carried out at SLBN starts from observation, design activities, coordination with partner related to the activities, socialization, and training PEMBASIINDO application. The results of these activities are related to increasing knowledge about Indonesian sign language by PEMBASIINDO application media, helping teachers to the learning process and students with hearing impairment to communicating with their friends and the environment related to support the learning process.

Indonesian sign language recognition using kinect and dynamic time warping

Sign Language Recognition System (SLRS) is a system to recognise sign language and then translate them into text. This system can be developed by using a sensor-based technique. Some studies have implemented various feature extraction and classification methods to recognise sign language in the different country. However, their systems were user dependent (the accuracy was high when the trained and the tested user were the same people, but it was getting worse when the tested user was different to the trained user). Therefore in this study, we proposed a feature extraction method which is invariant to a user. We used the distance between two users’ skeleton instead of using the users’ skeleton positions because the skeleton distance is independent to the user posture. Finally, forty-five features were extracted in this proposed method. Further, we classified the features by using a classification method that is suitable with sign language gestures characteristic (time-dependent sequence data). The classification method is Dynamic Time Wrapping. For the experiment, we used twenty Indonesian sign languages from different semantic groups (greetings, questions, pronouns, places, family and others) and different gesture characteristic (static gesture and dynamic gesture). Then the system was tested by a different user with the user who did the training. The result was promising, this proposed method produced high accuracy, reach 91% which shows that this proposed method is user independent.

Perancangan Sistem Deteksi Isyarat BISINDO Dengan Metode Adaptive Neuro-Fuzzy Inference System (ANFIS)

Deafness is a condition where an individual's hearing cannot function normally. So, sign language was created which was used as a solution to the problem. In Indonesia, the sign languages that are known are SIBI (Indonesian Sign Language System) and BISINDO (Indonesian Sign Language). Although SIBI has been recognized by the Indonesian government, in its use it is less desirable. This research was conducted to identify empty hand signals. Where it will help the user naturally without additional assistance. Experiments carried out using a dataset that was demonstrated by 1 display. In the process, the characteristics of the hand are taken using the Histogram Oriented Gradient (HOG) method. Whereas to separate it from the background image, color segmentation is used. The results of the process are then taken to classify. The classification process uses the Adaptive Neuro-Fuzzy Inference System method. The results of the tests carried out resulted in an accuracy of 78.31%. The problem is done.