Optimization of Modified Hidden Markov Model for Vision-Based Indonesian Sign Languages Recognition

Abstract

Sign language is one of the most popular language which is used as a communication bridge that depends on hand movement. As it is used worldwide, the variety of sign languages is very large and most of the people doesn’t understand it which makes the communication between deaf and normal people interrupted. This makes sign language recognition popular as it makes people doesn’t need to understand the sign but still understand the meaning of the sign. But sign language itself has many problems such as the possibility of different dataset has the same movement but different meaning, the method used of each dataset could be ineffective in other dataset, and many other else which makes it difficult to be implemented. Other than that, vision-based recognition is not as popular as sensor-based recognition because of the difference in feature accuracy even though it could give more area of improvement. That’s why the aim of this paper is to presents the combination of methods used to recognize vision-based Indonesian Sign Language and enhance the method using optimization technique. The methodology used in this study follows four steps framework of sign language recognition which is dataset collection, preprocessing, feature extraction, and recognition. Each method is improved in order to be compared and checked what is the method combination and optimization method is the best for sign language recognition. For dataset collection, the dataset that is used is formal Indonesian Sign Language which is called Sistem Isyarat Bahasa Indonesia (SIBI) with some constraint in order to make sure the quality of the dataset is good. The preprocessing methods are differentiated into two categories which is image enhancement and hand detection. Image enhancement methods include no image enhancement, Gaussian blur, Bilateral blur, and Contrast Limited Adaptive Histogram Equalization and hand detection methods include skin detection using YCbCr color space and edge detection using Canny algorithm. Feature that is used in this study is pair of left and right-side movement for each frame that is extracted by calculating the average value of each pixels from left and right-side image that could be a representative value of each frame. Lastly the recognition system that is used is Gaussian Hidden Markov Model and its’ state optimization which includes no optimization, Latin Hypercube optimization, Hill Climbing optimization, and Combination of Latin Hypercube and Hill Climbing optimization. The experiment result of the proposed method could recognize up to 82% accuracy rate. The improvement of this research could be implemented on educational studies or game development that need vision-based sign language recognition system.