Comparison of Gaussian Hidden Markov Model and Convolutional Neural Network in Sign Language Recognition System

Abstract

Sign language Recognition is the study to help bridging communication of deaf-mute people. Sign Language Recognition uses techniques to convert gestures of sign language into words or alphabet. In Indonesia, there are two types of sign languages which are used, Bahasa Isyarat Indonesia (BISINDO) and Sistem Isyarat Bahasa Indonesia (SIBI). The purpose of this research is comparing sign language recognition methods between Gaussian Hidden Markov Model and Convolutional Neural Network using indonesian sign language SIBI as a dataset. The dataset comes from 200 videos from 2 signers. Each signer performs 10 signs with 10 repetitions. To improve the recognition accuracy, modified histogram equalization is used as an image enhancement. Skin detection was used to track the movement of the gesture as input features in the Gaussian Hidden Markov Model and fine tuning was used in Convolutional Neural Network using transfer learning, freeze layer, and dropout. The results of the research are the Gaussian Hidden Markov Model provides accuracy value of 84.6% and Convolutional Neural Network provides accuracy value of 82%.