The Comparison of K-Nearest Neighbors and Random Forest Algorithm to Recognize Indonesian Sign Language in a Real-Time

Abstract

Purpose: Comparing 2 models or prototype programs which can recognize Indonesian Sign Language System or Sistem Isyarat Bahasa Indonesia (SIBI) fonts from hand gesture and translate it’s into writing Messages in real-time.Methods: After selecting datasets and reprocessed by the researcher into 1 dataset, which are a combination of several sign image datasets of the SIBI letters images available on the Kaggle website, the dataset is converted into landmarks. The landmarks are divided into 26 sign classes and preprocessed to a total of 19,826 rows of data, and then divided into 67% training data and 33% test data. Next, both K-NN and Random Forest algorithm are implemented into different program and get tested into 2 different tests, model evaluation and real-time. At the end, the result is compared to see the increase of accuracy level of both K-Nearest Neighbors (K-NN) and Random Forest algorithm.Result: The constructed and trained model is then evaluated and the results of Precision, Recall, Accuracy, and F1-Score are 99.88% using the Random Forest algorithm. The results of real-time program testing with the K-Nearest Neighbors algorithm get higher results, where the average accuracy value reaches 99%.Novelty: From the result shows that the model built with the Random Forest algorithm is superior, but the K-Nearest Neighbors algorithm is better in real-time testing. Therefore, image data and its diversity should be increased, in order to improve recognition accuracy. The program could be enhanced by adding a function where the program can recognize hand gesture, not only one or two hands but also can recognize a hand gesture with movements so the program can recognize static and dynamic letter (required hands movement).