Indonesian Sign Language (SIBI) Learning Media Application Based on Deep Learning Technology for Deaf Children

Abstract

One of the main methods in teaching deaf children effectively is through visualization of learning materials. Previous research has developed augmented reality-based interactive learning media for deaf children. However, these learning media have limitations related to smartphone memory, dependence on certain flashcards, and limited amount of material due to flashcard limitations. Here, researchers innovate to develop SIBI learning media applications based on deep learning technology that material does not depend on a specific flashcard, but the application is able to detect real objects and various images. TensorFlow Lite is considered lighter but still allows the system to run deep learning on mobile devices with low latency. This application is designed to capture a real object or image through the camera, then the object or image will be processed and recognized using TensorFlow Lite. The object recognition process is carried out at the classification layer using switch-case syntax and global class synchronization with the YouTube API. After the image or object is recognized, the system will issue an output in the form of an Indonesian sign language video link from the object.