Sign Language Detection Using Deep Learning

Abstract

Abstract: Millions of citizens worldwide suffer from deaf and hard of hearing (DHH), a communication impairment that makes speaking difficult and necessitates the use of sign language. This communication gap frequently hampers access to education and opportunities for employment. Although AI-driven technologies have been studied to tackle this problem, no research has specifically looked into the intelligent and automatic translation of American sign gestures to text in low-resource languages (LRL), such as Nigerian languages. We suggest a unique end-to-end system for translating the American Sign Language, or ASL, Our framework uses the "no language left behind" translation model and the Transformer-based model for ASL-to-Text generation. converting the text generated from LRL to English. We assessed the ASL-to-Text system's performance. The people who participated were able to understand the translated text and expressed satisfaction with both the Text-to-LRL and ASL-toText models, according to a qualitative analysis of the framework. Our suggested framework shows how AI-driven technologies can promote inclusivity in sociocultural interactions and education, particularly for individuals with DHH living in low-resource environments. To bridge the gap between the non-sign language and hearing/speech impaired communities, sign language recognition is crucial. Sentence detection is more useful in real-world situations than isolated recognition of words, yet it is also more difficult since isolated signs need to be accurately identified and continuous, high-quality sign data with distinct features needs to be collected. Here, we suggest a wearable system for understanding sign language using a convolutional neural network (CNN) that combines inertial measurement units attached to the body with flexible strain sensors to detect hand postures and movement trajectories. A total of 48 frequently used ASL sign language terms were gathered and utilized to train the CNN model. This resulted in an isolated sign language word recognition accuracy of 95.85%.