Converting South African sign language to verbal

Abstract

There is a significant population of hearing-impaired people who reside in South Africa; however, South African Sign Language (SASL) has not yet been recognized as South Africa's 12 <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">th</sup> official language, resulting in slow uptake of this important language. Since most people do not know SASL, there is a need for gesture recognition systems that convert Sign Language (SL) to verbal and/or text to reduce the communication barriers between the hearing and the hearing-impaired. This study presents an application for gesture recognition in converting SASL to both a verbal format and a textual format. By using gesture recognition from a single wearable glove, hand gestures were quantified, categorized, and then converted into an auditory format and played on a speaker, as well as the equivalent textual information displayed on an LCD screen. The complete prototype consists of a wearable glove with a transmitter and an associated receiver box which were all designed to cost less than $150. The glove consists of five flex sensors that measure the handshape and an inertial measurement unit which measures the hand motion. The handshape and motion data are processed and wirelessly transmitted to a receiver box. This then displays the associated English character on an LCD while also playing the audio on a speaker. The SL converter can convert the 26 letters of the SASL manual alphabet with an overall accuracy of 69%, It can also convert common words and phrases, as well as proper names when fingerspelled.