Based on wearable sensory device in 3D-printed humanoid: A new real-time sign language recognition system

Abstract

This study proposes a new real-time sign recognition system based on a wearable sensory glove, which has 17 sensors with 65 channels. We introduce the DataGlove to recognize various, possibly complex hand gestures of the Malaysian Sign Language (MSL). With 65 data channels, DataGlove can satisfy the requirement suggested by the analysis of hand anatomy, kinematic and gestures. Four groups of sensors were tested to select the optimal sensors that can capture hand gesture information. Also, a 3D-printed humanoid arm is used to validate the sensor mounted on the glove. In an extensive set of experiments to test our system, five well-known MSL participants were chosen to perform 75 gestures, all taken from the MSL numbers, alphabet, and words. The error rate was used to check system performance. Discussion confirms that the proposed system competes well with an advanced benchmark of previous works is up 100% based on 14 criteria in term of the type of captured signals, recognised gestures, and solved issues. Results show that our system is capable of recognizing the wide range of gestures with recognition accuracies of 99%, 96% and 93.4% for numbers, alphabet letters and words of MSL, respectively. This research contributes to enhancing the lifestyle of people with disabilities, and bridge the gap between people with hearing impairment and ordinary people.