A Kinect-based system for Arabic sign language to speech translation

Abstract

Speech-based smart systems have come to play an increasingly diverse role in today's pervasive technology. Moreover, it is quite common to experience all kinds of innovations on daily basis, ranging from retina identifiers at banks to electronic fingerprints readers. Such proliferation presents an opportunity and a challenge to integrate speech and hearing-challenged individuals into society by designing sign language to speech translation systems. In this paper, we tackle the problem of Arabic Sign Language to Speech transformation. We make use of commercial off-the-shelf components to capture the Sign Language gestures. Graphical gestures were transformed into Arabic text, which in turn can be translated into any spoken language. Web services were used to generate the spoken sounds. The majority of this paper is dedicated to explaining hand and fingers identification. In addition, motion recognition is also detailed. The accuracy in identifying the implemented characters was shown to exceed 80%.