Advances in machine translation for sign language: approaches, limitations, and challenges

Abstract

Sign languages are used by the deaf community around the globe to communicate with one another. These are gesture-based languages where a deaf person performs gestures using hands and facial expressions. Every gesture represents a word or a phrase in the natural language. There are more than 200 different sign languages in the world. In order to facilitate the learning of sign languages by the deaf community, researchers have compiled sign language repositories comprising of gestures. Similarly, algorithms have been proposed to translate the natural language into sign language, which is subsequently converted into gestures using avatar technology. On the other hand, several different approaches for gesture recognition have also been proposed in the literature, many of which use specialized hardware. Similarly, cell phone applications have been developed for learning and translation of sign languages. This article presents a systematic literature review of these multidisciplinary aspects of sign language translation. It provides a detailed analysis of carefully selected 147 high-quality research articles and books related to the subject matter. Specifically, it categorizes different approaches used for each component, discusses their theoretical foundations, and provides a comparative analysis of the proposed approaches. Lastly, open research challenges and future directions for each facet of the sign language translation problem have been discussed. To the best of our knowledge, this is the first comprehensive survey on sign language translation that discusses state-of-the-art research from multi-disciplinary perspectives.