Android mobile app for real-time bilateral Arabic sign language translation using leap motion controller

Abstract

This paper introduces an android mobile App for real-time bilateral Arabic sign language translation. The system is designed to operate on isolated sign language words. The mobile App has four main components; sign language capture for training, sign language capture for translation, voice to sign language translation and a sign language quiz game. Therefore, the Mobile App can be used for bilateral communication with the deaf community. Acquisition of sign language words is performed using a portable Leap Motion Controller that is connected to the USB of the mobile phone through an OTG adapter. We use simple feature extraction approaches based on statistical features such as mean, variance and covariance. We also use simple classification approach based on the minimum distance classifier. Classification results performed using 15 different sign language words revealed that excellent classification results are achieved. In comparison to a similar system that uses data gloves, our system has the advantage of higher accuracy and the advantage of real-time processing using a mobile App.