Arabic sign language recognition using Ada-Boosting based on a leap motion controller

Abstract

According to the World Health Organization (WHO), 466 million people are suffering from hearing loss, i.e., 5% of the world population, of which 432 million (93%) are adults and 34 million (17%) children. The main problem is how deaf and hearing-impaired communicate with people and each other, how they get education or do their daily activities. Sign language is the main communication method for them. Building automatic hand gestures recognition system has many challenges specially in Arabic. Solving recognition problem and practically develop real-time recognition system is another challenge. Several types of research have been conducted on sign language recognition systems but for Arabic Sign Language are very limited. In this paper, an Arabic Sign Language (ArSL) recognition system that uses a Leap Motion Controller and Latte Panda is introduced. The recognition phase depends on two machine learning algorithms: (a) KNN (k-Nearest Neighbor) and (b) SVM (Support Vector Machine). Afterwards, an Ada-Boosting technique is applied to enhance the accuracy of both algorithms. A direct matching technique, DTW (Dynamic Time Wrapping), is applied and compared with AdaBoost. The proposed system is applied on 30 hand gestures which are composed of 20 single-hand gestures and 10 double-hand gestures. The experimental results show that the DTW achieved an accuracy of 88% for single-hand gestures and 86% for double-hand gestures. Overall, the proposed model’s recognition rate reached 92.3% for single-hand gestures and 93% for double-hand gestures after applying the Ada-Boosting. Finally, a prototype of our model was implemented in a single board (Latte Panda) to increase the system’s reliability and mobility.