Mexican sign language recognition using kinect and data time warping algorithm

Abstract

Sign Language (SL) allows hearing and speech impaired people to communicate each other. Nevertheless, SL is not understood by most of common people. The development of a real-time Mexican Sign Language (MSL) translator could benefit the speech impaired community. In this work we propose a method capable of recognize in real-time a basic list of Mexican Sign Language (MSL) signs of 20 meaningful words and translate them into speech and text. The signs were collected from a group of 35 MSL signers executed in front of a Microsoft Kinect Sensor. The hand gesture recognition system use the RGB-D camera to store depth, color and skeleton tracking information. We propose a method to obtain the representative hand trajectory pat-tern information. A Dynamic Time Warping (DTW) algorithm is used to interpret the hand gestures. Finally, we use K-Fold Cross Validation method for testing stages. Our results achieve a mean accuracy of 99.1% using N-best strategy (N=5) after find the best-match approaches of our templates data-set. And a mean accuracy of 98.57% from real-time testing.