On-air English Capital Alphabet (ECA) recognition using depth information

Abstract

On-air writing can be considered as a time-dependent event where hand gesture is produced in a natural environment through index finger movement. A sequence of such movements containing several time steps in 3D space can be utilized to construct an English Capital Alphabet (ECA). While Previous researches investigated 2D features, we believe that depth information may play a significant role along with other features in recognition of these dynamic gestures. We have captured hand finger motion information using a depth camera and represented them as depth images for each ECA. The hand finger trajectory data were extracted from the depth image, and a combination of depth-based features and non-depth features were generated; depth variation was performed in the depth-based features, and then, all the feature values were converted into time-series data. Dynamic Time Warping distances were determined between a template ECA and a test ECA for each ECA collected from 15 participants. These distance-based features were then fed into a multi-class SVM for training and testing and got the recognition accuracy of $$80.77\%$$ without depth and $$88.21\%$$ with depth-based features. To cope with the over-fitting problem, we applied the resampling technique and got the highest recognition accuracy of 96.85%, and at last, we applied some feature selection techniques to analyze the recognition results.