Study of Text Segmentation and Recognition Using Leap Motion Sensor

Abstract

Recognition of 3-D texts drawn by fingers using Leap motion sensor can be challenging for existing text recognition frameworks. The texts sensed by Leap motion device are different from traditional offline and on-line writing systems. This is because of frequent jitters and non-uniform character sizes while writing using Leap motion interface. Moreover, because of air writing, characters, words, and lines are usually connected by continuous stroke that makes it difficult to recognize. In this paper, we present a study of segmentation and recognition of text recorded using Leap motion sensor. The segmentation task of continuous text into words is performed using a heuristic analysis of stroke length between two successive words. Next, the recognition of each segmented word is performed using sequential classifiers. In this paper, we have performed 3-D text recognition using hidden Markov model (HMM) and bidirectional long short-term memory neural networks (BLSTM-NNs). We have created a data set consisting of 560 Latin sentences drawn by ten participants using Leap motion sensor for experiments. An accuracy of 78.2% has been obtained in word segmentation, whereas 86.88% and 81.25% accuracies have been recorded in word recognition using BLSTM-NN and HMM classifiers, respectively.