3D word spotting using leap motion sensor

Abstract

Leap motion sensor provides a new way of interaction with computers or mobile devices. With this sensor, users can write in air by moving palm or finger, thus, avoiding traditional pen and paper for writing. The strokes of air-writing or 3D writing is different from conventional way of writing. In 3D writing, the words are connected by continuous lines instead of space between them. Also, the arbitrary size of characters and presence of frequent jitters in strokes make the recognition tasks of such words and sentences difficult. To understand the semantics of a word without recognizing each character of words, the alternative process called “word-spotting” is being used. Word-spotting is often useful than conventional recognition systems to understand complex handwriting. Hence, we propose a novel word spotting methodology for 3D text using Leap motion sensor data. Spotting/detection of a keyword in 3D sentences is carried out using Hidden Markov Model (HMM) framework. From experimental study, an average of 41.7 is recorded in terms of Mean-Average-Precision (MAP). The efficiency of the system is demonstrated by comparing traditional segmentation based system. The improved performance shows that the system could be used in developing novel applications in Human-Computer-Interaction (HCI) domain.