Visual interface system by character handwriting gestures in the air

Abstract

A visual interface system that recognizes handwriting of Japanese katakana characters in the air has been developed. Characters written in a single stroke have both on-strokes and off-strokes. Thus, the shapes of the hand-gesture characters are different from the shapes of characters written on paper. It is difficult to trace the shapes of characters in air because the writer cannot see the trajectories. In this study, a light emission diode (LED) pen and a TV camera are used to capture the LED light trajectory, and the movements of the light are converted into direction codes correcting the slant of the handwriting character. The codes are normalized to 100 data items to eliminate the effect of writing speed. The 100 direction codes are compared with model data in which the direction codes of 46 Japanese characters are defined. Next, the system has expanded to a multi-camera system. Two of four cameras are selected and the 3-D positions of the gesture trajectories are calculated by the stereo method, and the position data are converted into front view data. In the experiments, we attained a recognition rate of 92.9% for the single-camera system. The multi-camera system has the advantage that it can recognize gestures regardless of the origin directions of the gestures. The system also has the ability to recognize the directions of the gesture commands with an accuracy of 9°.