Gesture stroke recognition using computer vision and linear accelerometer

Abstract

In this paper we propose a method to recognize the arm motions performing within a short time, which are called “gesture strokes”, for instant interaction. We combine two modalities, computer vision and linear accelerometer, to obtain robust recognition results. The arm motion is first detected by the accelerometer, and a time window is created for this motion. Both modalities individually estimate the probability mass distribution of the gesture stroke classes from the information gathered inside this window. The estimation results of these two modalities are then combined by the dynamic model combination which is a log-linear combination with different weights for all probability masses. The set of weight exponents are learned by the Nelder-Mead method that minimizes the empirical error rate of classifying all training samples. The experiments show that these two modalities compensate for each other and the combination framework improves the recognition correct rate.