Robust hand tracking via novel multi-cue integration

Abstract

In this paper, we present a robust real-time hand tracking system via multi-cue integration. In practice, the motion information of the hand, such as optical flow, is hard to exploit, because images of hands lack texture. As a result, the integration of the color and motion cues using conventional integration algorithms is difficult. Here, we integrate the motion and color cues from a novel feature point selection view. The hand is tracked using feature points, and the integration is realized during the feature points generation and selection process. In the generation process, a bounding box estimated by the color cue is used to provide a region for the feature points generation. Then, the RCD (Representative, Compact and Diverse) criteria are proposed to control the feature point selection process. After the selection process, the feature points are tracked using estimates of the motion of each feature point. The centroid of the feature points in each frame is adopted as the position of the hand. The experimental results show that our integration algorithm outperforms tracking algorithms that only use a single cue. Also the proposed tracking algorithm is more robust in complex environments than other state-of-art algorithms.