Vision-Based Fingertip Tracking Utilizing Curvature Points Clustering and Hash Model Representation

Abstract

Fingertip tracking plays an increasingly important role in augmented reality and virtual-reality applications. However, existing approaches (either continuous-detection-based or separated-detection-tracking-based methods) cannot effectively learn temporal-spatial information or even separate fingertip tracking into unrelated stages, which causes poor real-time performance and incomplete tracking continuity. Moreover, due to the need for high-cost devices, the high degrees of freedom of the hand, and subtle differences among fingers, fingertip tracking remains a challenging task. To address these problems, we propose a novel tracking-combined-with-detection approach for vision-based fingertip tracking. By adopting clustering and geometric constraint analysis, we develop a curvature points clustering method for fingertip detection. Then, by exploiting the identified fingertip points for motion estimation with bidirectional optical flows and temporal-spatial probability calculation, the tracking stage is effectively integrated with the detection stage. To accurately locate the fingertip, we represent the fingertip model with a perceptual hash sequence and locate the fingertip by searching for the best-matching region. Extensive experimental results show the superiority of the proposed algorithm to commonly used and state-of-the-art methods and demonstrate its effectiveness and practicability.