Camera mouse implementation using 3D head pose estimation by monocular video camera and 2D to 3D point and line correspondences

Abstract

In this paper, we present a novel approach to estimate the 3D head pose from a monocular camera images for the control of mouse pointer movements on the screen and clicking events. This work is motivated by the goal of providing a non-contact instrument to control the mouse pointer on the PC screen for helping handicapped people with severe disabilities using low-cost and widely available hardware. The required information is derived from video data captured using a web camera mounted on the computer monitor. The proposed algorithm is based on the 2D tracking of the markers on the face. Our approach proceeds in three stages. First, the positions of the markers are detected and tracked over video frames by LK algorithm. Then, the 3D rotation and translation between the web camera and 3D head pose are estimated using point and line correspondences. Finally, the 3D rotation and translation matrix is used for estimating the mouse pointer movements on the PC screen and clicking events. Experimental results showed the promise of the algorithm.