Full-motion recovery from multiple video cameras applied to face tracking and recognition

Abstract

Robust object tracking still remains a difficult problem in computer vision research and surveillance applications. One promising development in this area is the increased availability of surveillance cameras with overlapping views. Intuitively, these overlapping views may lead to more robust object tracking and recognition. However, combining the information from the multiple cameras in a meaningful way is challenging. Our contribution in this work is a novel approach to object tracking by robustly and accurately recovering the full motion of the object from multiple cameras. This is accomplished by explicitly fusing the information from multiple cameras into a joint 3D motion calculation. We apply this approach to the tracking of faces in multiple video cameras and utilize the 3D cylinder model to realize the motion calculation. The method is demonstrated on a sequence of real data for pose estimation of the face. Also, the 3D cylinder texture map from the tracking result is used in face recognition. The performance of full-motion recovery from multiple cameras is shown to produce a significant increase in the accuracy of face pose estimation and results in a higher face recognition rate than from a single camera. Our approach may be applied to other types of object tracking such as vehicles.