A Novel Method for 3D Face Detection and Normalization

Abstract

When automatically analyzing images of human faces, either for recognition in biometry applications or fa- cial expression analysis in human machine interaction, one has to cope with challenges caused by different head pose, illumination and expression. In this article we propose a new stereo based method for effectively solving the pose problem through 3D face detection and normalization. The proposed method applies a model-based matching and is especially intended for the study of facial features and the description of their dynamic changes in image sequences under the assumption of non-cooperative persons. In our work, we are currently implementing a new application to observe and analyze single faces of post-operative patients. In the proposed method, face detection is based on color driven clustering of 3D points derived from stereo. A mesh model is matched with the post-processed face cluster using a variant of the Iterative Closest Point algorithm (ICP). Pose is derived from correspondence. Then, pose and model in- formation is used for the synthesis of the face normalization. Results show, stereo and color are powerful cues for finding the face and its pose under a wide range of poses, illumina- tions and expressions (PIE). Head orientation may vary in out of plane rotations up to ±45°. Key words—Image and Video Processing, ICP-Matching, Computer Vision, 3D Face Detection, Normalization