Hybrid clustering-based 3D face modeling upon non-perfect orthogonality of frontal and profile views

Abstract

Multi view imaging has attracted increasing attention recently and has become one of the potential avenues in future video systems. We aim to make more reliable and robust automatic feature extraction and natural 3D feature construction from 2D features detected on a pair of frontal and profile view face images. We propose several heuristic algorithms to minimize possible errors introduced by prevalent non-perfect orthogonal condition and non-coherent luminance. In our approach, we first extract the 2D features that are visible to both cameras in both views. Then, we estimate the coordinates of the features in the hidden profile view based on the visible features extracted in the two orthogonal views. After that, based on the coordinates of the extracted features, we deform a 3D generic model to perform the desired deformation based modeling. Finally, the face model is texture-mapped by projecting the input 2D images onto the vertices of the face model. As the reconstructed 3D face model is MPEG4 compliant, it can be readily animated by standard MPEG4 facial animation parameters (FAPs). Present study proves the scope of resulted facial models for practical applications like face recognition and performance driven facial animation.