Spatio-temporal Analysis of Human Faces Using Multi-Resolution Subdivision Surfaces

Abstract

We demonstrate a method to automatically extract spatio-temporal descriptions of human faces from synchronized and calibrated multi-view sequences. The head is modeled by a time-varying multi-resolution subdivision surface that is fitted to the observed person using spatio-temporal multi-view stereo information, as well as contour constraints. The stereo data is utilized by computing the normalized correlation between corresponding spatio-temporal image trajectories of surface patches, while the contour information is determined using incremental background subtraction. We globally optimize the shape of the spatio-temporal surface in a coarse-to-fine manner using the multiresolution structure of the subdivision mesh. The method presented incorporates the available image information in a unified framework and automatically reconstructs accurate spatio-temporal representations of complex non-rigidly moving objects.