Feature based global and local motion estimation for videoconference sequences

Abstract

We present an algorithm for face 3D motion estimation in videoconference scenes. The algorithm uses a modification of the CANDIDE face model and is based on feature tracking and the extended Kalman filter (EKF). Various techniques are adopted to increase the robustness of the feature tracking procedure and, in particular, a filtering technique on reference blocks. Global motion estimation is used as a starting point for local motion detection. To this purpose, we generate, by texture mapping, a synthetic image of the mouth, whose shape is changed using a set of action units (AU). The optimal AU values are determined via a gradient-based minimization procedure of the error energy between the template and the actual mouth image. The proposed scheme is quite robust and was tested with success on long sequences.