Facial Animation and Analysis Using 2D+3D Facial Motion Tracking

Abstract

This paper presents a facial animation system using real-time tracking of 3D facial motions from a depth camera. We first applied 2D facial motion tracking based on extended Active Shape Models (ASMs) from 2D texture image corresponding to captured 3D depth information. Based on the estimated feature point tracking from extended 2D facial motion tracking, 3D facial motions are estimated. From the estimated 3D facial motion using extended ASMs, we extract MPEG-4 facial animation parameters (FAPs) from the 3D facial motion tracking, which provides more accurate estimation of FAPs invariant to view variations. Facial animations can be achieved by any facial animation tools supporting facial animation based on FAPs.