Real-time facial landmarks tracking using active shape model and LK optical flow

Abstract

Active Shape Models (ASM) is a generative model widely used to model faces. ASM has been successfully used for face and emotion recognition, and it is one of the state-of-the-art approaches because of its efficiency and representational power. Although widely employed, applying only ASM is not adequate for the practical applications, because positions of the facial landmarks are unstably extracted like jittering movements in the sequential frames, which degrades the performance of the applications. In this paper, we propose a framework for real-time facial landmarks extraction and tracking using ASM and Lucas-Kanade (LK) optical flow which is considered desirable to estimate time-varying geometric parameters in sequential dynamic images of face. In addition, we introduce a straightforward method to avoid failure to extract the facial landmarks by occlusion using the positions of the extracted landmarks by ASM and tracked by LK optical flow. Experimental results validate our approach.