A fast and accurate facial expression synthesis system for color face images using face graph and deep belief network

Abstract

For a given animated avatar or face, synthesizing facial expressions in a fast and accurate way is a challenging problem. This paper presents a multi-cue methodology in order to generate facial expressions in a real time manner. The proposed approach first extracts the graph of the face using constrained local model (CLM) and generates a shape based feature vector. Secondly, it employs this feature vector to train a 3 layer deep belief network. After training, the deep belief network has the ability to generate the shape of an ideal facial expression for an input face graph. A post processing step then is applied to produce proper wrinkles and illumination changes which are related to that special facial expression. Employing a small feature vector, instead of a vector which includes all pixels of the face image, increases the speed of both training and generation phase for a deep belief network and makes it intrinsically suitable for real-time purposes. In addition, this approach is independent from the format of the input image and can be used for various types of images, including color images. The experimental results demonstrate the accuracy of our algorithm.