Modelling Facial Features for Emotion Recognition and Synthesis

Abstract

ABSTRACTEmotion recognition and synthesis is one of the most important challenges for effective human–computer interaction. In this paper, a novel approach for emotion recognition is presented by modelling facial feature deformations. The presented approach is based on the fact that facial features such as lip, nose, eyes, and eyebrows get deformed due to variation in emotions. To measure the change in shapes of different facial features, landmark points are extracted around the facial features. Thin plate spline (TPS) is used to model the deformation of these landmark points. The basic property of TPS mapping function is that it is capable of computing rigid as well as non-rigid transformations between neutral and emotion image frames. The rigid transformation parameters represent affine parameters caused by head movement and non-rigid transformation parameters are used as representatives of facial feature deformation caused by emotion. To prove the modelling ability of TPS, non-rigid parameters are fed to support vector machine for emotion recognition. Moreover, an attempt is made to synthesize emotion by using TPS warping function. The mean of non-rigid transformation for an emotion is used as a template to warp the neutral image to emotion image. To evaluate the proposed approach, extended Cohn-Kanade database and JAFFE database are used and experimental results show 95% and 70% accuracy for them, respectively.