Classifying Facial Expressions Based on Topo-Feature Representation

Abstract

Facial expression analysis and recognition could help humanize computers and design a new generation of human computer interface. A number of techniques were successfully exploited for facial expression recognition (Chang et al., 2004; Cohen et al., 2004; Cohen et al., 2003; Gu & Ji, 2004; and Littlewort et al, 2004), including feature estimation by optical flow (Mase, 1999; Yacoob & S. Davis, 2006), dynamic model ( Essa & Pentland, 1997), eigenmesh method (Matsuno et all.) and neural networks (Rosenblum et all., 1996). The excellent review of recent advances in this field can be found in (Y. Tian et al., 2001; Pantic & Rothkrantz, 2000; Zhao et al., 2000). The conventional methods on facial expression recognition concern themselves with extracting the expression data to describe the change of facial features, such as Action Units (AUs) defined in Facial Action Coding System (FACS) (Donato et al., 1999). Although the FACS is the most successful and commonly used technique for facial expression representation and recognition, the difficulty and complexity of the AUs extraction limit its application. As quoted by most previous works (Essa & Pentland, 1997; Yacoob & Davis, 1996), capturing the subtle change of facial skin movements is a dif cult task due to the difficulty to implement such an implicit representation. Currently, feature-based approaches (Reinders et al., 1995; Terzopoulos & Waters, 1993) and spatio-temporal based approaches (Essa & Pentland) are commonly used. Yacoob & Davis, 1996 integrated spatial and temporal information and studied the temporal model of each expression for the recognition purpose, a high recognition rate was achieved. Colmenarez et al. used a probabilistic framework based on the facial feature position and appearances to recognize the facial expressions, the recognition performance was improved, but only the feature regions other than the surface information were explored. Recently, Tian, Kanade and Cohn (Tian et al., 2001) noticed the importance of the transient features (i.e., furrow information) besides the permanent features (i.e, eyes, lips and brows) in facial expression recognition. They explored the furrow information for improving the accuracy of the AU parameters, an impressive result was achieved in recognizing a large variety of subtle expressions. To our knowledge, little investigation has been conducted on combining texture analysis and surface structure analysis for modeling and recognizing facial expressions. A detailed higher level face representation and tracking system is in high demand. In this paper, we explore the active texture information and facial surface features to meet the challenge – modeling the facial expression with sufficient accuracy.