Posed and spontaneous expression recognition through modeling their spatial patterns

Abstract

This paper presents a new method to recognize posed and spontaneous expressions through modeling their spatial patterns. Gender and expression categories are employed as privileged information to further improve the recognition. The proposed approach includes three steps. First, geometric features about facial shape and Action Unit variations are extracted from the differences between apex and onset facial images to capture the spatial facial variation. Second, statistical hypothesis testings are conducted to explore the differences between posed and spontaneous expressions using the defined geometric features from three aspects: all samples, samples given the gender information, and samples given expression categories. Third, several Bayesian networks are built to capture posed and spontaneous spatial facial patterns respectively given gender and expression categories. The statistical analysis results on the USTC-NVIE and SPOS databases both demonstrate the effectiveness of the proposed geometric features. The recognition results on the USTC-NVIE database indicate that the privileged information of gender and expression can help model the spatial patterns caused by posed and spontaneous expressions. The recognition results on both databases outperform those of the state of the art.