Perception driven 3D facial expression analysis based on reverse correlation and normal component

Abstract

Research on automated facial expression analysis (FEA) has been focused on applying different feature extraction methods on texture space and geometric space, using holistic or local facial regions based on regular grids or facial anatomical structure. Not much work has been investigated by taking human perception into account. In this paper, we propose to study the facial expressive regions using a reverse correlation method, and further develop a novel 3D local normal component feature representation based on human perceptions. The classification image (CI) accumulated in multiple trials reveals the shape features which alter the neutral Mona Lisa portrait to positive and negative domains. The differences can be identified by both humans and machine. Based on the CI and the derived local feature regions, a novel 3D normal component based feature (3D-NLBP) is proposed to represent positive and negative expressions (e.g., happiness and sadness). This approach achieves a good performance and has been validated by testing on both high-resolution database and real-time low resolution depth map videos.