Multi-view Geometry Consistency Network for Facial Micro-Expression Recognition From Various Perspectives

Abstract

Micro-expression can reveal underlying genuine emotions, but those rapid and subtle changes are hard to be captured by humans. Most existing research focuses on frontal face micro-expression recognition, which largely prevents the developed methods from the real applications and ignores the underlying geometry information. In this paper, we propose a multiview geometry consistency framework to enable the same emotion to be recognized under different perspectives, which is difficult for existing systems. Based on the developed 3D face reconstruction network, the multi-view micro-expression recognition framework empowers the emotion recognition capability to learn from multiple perspectives of the 3D reconstructed faces based on view-consistency, and a spiking neural network is further applied to capture omitted tiny and detailed changes. With a sequence of images, we explore the subtle changes across frames through optical flow, which, as a clue, enhances the performance of our designated network for micro-expression recognition. Extensive experiments on benchmark micro-expression datasets CAS(ME) <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> and SMIC demonstrate the proposed method achieves promising results on novel-view micro-expression recognition where existing methods mainly fail.