Eulerian Motion Based 3DCNN Architecture for Facial Micro-Expression Recognition

Abstract

Facial micro-expressions are fast and subtle muscular movements, which typically reveal the underlying mental state of an individual. Due to low intensity and short duration of micro-expressions, the task of micro-expressions recognition is a huge challenge. Our method adopts a new pre-processing technique on the basis of the Eulerian video magnification (EVM) for micro-expressions recognition. Further, we propose a micro-expressions recognition framework based on the simple yet effective Eulerian motion-based 3D convolution network (EM-C3D). Firstly, Eulerian motion feature maps are extracted by employing multiple spatial scales temporal filtering approach, then the multi-frame Eulerian motion feature maps are directly fed into the 3D convolution network with a global attention module (GAM) to encode rich spatiotemporal information instead of being added to the raw images. Our algorithm achieves state-of-the-art result \(69.76\%\) accuracy and \(65.75\%\) recall rate on the CASME II dataset, which surpasses all baselines. Cross-domain experiments are also performed to verify the robustness of the algorithm.