A novel micro-expression detection algorithm based on BERT and 3DCNN

Abstract

As a special type of facial expressions, the spontaneous micro-expressions can reveal the genuine emotions that people attempt to hide, therefore can provide potential information in criminal detection, lie detection, etc. Compared to ordinary facial expressions, micro-expressions are involuntary, transient and of low intensity. Consequently, micro-expression detection is difficult and overly dependent on expert experiences. Thus, we propose a novel micro-expression detection method based on the Bidirectional Encoder Representation from Transformers (BERT) network, namely R3D_BERT+Group, which includes the candidate segment generation module, spatio-temporal feature extraction module and grouping module. Specifically, the candidate segments are generated by the candidate segment generation module, then each candidate segment is divided into smaller time slots by the spatio-temporal feature extraction module, where spatio-temporal features are extracted through 3DCNN and BERT network. Finally, consecutive segments are merged and overlapping segments are suppressed by the grouping module to locate the position of the onset and offset frames of the micro-expression more accurately. Comprehensive experiments on CASME 2 and SDU_spotting databases firmly demonstrate the effectiveness of our method over other state-of-the-art detection methods.