Temporal augmented contrastive learning for micro-expression recognition

Abstract

Micro-expressions (MEs) can reveal the hidden but real emotion and are usually caused spontaneously. However, the characteristics of subtlety and temporariness with the lack of sufficient ME datasets make it hard for recognition. In this paper, we propose an adaptively temporal augmented momentum contrastive learning to alleviate these problems. For the small scale, we pre-train the model on a new interpolated dataset via contrastive learning with momentum contrast (MoCo). For the subtle and rapid facial movements, we augment the temporal dynamics using an adaptive dropout operation to redundant frames. Specifically, we use a recursive way to create a new interpolated dataset from raw datasets firstly. Then we design a shallow model with an inflated inception module as the encoder of the contrastive learning. Afterward, we pre-train the model on the new dataset via momentum contrastive learning. During the pre-training, we propose adaptively temporal augmentation via generative adversarial learning. After the pre-training, we take the encoder out and finetune it for recognition. Finally, we perform extensive experiments and ablation studies on three ME datasets. The results demonstrate the effectiveness of the MoCo-like pre-training and the temporal augmentation for recognition. Moreover, the pre-trained model outperforms other state-of-the-art models based on optical flow.