Bidirectional Cross-scale Feature Fusion for Long Video Micro-Expression 3D Spotting Network

Abstract

Psycho-cognitive computing is an important part of intelligent human-computer interaction technology, which has received extensive attention in recent years. The research of micro-expressions can reflect the depth and breadth of mental cognitive computing. Micro-expression(ME) is a spontaneous, short-lived, and inadvertent facial expression. The research on ME is of great significance in sentiment analysis, criminal investigation, and psychology research. ME spotting refers to locating sequences of micro-expressions in a long video. ME detection is an extremely important step in the field of ME analysis. Based on the I3D backbone network of long video optical flow extraction and original video feature extraction, this paper extracts effective feature layers, performs downsampling, and uses the BiFPN module with a spatial attention mechanism to selectively fuse the extracted multi-scale feature layers. The final classification-regression network discriminates the detected expressions and locates the temporal boundaries where the expressions occur. The experimental results show that the proposed method effectively improves the F1 score index. Compared with other deep learning methods, this method performs better on both CAS(ME) <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> and SAMM. The proposed method provides a reliable foundation for the downstream tasks of ME research, such as ME recognition, and also provides an effective method for collecting natural macro-or micro-expression datasets from a large amount of video data in the future.