A Two-stage Deep Neural Network for Macro- and Micro-Expression Spotting from Long-term Videos

Abstract

Facial macro- and micro-expression spotting is an important task in the micro-expression analysis. This paper presents a Two-Stage Macro- and Micro-expression Spotting Network (TSMSNet) to locate the temporal positions of macro-and micro-expression in long-term videos. It is composed of two sub-networks. The first sub-network is a Triplet-Stream Attention Network (TSANet), which uses three spatial feature extraction branches and attention mechanism to extract the spatial-temporal features. The TSANet is utilized to spot the macro- and micro-expression apex frames. According to the predicted apex frames, the initial expression intervals are recommended. The second network is a Spatial-Temporal Classification Network (STCNet), which utilizes the initial expression intervals to predict the multi-scale expression in-terval proposals. Comparative results show that the proposed expression spotting method has achieved the state-of-the-art performance in two benchmark databases CAS(ME) <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> and SAMM Long Videos.