Micro-Expression Recognition Method Based on Spatial Attention Mechanism and Optical Flow Features

Abstract

<p indent="0mm">In order to solve the locality problem of micro-expression movement, an automatic micro-expression recognition method based on spatial attention mechanism of deep learning and optical flow features of micro-expressions is proposed. First, the frame difference method is used to identify apex frame for some micro-expression samples lack of apex frame labels. Then, TV<italic>-L</italic>₁ optical flow method is used to extract the horizontal and vertical components map of optical flow between the onset frame and the apex frame of micro-expression, and the corresponding optical flow strain pattern map is derived according to the horizontal and vertical components of optical flow. The three optical flow maps are connected in the way of channel superposition to form an optical flow characteristic map of micro-expression. Finally, a kind of spatial attention unit with learnable parameters is designed in the convolutional neural network built by the Inception module, which makes the model pay more attention to the regions with micro-expression motion in the feature extraction process. In the spatial attention unit, two convolution kernels of 3×3 and 7×7 are used for spatial attention inference, so that the model can comprehensively consider the attention inference results of different scale convolution kernels. Experiments on the MEGC2019 comprehensive micro-expression datasets show that the accuracy of the method is 0.788, which is better than the existing automatic micro-expression recognition method.