Micro-expression spotting based on optical flow features

Abstract

• The nose tip location-based image alignment method improves the performance of Micro-expression spotting. • The optical flow method can accurately capture micro-expression movements. • When a micro-expression occurs, a peak appears on the time-domain change curve. • Get complete micro-expression intervals after fusing local movements. Expression is the changes of facial organs due to facial muscle movement and an important way of human emotional interaction. Neurophysiological studies show that micro-expressions (MEs) are not controlled by subjective consciousness and reflect people's real emotions. That is why MEs have significant value in public security applications. This paper proposes an automatic ME spotting method of high accuracy and interpretability. Firstly, we design the nose tip location-based image alignment method to remove global displacement caused by head shaking. Secondly, according to the action unit definition in the face coding system (FACS), we select fourteen regions of interest (ROI) to capture subtle facial movements. The dense optical flow is introduced to estimate local movements and time-domain variations of the ROIs. Thirdly, we design a peak detection method on the time-domain variation curves to locate the movement intervals precisely. Lastly, we propose an overlapping index to measure the consistency of changes in different organs. Evaluation on the CAS(ME) 2 and SAMM Long Video database shows that our ME spotting method may achieve better accuracy with a relatively lower computation cost and can be applied to similar facial image processing applications.