Motion magnification multi-feature relation network for facial microexpression recognition

Abstract

Microexpressions cannot be observed easily due to their short duration and small-expression range. These properties pose considerable challenges for the recognition of microexpressions. Thus, video motion magnification techniques help us to see small motions previously invisible to the naked eye. This study aimed to enhance the microexpression features with the help of motion amplification technology. Also, a motion magnification multi-feature relation network (MMFRN) combining video motion amplification and two feature relation modules was proposed. The spatial feature is enlarged while completing the spatial feature extraction, which is used for classification. In addition, we transferred Resnet50 network to extract the global features and improve feature comprehensiveness. The magnification of the features is controlled through hyperparameter amplification factor α. The effects of different magnification factors on the results are compared, and the best is selected. The experiments have verified that the enlarged network can resolve the misclassification problem caused by the one-to-one correspondence between microexpressions and facial action coding units. On CASME II datasets, MMFRN outperforms the traditional recognition methods and other neural networks.