Feature refinement: An expression-specific feature learning and fusion method for micro-expression recognition

Abstract

Micro-expression recognition has become challenging, as it is extremely difficult to extract the subtle facial changes of micro-expressions. Recently, several approaches have proposed various expression-shared features algorithms for micro-expression recognition. However, these approaches do not reveal the specific discriminative characteristics, which leads to sub-optimal performance. This paper proposes a novel Feature Refinement (FeatRef) with expression-specific feature learning and fusion for micro-expression recognition that aims to obtain salient and discriminative features for specific expressions and predicts expressions by fusing expression-specific features. FeatRef consists of an expression proposal module with an attention mechanism and a classification branch. First, an inception module is designed based on optical flow to obtain expression-shared features. Second, to extract salient and discriminative features for specific expressions, expression-shared features are fed into an expression proposal module with attention factors and proposal loss. Last, in the classification branch, category labels are predicted via a fusion of expression-specific features. Experiments on three publicly available databases validate the effectiveness of FeatRef under different protocols. The results on public benchmarks demonstrate that FeatRef provides salient and discriminative information for micro-expression recognition. The results also show that FeatRef achieves better or competitive performance with existing state-of-the-art methods on micro-expression recognition.