Exploring holistic discriminative representation for micro-expression recognition via contrastive learning

Abstract

Recently, deep learning-based micro-expression recognition (MER) has been remarkably successful in the affective computing and computer vision communities. However, the most challenging issue that hinders the performance of MER is low intensity. Instead of forcefully transforming the input from micro-expressions to exaggerated micro-expressions by a fixed video motion magnification factor, our approach introduces a sophisticated pretext task with an intensity-agnostic strategy to enhance the discriminative capacity of each micro-expression sample holistically through contrastive transfer learning. This strategy enables us to progressively transfer knowledge and leverage the rich facial expression information from macro-expression samples. In addition, we reconsider that the core of the MER task is to refine and incorporate the instance-level and class-level discriminative features from the initial indistinguishable information. As a result, we jointly merge the two views to learn a holistic-level representation. Simultaneously, to ensure a strong association and guidance between the instance-level view and the class-level view, we maintain their consistency through an alignment loss. The results showed that the proposed method could significantly improve the performance of MER on CASME II, SAMM, SMIC, and CAS(ME)3 datasets.