GEME: Dual-stream multi-task GEnder-based micro-expression recognition

Abstract

Recognition of micro-expressions remains a topic of concern considering its brief span and low intensity. This issue is addressed through convolutional neural networks (CNNs) by developing multi-task learning (MTL) method to effectively leverage a side task: gender detection. A dual-stream multi-task framework called GEME is introduced that recognises micro-expressions by incorporating unique gender characteristics and subsequently improves the micro-expression recognition accuracy. This research aims to examine how gender differences influence the way micro-expressions are displayed. The current study proves that selecting relevant features of micro-expressions distinctive to the gender and added to the micro-expression features improves the micro-expression recognition accuracy. This network learns gender-specific features and micro-expression features and adds them together to learn the combination of shared and task-specific representations. A multi-class focal loss is used to mitigate the class imbalance issue by down-weighing the easy samples and concentrate more on misclassified samples. The Class-Balanced (CB) focal loss is also implemented for a better class balancing during Leave-One-Subject-Out (LOSO) validations where CB loss re-balances and re-weights the loss. The experimental results on three widely used databases demonstrate the improved performance of the proposed network and achieve comparable results with the state-of-the-art methods.