Learning from feature and label spaces’ bias for uncertainty-adaptive facial emotion recognition

Abstract

Developing an accurate deep model for facial emotion recognition is a long-term challenge. It is because the uncertainty of emotions, stemming from the ambiguity of different emotional categories and the difference of subjective annotations, can ruin the ability of model to achieve the desired optimization. This paper constructs two distinct datasets, namely original sample set and ambiguous sample set, to explore an effective ambiguous knowledge transfer method to realize the adaptive awareness of uncertainty in facial emotion recognition. The original sample set is the weakly-augmented data with relatively low uncertainty, as most emotions are clean in reality. Meanwhile, the ambiguous sample set is strongly-augmented data that introduces feature and label bias with regard to emotion, which are with relatively high uncertainty. The proposed framework consists of two sub-nets, which are trained using the original set and the ambiguous set respectively. To achieve uncertainty-adaptive learning for two sub-nets, we introduce two modules. One is the cross-space attention consistency learning module that performs attention coupling across original and ambiguous feature spaces, achieving uncertainty-aware representation learning in feature granularity. The other is the soft-label learning module that models and utilizes uncertainty in label granularity, through aligning the posterior distributions between original label space and ambiguous label space. Experimental studies on public datasets indicate that our method is competitive with the state-of-the-art.