Identity–Expression Dual Branch Network for Facial Expression Recognition

Abstract

Accurate facial expression recognition is challenging because identity biases introduce large intraclass variations and high interclass similarities. Most existing facial expression recognition approaches are devoted to alleviate the effects of identity. However, based on the theories of cognitive science, psychology, and physiology, this article argues that the identity information is important and can promote expression recognition. Motivated by our investigation of the influences of identity on facial expression recognition, this article proposes an identity–expression dual branch network (IE-DBN) for facial expression recognition. First, identity-related features and expression-related features are learnt from the same input facial expression image by two branches respectively. Then, those two features are aggregated with our bilinear module. The bilinear aggregation not only emphasizes the impact of identity but also improves interclass variations and intraclass similarities. The joint training strategy is proposed to regularize the identity-related features. It can force our network to generate expression-guided identity-related features and suppresses negative identity factors in the same time. Experiments on three popular facial expression databases, including two popular posed facial expression databases and one spontaneous facial expression database, show that our IE-DBN outperforms most of the state of the arts, which demonstrates our superiority in facial expression recognition.