Gated Fusion of Discriminant Features for Caricature Recognition

Abstract

Caricature recognition is a challenging problem, because there are typically geometric deformations between photographs and caricatures. It is nontrivial to learn discriminant large-margin features. To combat this challenge, we propose a novel framework by using a gated fusion of global and local discriminant features. First, we employ A-Softmax loss to jointly learn angularly discriminant features of the whole face and local facial parts. Besides, we use the convolutional block attention module (CBAM) to further boost the discriminant ability of the learnt features. Next, we use global features as dominant representation and local features as supplemental ones; and propose a gated fusion unit to automatically learn the weighting factors for these local parts and moderate local features correspondingly. Finally, an integration of all these features is used for caricature recognition. Extensive experiments are conducted on the cross-modal face recognition task. Results show that, our method significantly boosts previous state-of-the-art Rank-1 and Rank-10 from 36.27% to 55.29% and from 64.37% to 85.78%, respectively, for caricature-to-photograph (C2P) recognition. Besides, our method achieves a Rank-1 of 60.81% and Rank-10 of 89.26% for photograph-to-caricature (P2C) recognition.