Learning Domain-Invariant Discriminative Features for Heterogeneous Face Recognition

Shanmin Yang,Keren Fu,Xiao Yang,Cheng Peng,Jianwei Zhang,Ye Lin

doi:10.1109/access.2020.3038906

Shanmin Yang, Keren Fu + Show 4 more

Open Access

https://doi.org/10.1109/access.2020.3038906

Copy DOI

Abstract

Heterogeneous face recognition (HFR), referring to matching face images across different domains, is a challenging problem due to the vast cross-domain discrepancy and insufficient pairwise cross-domain training data. This article proposes a quadruplet framework for learning domain-invariant discriminative features (DIDF) for HFR, which integrates domain-level and class-level alignment in one unified network. The domain-level alignment reduces the cross-domain distribution discrepancy. The class-level alignment based on a special quadruplet loss is developed to further diminish the intra-class variations and enlarge the inter-class separability among instances, thus handling the misalignment and adversarial equilibrium problems confronted by the domain-level alignment. With a bidirectional cross-domain data selection strategy, the quadruplet loss-based method prominently enriches the training set and further eliminates the cross-modality shift. Benefiting from the joint supervision and mutual reinforcement of these two components, the domain invariance and class discrimination of identity features are guaranteed. Extensive experiments on the challenging CASIA NIR-VIS 2.0 database, the Oulu-CASIA NIR&VIS database, the BUAA-VisNir database, and the IIIT-D viewed sketch database demonstrate the effectiveness and preferable generalization capability of the proposed method.

Highlights

Deep convolution neural network (CNN) based face recognition (FR) has made impressive progress in recent years [1]
We briefly review the adversarial domain adaptation approaches and deep metric learning algorithms that are associated with heterogeneous face recognition (HFR)
We describe the CNN feature extraction process as fi = Gf (xi; θf ), where Gf is the feature extractor defined by convolution neural network with parameters θf, xi ∈ {Xs ∪ Xt} is the ith training sample, fi ∈ Rm is the face representation

Summary

Introduction

Deep convolution neural network (CNN) based face recognition (FR) has made impressive progress in recent years [1]. The performance of most face recognition systems degrades severely in specific real-world applications, e.g., identifying faces captured with near-infrared (NIR) sensory devices in surveillance under night-time and low-light conditions [2] and (or) recognizing sketch drawings based on the description of witnesses [3]. This problem mainly results from that most pre-enrolled face databases are collected in visual (VIS) conditions, which have substantial appearance differences from NIR and sketch counterparts. Compared with traditional face recognition, HFR further faces more challenges

Methods

Results

Conclusion