Tri-modality consistency optimization with heterogeneous augmented images for visible-infrared person re-identification

Abstract

Listen

Translate article

Visible-infrared person re-identification (VI-ReID) is a challenging technology due to the large gap between daytime visible modality and night-time infrared modality. Previous studies mainly investigate the invariant modality-shared information by the feature-level constraint. They hardly eliminate the large discrepancy between both the inter- and intra- modality, obtaining suboptimal results. In this paper, we propose a novel Tri-modality Consistency Optimization Model (TCOM) to adequately decrease inter- and intra- modality discrepancies for VI-ReID. To this end, a pivotal heterogeneous augmented modality is generated by fusing visible images and infrared images. For tackling the distribution discrepancy, we design a Triplet Center Loss (TCL) to maintain the feature consistency by mitigating the relative distance among different modalities. Furthermore, we define a regularization term named Compact Intra-modality Constraint (CIC) that forces the same pedestrian within each modality to possess the compact feature distribution. With the two invariant constraints, TCOM explores the inter- and intra- modality space-invariant representation and compels the feature distribution from different modalities to be close to each other. Extensive experiments on mainstream databases demonstrate that TCOM achieves superior performance.