Multi‐granularity re‐ranking for visible‐infrared person re‐identification

Abstract

AbstractVisible‐infrared person re‐identification (VI‐ReID) is a supplementary task of single‐modality re‐identification, which makes up for the defect of conventional re‐identification under insufficient illumination. It is more challenging than single‐modality ReID because, in addition to difficulties in pedestrian posture, camera shooting angle and background change, there are also difficulties in the cross‐modality gap. Existing works only involve coarse‐grained global features in the re‐ranking calculation, which cannot effectively use fine‐grained features. However, fine‐grained features are particularly important due to the lack of information in cross‐modality re‐ID. To this end, the Q‐center Multi‐granularity K‐reciprocal Re‐ranking Algorithm (termed QCMR) is proposed, including a Q‐nearest neighbour centre encoder (termed QNC) and a Multi‐granularity K‐reciprocal Encoder (termed MGK) for a more comprehensive feature representation. QNC converts the probe‐corresponding modality features into gallery corresponding modality features through modality transfer to narrow the modality gap. MGK takes a coarse‐grained mutual nearest neighbour as the dominant and combines a fine‐grained nearest neighbour as a supplement for similarity measurement. Extensive experiments on two widely used VI‐ReID benchmarks, SYSU‐MM01 and RegDB have shown that our method achieves state‐of‐the‐art results. Especially, the mAP of SYSU‐MM01 is increased by 5.9% in all‐search mode.