MP2PMatch: A Mask-guided Part-to-Part Matching network based on transformer for occluded person re-identification

Abstract

Occluded person re-identification (ReID) remains challenging due to the misaligned body parts. Existing works, mainly utilizing extra clues, excel in predicting holistic person images but falter when confronting substantial occlusion. This paper proposes a transformer-based Mask-guided Part-to-Part Matching (MP2PMatch) network for fine-grained matching. Firstly, the Consistency Occlusion Augmentation (COA) processes holistic person images and corresponding body part masks to construct occluded “image-mask” pairs. Next, we introduce learnable part tokens to capture semantic features of various body parts, performing “pull close” and “push apart” operations based on identity labels and part visibility, ensuring the one-to-one correspondence between part features and body parts. Additionally, the proposed Body Region Attention (BRA) utilizes the overall attention on body regions to guide the network to overcome interference from both occlusion and background. Extensive experiments demonstrate that MP2PMatch achieves exceptional occluded ReID performance.