Cross-modal Person Re-identification Based on Hybrid Learning Networks

Abstract

Listen

Translate article

Objective: Person re-identification (cross-modal person re-identification, Re-ID) aims to retrieve query identities from a gallery of non-overlapping camera views. It has many practical applications such as missing person search, forensics, etc. Much pedestrian Re-ID research mainly focuses on single-modal retrieval tasks, where both indoor and outdoor pedestrian images are from red, green, and blue (RGB) modalities. However, the security monitoring system will automatically switch from the visible state to the infrared state in dark situations. In order to meet the needs of practical applications, research on cross-modal pedestrian Re-ID based on IR-RGB has attracted more and more attention. The features of different modality images essentially lack modality sharing information. Methods: Most of the existing methods use the network to extract shared features to deal with differences, but they ignore the unique feature information that is beneficial to recognition in these two modality images. In this paper, a hybrid learning network is proposed to solve the problem of learning the unique and common features of the two modalities by mixing the single-modal branch and the cross-modal branch to learn from each other. At the same time, the graph convolution attention mechanism and granular feature learning are used to process the features extracted by the cross-modal branch and the single-modal branch, so as to enrich the information input by different branch classifiers, thereby improving the discriminability of features. Results: And achieved 59.68% Rank-1 accuracy and 58.28% mean average precision average mean precision on the cross-modal data set SYSU-MM01. Conclusion: The experimental results also show that this is the effectiveness of the method.