Vision-language constraint graph representation learning for unsupervised vehicle re-identification

Abstract

Existing vehicle re-identification methods rely on visual features to extract vehicle identity information. However, while individual visual features enable the model to learn limited semantic information, multimodal representations are difficult to extract. A vision-language constraint graph representation learning method guided by textual descriptions is proposed to exploit the cross-modal robustness capacity. Initially, the training set creates the unique conditional prompts for textual feature extraction. These prompts are employed to improve the understanding of visual modalities. We subsequently designed a vision-language constraint graph topology, where each training sample is considered a node in the graph. Under the dual constraints of visual and textual features, the relationship between graph nodes is further explored to construct more reliable positive and negative sample pairs for graph representation learning. Then, neighboring node label smoothing is introduced to mitigate label noise generated by visual feature clustering and achieved by combining pseudo-label assignment results from neighboring node pairs in the graph topology. Extensive experiments have confirmed that the proposed method achieves state-of-the-art performance by combining salient information from visual and textual modalities.