Abstract
This paper proposes a novel method, named Refined Knowledge Transfer (RKT), for language-based person search. Existing state-of-the-art methods do not deal with knowledge imbalance between image and text. In detail, textual identity knowledge is limited, but the image contains more identity knowledge. We propose Cross-Modal Knowledge Transfer (CMKT) to enhance textual identity knowledge by image to address this problem. Besides, multiple texts of one image include more identity knowledge than a single text. Thus, we propose Intra-Modal Knowledge Transfer (IMKT) to enhance textual identity knowledge by other texts. These two types of knowledge transfer will enhance the identity knowledge in text. Additionally, by considering that identity-irrelevant knowledge is transferred to text, we propose Knowledge Refiner (KR) to refine the knowledge in text. KR is capable of preserving identity knowledge and discarding identity-irrelevant knowledge. By combining CMKT, IMKT, and KR, RKT makes textual identity knowledge more salient. Extensive experiments show the state-of-the-art performance of RKT on the CUHK-PEDES and our proposed PRWPEDES-CN datasets. In addition, the decent generalization ability of RKT is also validated on the Flickr30K, CUB, and Flowers datasets.
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Disclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.