Improving Person Re-Identification With Multi-Cue Similarity Embedding and Propagation

Abstract

Most existing person re-identification (Re-ID) methods rely on the visual appearance of the human body. However, face cues are rarely explored in the Re-ID community despite the face that it is an important biometric identifier for human beings. In this work, we propose a Similarity Ensemble Framework (SEF) that uses multi-cue similarity embedding and propagation to effectively fuse body and face information for person re-identification. Specifically, for each query, we first perform standard pedestrian retrieval using body and face cues, respectively, to obtain some candidate results with high confidence. Next, the body and face similarities are combined and embedded into a shared space as node features, and two graphs with the same nodes and different edges with respect to body and face affinities are constructed. Then, the similarity features are propagated in both body and face graphs using graph convolution to capture the relationship among the candidates using different cues. Lastly, the refined features are used to compute the final similarities with the query. The proposed method not only combines the similarities of body and face, but also takes into account the relationship among all the other candidate samples under different cues. Extensive experiments demonstrate that the use of face cues effectively improves the performance of person Re-ID even if the performance obtained by the face alone is much lower than that of the body, suggesting that our approach is able to capture valuable information beyond body from weaker face cues in person Re-ID scenarios.