Person re-identification in indoor videos by information fusion using Graph Convolutional Networks

Abstract

Information contained in visual appearance and gait features can play an important role in designing computer vision assisted person re-identification (ReID) systems. Fusion of appearance and gait features has not yet been tested in this domain of research despite its great potential to solve some of the intriguing challenges faced by the research community due to viewpoint variations, illumination change, varying recording setups, etc. This paper proposes a new deep learning framework for person re-identification in videos. The framework referred to as Fused Graph Network (FGN-ReID) uses an information fusion strategy to deal with the aforementioned variations. We have used visual appearance and gait features independently and fused these features for re-identification of persons in close shot indoor video recordings. A feature similarity-based score-level fusion strategy has been adopted to fuse spatio-temporal and gait features using Graph Convolutional Networks (GCN). The ReID problem has been mapped to an inherent graph searching problem. In our proposed framework, appearance and gait features represent the nodes. And the relations between spatio-temporal segments denote the edges. Experiments using CASIA-B dataset reveal that the proposed method is more effective as compared to existing methods. We have improved the rank-1 accuracy by a margin of 8%–12% as compared to the baseline algorithms. The mean average precision (mAP) has improved significantly. The method has several applications including visual surveillance, biometric authentication, etc.