Spatiotemporal Interaction Transformer Network for Video-Based Person Reidentification in Internet of Things

Abstract

Video-based person re-identification, which is a significant application in the Internet of Things, aims to identify the same person in different video sequences across non-overlapping cameras. Existing methods usually utilize temporal cues to enhance spatial features. However, these methods learn the temporal and spatial information separately, which breaks the relationship between them and ignores the positive role of temporal information for learning frame-level spatial representation in the process of spatial representation learning. In this paper, we propose a novel Spatiotemporal Interaction Transformer Network (SITN) to solve this problem. To model the temporal information and the relationship between frames, we introduce a Temporal Interaction Module (TIM) to interact between frame information. Meanwhile, we combine TIM with Spatial Transformer Encoder to explore the positive role of temporal information in the learning procedure of the frame-level spatial feature. Moreover, we propose a Transformer local learning scheme by reconstructing the 2D spatial information of the frame patch sequences and extracting local features in a striped manner to strengthen the discriminative capability of our model. Extensive experiments are conducted on four public benchmarks. The results show that our model is superior compared with state-of-the-art methods.