Person Re-Identification with Weighted Spatial-Temporal Features

Abstract

Person re-identification (re-id) which resolves to recognize a person from the non-overlapped cameras has received increasing research. In this paper, we addressed a new problem of person re-id, i.e., image-to-video (ImtoV) person re-id, in which the probe is an image and the gallery consists of videos from nonoverlapping cameras with different views of probe image as shown in Fig. 1. It is different from the traditional image-based person re-id in which the probe and gallery are all images. Although more information in the video is brought into ImtoV, it remains a challenging problem because of the large variations of light conditions, viewing angles, body pose, and occlusions in different views of videos. One problem is that most of the current models ignore that different frames play different importance in the matching, and assign equal weights to feature vector of each frame of videos. However, frames with serious occlusion and dramatical illumination change have the negative effect in improving the re-id performance. In order to overcome this problem, we proposed a novel framework for this task. We adopted CNNs for the feature extraction of images and videos, and further employed LSTM network for the spatiotemporal feature representation of videos. We added a weight modular to learn the weights for different frames of videos adaptively. We evaluated the proposed framework on three public person re-id datasets, and the experimental results showed that the proposed approach was effective for the ImtoV person re-id.