Enforcing Affinity Feature Learning through Self-attention for Person Re-identification

Abstract

Person re-identification is the task of recognizing an individual across heterogeneous non-overlapping camera views. It has become a crucial capability needed by many applications in public space video surveillance. However, it remains a challenging task due to the subtle inter-class similarity and large intra-class variation found in person images. Current CNN-based approaches have focused and investigated traditional identification or verification frameworks. Such approaches typically use the whole input image including the background and fail to pay attention to specific body parts, deviating the feature representation learning from informative parts. In this article, we introduce a self-attention mechanism coupled with cross-resolution to improve the feature representation learning of person re-identification task. The proposed self-attention module reinforces the most informative parts from a high-resolution image using its internal representation at the low-resolution. In particular, the model is fed with a pair of images on a different scale and consists of two branches. The upper branch processes the high-resolution image and learns high dimensional feature representation while the lower branch processes the low-resolution image and learns a filtering attention heatmap. The feature maps on the lower branch are subsequently weighted to reflect the importance of each patch of the input image using a softmax operation; whereas, on the upper branch, we apply a max pooling operation to downsample the high-resolution feature map before element-wise multiplied with the attention heatmap. Our attention module helps the network learn the most discriminative visual features of multiple regions of the image and is specifically optimized to attend and enforce feature representation at different scales. Extensive experiments on three large-scale datasets show that network architectures augmented with our self-attention module systematically improve their accuracy and outperform various state-of-the-art models by a large margin.