Visualization of spatial matching features during deep person re-identification

Abstract

Person re-identification (Re-ID) based on deep learning has made great progress and achieved state-of-the-art performance in recent years. However, the end-to-end properties of deep neural networks allow us to directly feedback the output results based on its input, making the inner working mechanism of the deep person Re-ID model and its decision reasons lack of transparency and explainability. This further impedes improvements to pedestrian recognition performance. As feature visualization has been proven to be an effective method for characterizing the middle layer of a neural network, we propose a novel gradient-based visualization method to interpret the internal features learned by deep person Re-ID. Based on the idea of transfer learning, this model regards the pretrained ResNet-50 on the ImageNet dataset as a basic network for deep person Re-ID. First, the network is fine-tuned on the person Re-ID dataset to achieve pedestrian classification, and then, the gradient-based visualization of the trained network is performed to highlight important regions contributing to image similarity. Experiments conducted on the Market-1501 dataset verify that our model can not only enable the network to identify key features of an individual across different images, but also provide visual interpretation for the pedestrian classification results to improve the reliability of person Re-ID and foster trust from users regarding its decisions.