Thermal infrared pedestrian tracking using joint siamese network and exemplar prediction model

Abstract

• A CNN-based exemplar prediction model is designed to enhance the original Siam-RPN. • Both the temporal and spatial information around an object are introduced to the prediction model. • The proposed tracker performs better than the existing trackers on the PTB-TIR benchmark. • The proposed tracker runs in real-time. Tracking pedestrian targets over a thermal infrared (TIR) image sequence is a hot topic in visual tracking. The imagery characteristics of TIR targets such as low target-background contrast and far imaging distance make TIR object tracking very difficult. In this paper, based on a convolutional neural network (CNN) and the siamese region proposal network (SiamRPN), we design an improved TIR pedestrian tracker. By fully considering the temporal and spatial information around an object, we firstly construct a CNN-based prediction model to produce the exemplar of a pedestrian target. Then the predicted exemplar is combined with SiamRPN to form an improved real-time TIR pedestrian tracker. The proposed tracker is evaluated on the TIR pedestrian tracking benchmark dataset PTB-TIR. Our experimental results demonstrate that the proposed tracker achieves promising tracking performance. In terms of tracking success rate and precision, our tracker outperforms traditional trackers such as KCF, and state-of-the-art trackers such as SiamRPN, SRDCF, and DSST. Moreover, similar to other siamese-network-based trackers, our tracker runs in real-time.