Spatio-temporal hierarchical feature transformer for UAV object tracking

Abstract

Most existing Siamese network based trackers calculate the similarity between template and search region through a simple correlation operation, and these trackers classify and regress the target object based on the similarity maps obtained by the correlation operation. However, the correlation operation itself is a local linear matching process, if these Siamese network based trackers decide only on feature maps obtained by the correlation operation, using only a single similarity feature map will reduce position accuracy in challenging environments, using multiple similarity feature maps separately will introduce complicated computations. Thus, we propose an efficient spatio-temporal hierarchical feature transformer (STHFT) for UAV tracking. The spatio-temporal hierarchical similarity maps generated by the multi-level convolutional layers are sent to the transformer to achieve the interactive fusion of shallow spatio-temporal and deep semantic information. Therefore, our STHFT not only enhances the learning ability for global contextual information, but also effectively learns the interdependencies among multi-level features. Moreover, STHFT treats object tracking as a straightforward bounding box prediction task, only using a simple fully convolutional neural network to directly estimate the corners of the target and localize it. We have conducted extensive experiments on the UAV123 and UAV20L benchmarks, STHFT outperforms other traditional trackers. Specifically, compared to the conventional tracker DaSiamRPN the success and accuracy scores of STHFT improved by 7.3% and 6.5% on the UAV123 dataset and 10.9% and 10.4% on UAV20L, which show that STHFT is robust to tracking challenges.