Two-Way Complementary Tracking Guidance

Abstract

Recently, most impressive Siamese network-based trackers are equipped with two independent branches: tracked object classification and bounding box regression. However, there is no tracking information exchange between them during the tracking optimization process. This may lead to the task-mismatch and accuracy inconsistency between both classification and regression branches during inference. To tackle the problems, we propose a novel Mutual Guidance (MG) strategy for visual object tracking, which constructs the bidirectional and complementary tracking information interaction to maintain the tracked object is well-classified to also be well-localized, between classification and regression branches. Specifically, the classification branch can guide the regression one to pay more attention to the sample with high classified scores, by re-weighting the regression loss with the classification confidence. Similarity, the regression branch also guides the classifier optimization process to focus on samples with larger IoU values. And then, the proposed Mutual Guidance is completed by a series of regularization designs on classification score and regression IoU, which dynamically re-assign the adaptive weights to the losses for each sample during the joint tracking optimization. The developed MG is generic and easy to be plugged into various tracking frameworks such as anchor-based, anchor-free based and transformer based, and boost their performance to some extent with negligible additional cost. In addition, we also develop an adaptive localization(L) branch selection scheme to further assist trackers, which determines proper localization branch for different trackers according to the difference in the way of discriminating positive and negative samples. Extensive experiments verify the effectiveness of MGL and its superiority against the state-of-the-art tracking modules on OTB100, GOT-10K, LaSOT, TrackingNet, UAV123, VOT2018 and VOT2019.