Learning complementary Siamese networks for real-time high-performance visual tracking

Abstract

Recently, Siamese based methods have made a breakthrough in the visual tracking field. However, the existing trackers still cannot take full advantage of the deep features. In this work, we improve the performances of Siamese trackers by complementary learning with different types of matching features. Specifically, a Matching Activation Network (MAN) is firstly designed to highlight the matching regions of the search image given a template. Since only sparse parts of feature maps contribute to the matching result, an important design choice is to emphasize the weak-matching features by erasing the strong-matching ones and learn complementary classifiers from both types of features. Then we propose a novel complementary region proposal network (CoRPN) to take complementary features as inputs and their outputs complement to each other, which are fused to improve the performance. Experiments show that our proposed tracker achieves leading performances on five tracking datasets while retaining real-time speed.