Ensemble learning with siamese networks for visual tracking

Abstract

Ensemble learning (EL) is an effective and commonly used technique to improve visual tasks’ accuracy, such as classification and detection. However, EL is rarely used in visual tracking. To fill this knowledge gap, we first have completed some research to investigate why knowledge distillation was ineffective in visual tracking tasks. Comparing the difference between the classification and visual tracking, conclusions are given: (i) Numerous simple negative examples are redundant, while only a few hard negative samples are valid for visual tracking knowledge distillation. (ii) The hint knowledge flows differently between classification and visual tracking. To solve the above problems, we design two new loss functions and integrate them into the proposed Ensemble Learning (EL) framework that can be employed in Siamese architectures such as SiamFC, SiamRPN, SiamFC+, and SiamRPN+. The EL treats two Siamese networks as students and enables them to learn collaboratively. A better solution is yielded by the EL framework than training students individually. Experiments on OTB-2013, OTB-2015, VOT2015, VOT2016, VOT2017, VOT2018, LaSOT and TrackingNet have verified the effectiveness of our proposed technique on boosting the performance for the four Siamese algorithms. The EL-SiamRPN+ achieves leading performance in the challenges.