Long-Term Visual Object Tracking via Continual Learning

Abstract

Long-term visual tracking is one of the most challenging problems in computer vision and is closer to practical application needs. In long-term video sequences, tracking targets often undergo dramatic appearance changes over time due to various factors such as scale variation, illumination change, occlusions and so on. In this work, we propose a novel robust long-term tracking framework based on continual learning and dynamic sample set modules. We transform the online tracking process into a continual learning process of the target model, and continuously learn various appearance changes to adapt to different scenarios. The continual learning module distills the beneficial knowledge of the old network to the new network through warm-up and joint training to achieve a comprehensive and holistic memory of the target appearance. Combining the dynamic sample set can effectively balance the short-term memory and long-term memory of the model, and establish a near-complete target appearance description in the long-term dimension to cope with various challenging situations. Experimental results on the large-scale long-term benchmark datasets LaSOT and UAV20L show that the proposed method performs favourably against other state-of-the-art trackers.