SiamSTC: Updatable Siamese tracking network via Spatio-Temporal Context

Abstract

To adapt to the target appearance, some trackers focus only on learning the target model online with the spatial context or only on learning an adaptive template with the temporal context. However, these trackers do not fully utilize spatial and temporal contexts and therefore cannot robustly adapt to sharp appearance variations in challenging situations, such as deformation, rotation, out-of-view, and background clutter. To address the above problem, we propose a novel online updatable Siamese tracker (SiamSTC) that takes advantage of the spatiotemporal context to be aware of the appearance variations in the target to improve the accuracy and robustness of tracking. Specifically, we build a memory bank to collect the spatial and temporal contexts of the target, and design a template update module to learn an adaptive target template using the historical knowledge in memory. We propose a novel update mechanism to ensure that high-quality samples are utilized to update the memory bank. In addition, we propose a target alignment module to explore the fine-grained relationships between the adaptive template and the search image, and locate discriminative regions for tracking based on fine-grained relationships. The proposed method achieves state-of-the-art tracking performance on seven challenging benchmarks, including OTB100, LaSOT, TC128, GOT10k, VOT2016, VOT2018, and VOT2019.