Siamese Network For Visual Tracking Research Articles

SiamATA: an asymmetric target-aware and frequency domain task-aware Siamese network for visual tracking

SiamATA: an asymmetric target-aware and frequency domain task-aware Siamese network for visual tracking

RETRACTED ARTICLE: SCSiam: spatial and channel attentional Siamese network for visual tracking

RETRACTED ARTICLE: SCSiam: spatial and channel attentional Siamese network for visual tracking

Distilled Siamese Networks for Visual Tracking

In recent years, Siamese network based trackers have significantly advanced the state-of-the-art in real-time tracking. Despite their success, Siamese trackers tend to suffer from high memory costs, which restrict their applicability to mobile devices with tight memory budgets. To address this issue, we propose a distilled Siamese tracking framework to learn small, fast and accurate trackers (students), which capture critical knowledge from large Siamese trackers (teachers) by a teacher-students knowledge distillation model. This model is intuitively inspired by the one teacher versus multiple students learning method typically employed in schools. In particular, our model contains a single teacher-student distillation module and a student-student knowledge sharing mechanism. The former is designed using a tracking-specific distillation strategy to transfer knowledge from a teacher to students. The latter is utilized for mutual learning between students to enable in-depth knowledge understanding. Extensive empirical evaluations on several popular Siamese trackers demonstrate the generality and effectiveness of our framework. Moreover, the results on five tracking benchmarks show that the proposed distilled trackers achieve compression rates of up to 18× and frame-rates of 265 FPS, while obtaining comparable tracking accuracy compared to base models.

SGAT: Shuffle and graph attention based Siamese networks for visual tracking.

Siamese-based trackers have achieved excellent performance and attracted extensive attention, which regard the tracking task as a similarity learning between the target template and search regions. However, most Siamese-based trackers do not effectively exploit correlations of the spatial and channel-wise information to represent targets. Meanwhile, the cross-correlation is a linear matching method and neglects the structured and part-level information. In this paper, we propose a novel tracking algorithm for feature extraction of target templates and search region images. Based on convolutional neural networks and shuffle attention, the tracking algorithm computes the similarity between the template and a search region through a graph attention matching. The proposed tracking algorithm exploits the correlations between the spatial and channel-wise information to highlight the target region. Moreover, the graph matching can greatly alleviate the influences of appearance variations such as partial occlusions. Extensive experiments demonstrate that the proposed tracking algorithm achieves excellent tracking results on multiple challenging benchmarks. Compared with other state-of-the-art methods, the proposed tracking algorithm achieves excellent tracking performance.

A ConvNext-Based and Feature Enhancement Anchor-Free Siamese Network for Visual Tracking

Existing anchor-based Siamese trackers rely on the anchor’s design to predict the scale and aspect ratio of the target. However, these methods introduce many hyperparameters, leading to computational redundancy. In this paper, to achieve outstanding network efficiency, we propose a ConvNext-based anchor-free Siamese tracking network (CAFSN), which employs an anchor-free design to increase network flexibility and versatility. In CAFSN, to obtain an appropriate backbone network, the state-of-the-art ConvNext network is applied to the visual tracking field for the first time by improving the network stride and receptive field. Moreover, A central confidence branch based on Euclidean distance is offered to suppress low-quality prediction frames in the classification prediction network of CAFSN for robust visual tracking. In particular, we discuss that the Siamese network cannot establish a complete identification model for the tracking target and similar objects, which negatively impacts network performance. We build a Fusion network consisting of crop and 3Dmaxpooling to better distinguish the targets and similar objects’ abilities. This module uses 3DMaxpooling to select the highest activation value to improve the difference between it and other similar objects. Crop unifies the dimensions of different features and reduces the amount of computation. Ablation experiments demonstrate that this module increased success rates by 1.7% and precision by 0.5%. We evaluate CAFSN on challenging benchmarks such as OTB100, UAV123, and GOT-10K, validating advanced performance in noise immunity and similar target identification with 58.44 FPS in real time.

Triple attention and global reasoning Siamese networks for visual tracking

Triple attention and global reasoning Siamese networks for visual tracking

Dual Attentional Siamese Network for Visual Tracking

Attention mechanism in siamese-based trackers has drawn great attention on visual object tracking. However, we experimentally discover that the feature information after being enhanced by attention models with different styles may be interfered with each other for visual tracking. To address this issue, we propose a Dual Attentional Siamese Network (DASNet) to separate channel attention and spatial attention to alleviate the feature interference. Specifically, we propose a dual attention module to enhance the acquired semantic feature expression ability. Especially in spatial dimension, we introduce spatial pyramid attention on multiscale feature pyramid to capture the long-range interdependencies with different granularity. Moreover, we exploit adaptive decision fusion strategy to integrate the outputs from two parallel tracking-heads along two separate dimensions, channel and spatial, so as to effectively take the advantages of attention models with different styles. Extensive experiments are conducted on publicly challenging benchmarks, where the proposed DASNet achieves new state-of-the-art results, by 0.4299 → 0.4515 and 0.3835 → 0.3934 EAO compared with the strong baseline SiamMask (Wang et al., 2019) on VOT 2016 and VOT 2018. Our source code is available at: https://github.com/jwma0725/DASNet.

A flow-guided self-calibration Siamese network for visual tracking

A flow-guided self-calibration Siamese network for visual tracking

Learning bi-grained cross-correlation siamese networks for visual tracking

Learning bi-grained cross-correlation siamese networks for visual tracking

Learning Temporal-Correlated and Channel- Decorrelated Siamese Networks for Visual Tracking

Recently, Siamese network based trackers have attracted growing popularity in visual tracking, which tackle the tracking by template matching between the initial template and successive search regions. The initial template patch is generally encoded into a convolutional feature for matching. However, the limited representational capability of the template feature limits the tracking accuracy. Besides, this fixed representation also fails to adapt to the target appearance changes. To alleviate these issues, we improve the Siamese trackers by introducing temporal correlation and channel decorrelation mechanisms. On the one hand, we consider the channel-wise correlations between the initial and historical template features to adaptively aggregate informative channel-wise representations for template update. On the other hand, we propose a decorrelation regularization to weaken the channel-wise correlations of individual template features. By end-to-end training, we learn a more complete and adaptive template for accurate object tracking. We demonstrate the generality of our approach by applying it to two prevalent Siamese trackers, i.e., SiamFC and SiamRPN. Extensive experiments on seven benchmark datasets verify the effectiveness of our method.

Two-stage aware attentional Siamese network for visual tracking

Siamese networks have achieved great success in visual tracking with the advantages of speed and accuracy. However, how to track an object precisely and robustly still remains challenging. One reason is that multiple types of features are required to achieve good precision and robustness, which are unattainable by a single training phase. Moreover, Siamese networks usually struggle with online adaption problem. In this paper, we present a novel two-stage aware attentional Siamese network for tracking (Ta-ASiam). Concretely, we first propose a position-aware and an appearance-aware training strategy to optimize different layers of Siamese network. By introducing diverse training patterns, two types of required features can be captured simultaneously. Then, following the rule of feature distribution, an effective feature selection module is constructed by combining both channel and spatial attention networks to adapt to rapid appearance changes of the object. Extensive experiments on various latest benchmarks have well demonstrated the effectiveness of our method, which significantly outperforms state-of-the-art trackers.

Ensemble learning with siamese networks for visual tracking

Ensemble learning (EL) is an effective and commonly used technique to improve visual tasks’ accuracy, such as classification and detection. However, EL is rarely used in visual tracking. To fill this knowledge gap, we first have completed some research to investigate why knowledge distillation was ineffective in visual tracking tasks. Comparing the difference between the classification and visual tracking, conclusions are given: (i) Numerous simple negative examples are redundant, while only a few hard negative samples are valid for visual tracking knowledge distillation. (ii) The hint knowledge flows differently between classification and visual tracking. To solve the above problems, we design two new loss functions and integrate them into the proposed Ensemble Learning (EL) framework that can be employed in Siamese architectures such as SiamFC, SiamRPN, SiamFC+, and SiamRPN+. The EL treats two Siamese networks as students and enables them to learn collaboratively. A better solution is yielded by the EL framework than training students individually. Experiments on OTB-2013, OTB-2015, VOT2015, VOT2016, VOT2017, VOT2018, LaSOT and TrackingNet have verified the effectiveness of our proposed technique on boosting the performance for the four Siamese algorithms. The EL-SiamRPN+ achieves leading performance in the challenges.

End-to-End Correlation Tracking With Enhanced Multi-Level Feature Fusion

Discriminative correlation filters (DCF) have drawn increasing interest in visual tracking. In particular, a few recent works treat DCF as a special layer and add it into a Siamese network for visual tracking. However, most of them adopt shallow networks to learn target representations, which lack robust semantic information of deeper layers and make these works fail to handle significant appearance changes. In this paper, we design a novel Siamese network to fuse high-level semantic features and low-level spatial detail features for correlation tracking. Specifically, to introduce more semantic information into low-level features, we specially design a residual semantic embedding module to adaptively involve more semantic information from high-level features to guide the feature fusion. Furthermore, we adopt an effective and efficient channel attention mechanism to filter out noise information and make the network focus more on valuable features that are beneficial for visual tracking. The overall architecture is trained end-to-end offline to adaptively learn target representations, which are not only enabled to encode high-level semantic features and low-level spatial detail features, but also closely related to correlation filters. Experimental results on widely used OTB2013, OTB2015, VOT2016, TC-128, and UAV123 benchmarks show that our proposed tracker performs favorably against several state-of-the-art trackers.

Manifold Siamese Network: A Novel Visual Tracking ConvNet for Autonomous Vehicles

Visual tracking is a vital component of autonomous driving perception system. Siamese networks have achieved great success in both accuracy and speed for visual tracking tasks. These Siamese trackers share a similar framework in which each tracker consists of two network branches for exploring semantic information. However, the performance of Siamese trackers is limited by an insufficient semantic template and an unsatisfactory updating strategy. To tackle these problems, we propose a manifold Siamese network for visual tracking that can simultaneously utilize semantic and geometric information. A manifold sample pool is constructed to exploit the manifold structure of image object sequences. This sample pool is dynamically learned via a fast Gaussian mixture model (GMM). After obtaining a manifold sample template, we design a deep architecture based on a correlation filter (CF) network and append a novel manifold feature branch. The network remains fully convolutional and can train a template to discriminate exemplar image and arbitrarily size search image. Then, a triplet occlusion score function cooperates with an effective update method that is established to prevent model drift. Extensive experiments show that the proposed tracking algorithm performs favorably compared with the state-of-the-art methods on three standard benchmark datasets at a high framerate, which is very suitable for autonomous driving.

Initial Matting-Guided Visual Tracking With Siamese Network

Fully-convolutional Siamese networks for visual tracking have drawn great attention in balancing tracking accuracy and speed. However, there is still some inherent inaccuracy with advanced trackers, since they only learn a general matching model from large scale datasets by off-line training. This generates the target template without sufficient discriminant information and does not adapt well to the current tracking sequence. In this paper, we introduce the channel attention mechanism into the network to better learn the matching model and, during the online tracking phase, we design an initial matting guidance strategy in which: 1) the superpixel matting algorithm is applied to extract the target foreground in the initial frame, and 2) the matted image with foreground only is fed into the network and fused with the original image feature. Under matting guidance, the fused target template has more details for representation of target appearance and more structural information from superpixels for robust tracking. The experimental results on object tracking benchmark (OTB) show that our approach achieves excellent performance while it also provides real-time tracking speed.