Advanced Trackers Research Articles

Visual target tracking based on fractional – order bidirectional hybrid attentional feature fusion

This paper mainly designs and implements target tracking based on fractional – order feature fusion to solve tracking drift and tracking box jumping in complex scenes. Firstly, the starting point and overall structure of the model design were introduced. Secondly, in response to the low information utilization of the feature extraction network in the target tracking framework at the local scale of the target, a fractional – order node function based attention CNN hybrid attention extraction module was proposed to improve the robustness of target tracking. Finally, overall performance was quantitatively and qualitatively evaluated on multiple evaluation datasets with various advanced trackers. The results showed that the algorithm proposed in this paper had a high tracking advantage under severe changes in scale morphology, dynamic blurring, and similar interference attributes.

Temporal relation transformer for robust visual tracking with dual-memory learning

Temporal relation transformer for robust visual tracking with dual-memory learning

UAV Tracking via Saliency-Aware and Spatial–Temporal Regularization Correlation Filter Learning

Due to their great balance between excellent performance and high efficiency, discriminative correlation filter (DCF) tracking methods for unmanned aerial vehicles (UAVs) have gained much attention. Due to these correlations being capable of being efficiently computed in a Fourier domain by discrete Fourier transform (DFT), the DFT of an image has symmetry in the Fourier domain. However, DCF tracking methods easily generate unwanted boundary effects where the tracking object suffers from challenging situations, such as deformation, fast motion and occlusion. To tackle the above issue, this work proposes a novel saliency-aware and spatial–temporal regularized correlation filter (SSTCF) model for visual object tracking. First, the introduced spatial–temporal regularization helps build a more robust correlation filter (CF) and improve the temporal continuity and consistency of the model to effectively lower boundary effects and enhance tracking performance. In addition, the relevant objective function can be optimized into three closed-form subproblems which can be addressed by using the alternating direction method of multipliers (ADMM) competently. Furthermore, utilizing a saliency detection method to acquire a saliency-aware weight enables the tracker to adjust to variations in appearance and mitigate disturbances from the surroundings environment. Finally, we conducted numerous experiments based on three different benchmarks, and the results showed that our proposed model had better performance and higher efficiency compared to the most advanced trackers. For example, the distance precision (DP) score was 0.883, and the area under the curve (AUC) score was 0.676 on the OTB2015 dataset.

FQTrack:Object Tracking Method Based on a Feature-Enhanced Memory Network and Memory Quality Selection Mechanism

Visual object tracking technology is widely used in intelligent security, automatic driving and other fields, and also plays an important role in frontier fields such as human–computer interactions and virtual reality. The memory network improves the stability and accuracy of tracking by using historical frame information to assist in the positioning of the current frame in object tracking. However, the memory network is still insufficient in feature mining and the accuracy and robustness of the model may be reduced when using noisy observation samples to update it. In view of the above problems, we propose a new tracking framework, which uses the attention mechanism to establish a feature-enhanced memory network and combines cross-attention to aggregate the spatial and temporal context information of the target. The former introduces spatio-temporal adaptive attention and cross-spatial attention, embeds spatial location information into channels, realizes multi-scale feature fusion, dynamically emphasizes target location information, and obtains richer feature maps. The latter guides the tracker to focus on the area with the largest amount of information in the current frame to better distinguish the foreground and background. In addition, through the memory quality selection mechanism, the accuracy and richness of the feature samples are improved, thereby enhancing the adaptability and discrimination ability of the tracking model. Experiments on benchmark test sets such as OTB2015, TrackingNet, GOT-10k, LaSOT and UAV 123 show that this method achieves comparable performance with advanced trackers.

An Alternative Immune Cell Labeling System Based on the New Two-Dimensional Nanomaterials MXenes

Abstract Deepening our understanding of immune cell behavior is vital for creating safe, effective treatments, particularly as immune cell therapy, also known as cellular immunotherapy or cell-based immunotherapy, represents a groundbreaking approach in the treatment of various diseases, including cancer. Transition metal carbides and nitrides (MXenes), a novel family of 2D nanomaterials, show promise as advanced trackers for immune cells—key for precise diagnostics and therapies. Traditional cell labeling methods have stagnated due to limited chemical options, impeding progress in applied medicine. Moreover, these methods are incompatible with single-cell mass cytometry by time-of-flight (CyTOF), a globally adopted technology that improves classical flow cytometry. We propose an innovative solution utilizing MXenes to overcome these challenges. Our method, Label-free sINgle-cell tracKing of 2D matErials by mass cytometry (LINKED), leverages a novel, biocompatible, multiplexed, label-free detection approach via CyTOF and Mass Ion Beam Imaging by Time-of-Flight (MIBI-TOF). This technique overcomes chemical limitations and integrates seamlessly with CyTOF, allowing for nanomaterial detection and simultaneous measurement of diverse immune cell and tissue features. Our work promises to advance immunological research significantly, offering refined cell labeling and tracking techniques crucial for the advancement of translational medicine.

MASNet: mixed attention Siamese network for visual object tracking

Most Siamese-based trackers adopt correlation operation to perform similarity matching on feature fusion of template branch and search branch. However, the correlation operation directly uses the template feature to slide the window on the search area feature, and it is difficult to distinguish the target and background information when encountering similar target interference and background clutter, which can easily lead to tracking failure. In this paper, a mixed attention Siamese Network (MASNet) is proposed, which neatly integrates the modelling of global information and mutual information into the Siamese tracking framework. Specifically, through the learnable attention module, the local and global dependencies are adaptively integrated, and the rich context information is captured to improve the discrimination of the network for similar targets. After that, the correlation of the context information of the two branches is aggregated and adaptively encoded into the two branches, which makes up for the shortcomings of single correlation operation. In addition, the framework adopts a three-layer feature fusion method, which enables the tracker to better adapt to object deformation and similar object interference. Extensive experiments on OTB100, UAV123, GOT-10k and LaSOT show that the proposed tracking method achieves comparable performance to the advanced trackers.

EMAT: Efficient feature fusion network for visual tracking via optimized multi-head attention

EMAT: Efficient feature fusion network for visual tracking via optimized multi-head attention

Toward Class-Agnostic Tracking Using Feature Decorrelation in Point Clouds.

Single object tracking in point clouds has been attracting more and more attention owing to the presence of LiDAR sensors in 3D vision. However, existing methods based on deep neural networks mainly focus on training different models for different categories, which makes them unable to perform well in real-world applications when encountering classes unseen during the training phase. In this work, we investigate a more challenging task in LiDAR point clouds, namely class-agnostic tracking, where a general model is supposed to be learned to handle targets of both observed and unseen categories. In particular, we first investigate the class-agnostic performance of state-of-the-art trackers by exposing the unseen categories to them during testing. It is found that as the distribution shifts from observed to unseen classes, how to constrain the fused features between the template and the search region to maintain generalization is a key factor in class-agnostic tracking. Therefore, we propose a feature decorrelation method to address this problem, which eliminates the spurious correlations of the fused features through a set of learned weights, and further makes the search region consistent among foreground points and distinctive between foreground and background points. Experiments on KITTI and NuScenes demonstrate that the proposed method can achieve considerable improvements by benchmarking against the advanced trackers P2B and BAT, especially when tracking unseen objects.

Multi-Task Probabilistic Regression With Overlap Maximization for Visual Tracking

Recent researches made a breakthrough in visual tracking accuracy. Many trackers benefit from the object state representations and network loss functions, which mine the output space and improve the power of supervision, respectively. Probabilistic regression method models the noises and uncertainties in the annotations. However, advanced trackers with probabilistic regression are not studied sufficiently in the aspect of supervision and the aspect of robustness of evaluation maximization. In this paper, an overlap maximization network in the manner of probabilistic regression is proposed to improve the learning ability of the network and the discriminative ability in the evaluation maximization. Firstly, the probabilistic regression is extended with the intersection over union (IoU) evaluation, which is normalized as a probability density in the regression space. Secondly, the classification probability is added as a branch of the iterative evaluation module to improve the ability of distinguishing objects in the evaluation maximization. Moreover, the two branches are constructed into a joint probabilistic regression task of IoU evaluation, which makes the network learn from two types of ground truth and provide a consistent result with multi-branch outputs. For feature interpretation, the strip pooling network and the space-time memory network are introduced to encode long-range context and provide dynamic features, respectively. Compared to the state-of-the-art probabilistic regression trackers and other advanced trackers, the experiments show that the proposed tracker achieves outstanding performance across the six datasets, including GOT-10k, LaSOT, TrackingNet, UAV123, OTB-100 and VOT2018.

IoUformer: Pseudo-IoU prediction with transformer for visual tracking

IoUformer: Pseudo-IoU prediction with transformer for visual tracking

Sparse mixed attention aggregation network for multimodal images fusion tracking

Sparse mixed attention aggregation network for multimodal images fusion tracking

Object tracking based on siamese network with 3D attention and multiple graph attention

Object tracking based on siamese network with 3D attention and multiple graph attention

WebUAV-3M: A Benchmark for Unveiling the Power of Million-Scale Deep UAV Tracking.

Unmanned aerial vehicle (UAV) tracking is of great significance for a wide range of applications, such as delivery and agriculture. Previous benchmarks in this area mainly focused on small-scale tracking problems while ignoring the amounts of data, types of data modalities, diversities of target categories and scenarios, and evaluation protocols involved, greatly hiding the massive power of deep UAV tracking. In this article, we propose WebUAV-3M, the largest public UAV tracking benchmark to date, to facilitate both the development and evaluation of deep UAV trackers. WebUAV-3M contains over 3.3 million frames across 4,500 videos and offers 223 highly diverse target categories. Each video is densely annotated with bounding boxes by an efficient and scalable semi-automatic target annotation (SATA) pipeline. Importantly, to take advantage of the complementary superiority of language and audio, we enrich WebUAV-3M by innovatively providing both natural language specifications and audio descriptions. We believe that such additions will greatly boost future research in terms of exploring language features and audio cues for multi-modal UAV tracking. In addition, a fine-grained UAV tracking-under-scenario constraint (UTUSC) evaluation protocol and seven challenging scenario subtest sets are constructed to enable the community to develop, adapt and evaluate various types of advanced trackers. We provide extensive evaluations and detailed analyses of 43 representative trackers and envision future research directions in the field of deep UAV tracking and beyond. The dataset, toolkits, and baseline results are available at https://github.com/983632847/WebUAV-3M.

STFTrack: Spatio-Temporal-Focused Siamese Network for Infrared UAV Tracking

The rapid popularity of UAVs has encouraged the development of Anti-UAV technology. Infrared-detector-based visual tracking for UAVs provides an encouraging solution for Anti-UAVs. However, it still faces the problem of tracking instability caused by environmental thermal crossover and similar distractors. To address these issues, we propose a spatio-temporal-focused Siamese network for infrared UAV tracking, called STFTrack. This method employs a two-level target focusing strategy from global to local. First, a feature pyramid-based Siamese backbone is constructed to enhance the feature expression of infrared UAVs through cross-scale feature fusion. By combining template and motion features, we guide prior anchor boxes towards the suspicious region to enable adaptive search region selection, thus effectively suppressing background interference and generating high-quality candidates. Furthermore, we propose an instance-discriminative RCNN based on metric learning to focus on the target UAV among candidates. By measuring calculating the feature distance between the candidates and the template, it assists in discriminating the optimal target from the candidates, thus improving the discrimination of the proposed method to infrared UAV. Extensive experiments on the Anti-UAV dataset demonstrate that the proposed method achieves outstanding performance for infrared tracking, with 91.2% precision, 66.6% success rate, and 67.7% average overlap accuracy, and it exceeded the baseline algorithm by 2.3%, 2.7%, and 3.5%, respectively. The attribute-based evaluation demonstrates that the proposed method achieves robust tracking effects on challenging scenes such as fast motion, thermal crossover, and similar distractors. Evaluation on the LSOTB-TIR dataset shows that the proposed method reaches a precision of 77.2% and a success rate of 63.4%, outperforming other advanced trackers.

Transformer tracking with multi-scale dual-attention

Transformer-based trackers greatly improve tracking success rate and precision rate. Attention mechanism in Transformer can fully explore the context information across successive frames. Nevertheless, it ignores the equally important local information and structured spatial information. And irrelevant regions may also affect the template features and search region features. In this work, a multi-scale feature fusion network is designed with box attention and instance attention in Encoder–Decoder architecture based on Transformer. After extracting features, the local information and structured spatial information is learnt by multi-scale box attention, and the global context information is explored by instance attention. Box attention samples grid features from the region of interest. Therefore, it effectively focuses on the region of interest (ROI) and avoids the influence of irrelevant regions in feature extraction. At the same time, instance attention can also pay attention to the context information across successive frames, and avoid falling into local optimum. The long-range feature dependencies are learned in this stage. Extensive experiments are conducted on six challenging tracking datasets to demonstrate the superiority of the proposed tracker MDTT, including UAV123, GOT-10k, LaSOT, VOT2018, TrackingNet, and NfS. In particular, the proposed tracker achieves AUC score of 64.7 % on LaSOT, 78.1 % on TrackingNet and precision score of 89.2 % on UAV123, which outperforms the baseline and most recent advanced trackers.

SiamST: Siamese network with spatio-temporal awareness for object tracking

SiamST: Siamese network with spatio-temporal awareness for object tracking

TMTNet: A Transformer-Based Multimodality Information Transfer Network for Hyperspectral Object Tracking

Hyperspectral video with spatial and spectral information has great potential to improve object tracking performance. However, the limited hyperspectral training samples hinder the development of hyperspectral object tracking. Since hyperspectral data has multiple bands, from which any three bands can be extracted to form pseudocolor images, we propose a Transformer-based multimodality information transfer network (TMTNet), aiming to improve the tracking performance by efficiently transferring the information of multimodality data composed of RGB and hyperspectral in the hyperspectral tracking process. The multimodality information needed to be transferred mainly includes the RGB and hyperspectral multimodality fusion information and the RGB modality information. Specifically, we construct two subnetworks to transfer the multimodality fusion information and the robust RGB visual information, respectively. Among them, the multimodality fusion information transfer subnetwork is designed based on the dual Siamese branch structure. The subnetwork employs the pretrained RGB tracking model as the RGB branch to guide the training of the hyperspectral branch with little training samples. The RGB modality information transfer subnetwork is designed based on a pretrained RGB tracking model with good performance to improve the tracking network’s generalization and accuracy in unknown complex scenes. In addition, we design an information interaction module based on Transformer in the multimodality fusion information transfer subnetwork. The module can fuse multimodality information by capturing the potential interaction between different modalities. We also add a spatial optimization module to TMTNet, which further optimizes the object position predicted by the subject network by fully retaining and utilizing detailed spatial information. Experimental results on the only available hyperspectral tracking benchmark dataset show that the proposed TMTNet tracker outperforms the advanced trackers, demonstrating the effectiveness of this method.

Adaptive temporal feature modeling for visual tracking via cross-channel learning

Adaptive temporal feature modeling for visual tracking via cross-channel learning

Anchor-Free Tracker Based on Space-Time Memory Network

In the visual object tracking task, the existing trackers cannot well solve the appearance deformation, occlusion, and similar object interference, etc. To address these problems, this article proposes a new Anchor-free Tracker based on Space-time Memory Network (ATSMN). In this work, we innovatively use the space-time memory network, memory feature fusion network, and transformer feature cross fusion network. Through the synergy of above-mentioned innovations, tracker can make full use of temporal context information in the memory frames related to the object and better adapt to the appearance change of the object, which can obtain accurate classification and regression results. Extensive experimental results on challenging benchmarks show that ATSMN can achieve the SOTA level tracking performance compared with other advanced trackers.

Context‐aware Siamese network for object tracking

At present, temporal and spatial contexts are widely used to improve the adaptability of a tracker. However, most existing methods usually focus on one aspect of the temporal or spatial context and rarely exploit them simultaneously. In this paper, a context-aware Siamese Network (CSNet) is proposed, which skilfully integrates the modelling of temporal and spatial context into the Siamese tracking framework. Specifically, CSNet consists of a context-based channel attention module and a context-based cross-attention module. The former aggregates spatial context information from different channels and dynamically emphasizes target features, which makes it easier for the tracker to distinguish the target from the background. The latter propagates the temporal context from the previous frames to the current frame to establish the part-level relationship between the search region and the historical target state, which enables the tracker better adapt to the target deformation. In addition, to further mine context information, the CSNet is equipped with a state-aware strategy to control the contribution of different context information in tracking. Extensive experiments on OTB2015, UAV123, GOT-10k, LaSOT, and TrackingNet show that the proposed tracking method achieves comparable performance to the advanced trackers.