Target Appearance Changes Research Articles

Siamese Tracking Network with Multi-attention Mechanism

Object trackers based on Siamese networks view tracking as a similarity-matching process. However, the correlation operation operates as a local linear matching process, limiting the tracker’s ability to capture the intricate nonlinear relationship between the template and search region branches. Moreover, most trackers don’t update the template and often use the first frame of an image as the initial template, which will easily lead to poor tracking performance of the algorithm when facing instances of deformation, scale variation, and occlusion of the tracking target. To this end, we propose a Simases tracking network with a multi-attention mechanism, including a template branch and a search branch. To adapt to changes in target appearance, we integrate dynamic templates and multi-attention mechanisms in the template branch to obtain more effective feature representation by fusing the features of initial templates and dynamic templates. To enhance the robustness of the tracking model, we utilize a multi-attention mechanism in the search branch that shares weights with the template branch to obtain multi-scale feature representation by fusing search region features at different scales. In addition, we design a lightweight and simple feature fusion mechanism, in which the Transformer encoder structure is utilized to fuse the information of the template area and search area, and the dynamic template is updated online based on confidence. Experimental results on publicly tracking datasets show that the proposed method achieves competitive results compared to several state-of-the-art trackers.

UAV Complex-Scene Single-Target Tracking Based on Improved Re-Detection Staple Algorithm

With the advancement of remote sensing technology, the demand for the accurate monitoring and tracking of various targets utilizing unmanned aerial vehicles (UAVs) is increasing. However, challenges such as object deformation, motion blur, and object occlusion during the tracking process could significantly affect tracking performance and ultimately lead to tracking drift. To address this issue, this paper introduces a high-precision target-tracking method with anomaly tracking status detection and recovery. An adaptive feature fusion strategy is proposed to improve the adaptability of the traditional sum of template and pixel-wise learners (Staple) algorithm to changes in target appearance and environmental conditions. Additionally, the Moth Flame Optimization (MFO) algorithm, known for its strong global search capability, is introduced as a re-detection algorithm in case of tracking failure. Furthermore, a trajectory-guided Gaussian initialization technique and an iteration speed update strategy are proposed based on sexual pheromone density to enhance the tracking performance of the introduced re-detection algorithm. Comparative experiments conducted on UAV123 and UAVDT datasets demonstrate the excellent stability and robustness of the proposed algorithm.

Explicit Visual Prompts for Visual Object Tracking

How to effectively exploit spatio-temporal information is crucial to capture target appearance changes in visual tracking. However, most deep learning-based trackers mainly focus on designing a complicated appearance model or template updating strategy, while lacking the exploitation of context between consecutive frames and thus entailing the when-and-how-to-update dilemma. To address these issues, we propose a novel explicit visual prompts framework for visual tracking, dubbed EVPTrack. Specifically, we utilize spatio-temporal tokens to propagate information between consecutive frames without focusing on updating templates. As a result, we cannot only alleviate the challenge of when-to-update, but also avoid the hyper-parameters associated with updating strategies. Then, we utilize the spatio-temporal tokens to generate explicit visual prompts that facilitate inference in the current frame. The prompts are fed into a transformer encoder together with the image tokens without additional processing. Consequently, the efficiency of our model is improved by avoiding how-to-update. In addition, we consider multi-scale information as explicit visual prompts, providing multiscale template features to enhance the EVPTrack's ability to handle target scale changes. Extensive experimental results on six benchmarks (i.e., LaSOT, LaSOText, GOT-10k, UAV123, TrackingNet, and TNL2K.) validate that our EVPTrack can achieve competitive performance at a real-time speed by effectively exploiting both spatio-temporal and multi-scale information. Code and models are available at https://github.com/GXNU-ZhongLab/EVPTrack.

Lightweight Spatial-Temporal Contextual Aggregation Siamese Network for Unmanned Aerial Vehicle Tracking

Benefiting from the powerful feature extraction capability of deep learning, the Siamese tracker stands out due to its advanced tracking performance. However, constrained by the complex backgrounds of aerial tracking, such as low resolution, occlusion, similar objects, small objects, scale variation, aspect ratio change, deformation and limited computational resources, efficient and accurate aerial tracking is still difficult to realize. In this work, we design a lightweight and efficient adaptive temporal contextual aggregation Siamese network for aerial tracking, which is designed with a parallel atrous module (PAM) and adaptive temporal context aggregation model (ATCAM) to mitigate the above problems. Firstly, by using a series of atrous convolutions with different dilation rates in parallel, the PAM can simultaneously extract and aggregate multi-scale features with spatial contextual information at the same feature map, which effectively improves the ability to cope with changes in target appearance caused by challenges such as aspect ratio change, occlusion, scale variation, etc. Secondly, the ATCAM adaptively introduces temporal contextual information to the target frame through the encoder-decoder structure, which helps the tracker resist interference and recognize the target when it is difficult to extract high-resolution features such as low-resolution, similar objects. Finally, experiments on the UAV20L, UAV123@10fps and DTB70 benchmarks demonstrate the impressive performance of the proposed network running at a high speed of over 75.5 fps on the NVIDIA 3060Ti.

RGB-T Tracking with Template-Bridged Search Interaction and Target-Preserved Template Updating.

The goal of RGB-Thermal (RGB-T) tracking is to utilize the synergistic and complementary strengths of RGB and TIR modalities to enhance tracking in diverse situations, with cross-modal interaction being a crucial element. Earlier methods often simply combine the features of the RGB and TIR search frames, leading to a coarse interaction that also introduced unnecessary background noise. Many other approaches sample candidate boxes from search frames and apply different fusion techniques to individual pairs of RGB and TIR boxes, which confines cross-modal interactions to local areas and results in insufficient context modeling. Additionally, mining video temporal contexts is also under-explored in RGB-T tracking. To alleviate these limitations, we propose a novel Template-Bridged Search region Interaction (TBSI) module that exploits templates as the medium to bridge the cross-modal interaction between RGB and TIR search regions by gathering and distributing target-relevant object and environment contexts. An Illumination Guided Fusion (IGF) module is designed to adaptively fuse RGB and TIR search region tokens with a global illumination factor. Furthermore, in the inference stage, we also propose an efficient Target-Preserved Template Updating (TPTU) strategy, leveraging the temporal context within video sequences to accommodate the target's appearance change. Our proposed modules are integrated into a ViT backbone for joint feature extraction, search-template matching, and cross-modal interaction. Extensive experiments on three popular RGB-T tracking benchmarks demonstrate our method achieves new state-of-the-art performances. Code is available at https://github.com/RyanHTR/TBSI.

Exploring reliable infrared object tracking with spatio-temporal fusion transformer

Currently, most thermal infrared (TIR) trackers rely on feature matching between the search image and a fixed template cropped from the first frame. Some Siam-based TIR trackers with a template update mechanism introduce historical prediction information in the temporal dimension through correlation filters. However, their feature characterization capability is inadequate to resist target scale variations, appearance changes, and occlusion. To address this challenge, we explore a novel spatio-temporal fusion Transformer (STFT) model to realize robust TIR object tracking. Our approach involves a Transformer-based encoder–decoder that fuses spatio-temporal information. Specifically, we design a dynamic template update strategy based on salient points feature(SPF) representation, which allows the model to leverage the most powerful spatio-temporal information by retrieving multiple salient points on the target image. To further fortify the dynamic template update strategy, we propose an IoU-Aware target state estimation head that utilizes the joint representation of target classification and localization. An IoU-Aware criterion is developed for quality estimation of the dynamic template. The proposed STFT-Net approach has been put to the evaluation on three challenging benchmarks, with extensive experimental results showcasing its superior performance in contrast to acclaimed tracking algorithms. The code is available at https://github.com/qinxin-wh/STFT-Net.

Interframe Saliency Transformer and Lightweight Multidimensional Attention Network for Real-Time Unmanned Aerial Vehicle Tracking

UAV visual-object-tracking technology based on Siamese neural networks has great scientific research and practical application value, and it is widely used in geological surveying, reconnaissance monitoring, and environmental monitoring. Due to the limited onboard computational resources and complex real-world environments of drones, most of the existing tracking systems based on Siamese neural networks struggle to combine excellent performance with high efficiency. Therefore, the key issue is to study how to improve the accuracy of target tracking under the challenges of real-time performance and the above factors. In response to this problem, this paper proposes a real-time UAV tracking system based on interframe saliency transformer and lightweight multidimensional attention network (SiamITL). Specifically, interframe saliency transformer is used to continuously perceive spatial and temporal information, making the network more closely related to the essence of the tracking task. Additionally, a lightweight multidimensional attention network is used to better capture changes in both target appearance and background information, improving the ability of the tracker to distinguish between the target and background. SiamITL is effective and efficient: extensive comparative experiments and ablation experiments have been conducted on multiple aerial tracking benchmarks, demonstrating that our algorithm can achieve more robust feature representation and more accurate target state estimation. Among them, SiamITL achieved success and accuracy rates of 0.625 and 0.818 in the UAV123 benchmark, respectively, demonstrating a certain level of leadership in this field. Furthermore, SiamITL demonstrates the potential for real-time operation on the embedded platform Xavier, highlighting its potential for practical application in real-world scenarios.

DASTSiam: Spatio‐temporal fusion and discriminative enhancement for Siamese visual tracking

AbstractThe use of deep neural networks has revolutionised object tracking tasks, and Siamese trackers have emerged as a prominent technique for this purpose. Existing Siamese trackers use a fixed template or template updating technique, but it is prone to overfitting, lacks the capacity to exploit global temporal sequences, and cannot utilise multi‐layer features. As a result, it is challenging to deal with dramatic appearance changes in complicated scenarios. Siamese trackers also struggle to learn background information, which impairs their discriminative ability. Hence, two transformer‐based modules, the Spatio‐Temporal Fusion (ST) module and the Discriminative Enhancement (DE) module, are proposed to improve the performance of Siamese trackers. The ST module leverages cross‐attention to accumulate global temporal cues and generates an attention matrix with ST similarity to enhance the template's adaptability to changes in target appearance. The DE module associates semantically similar points from the template and search area, thereby generating a learnable discriminative mask to enhance the discriminative ability of the Siamese trackers. In addition, a Multi‐Layer ST module (ST + ML) was constructed, which can be integrated into Siamese trackers based on multi‐layer cross‐correlation for further improvement. The authors evaluate the proposed modules on four public datasets and show comparative performance compared to existing Siamese trackers.

SiamST: Siamese network with spatio-temporal awareness for object tracking

Recently, Siamese trackers have achieved remarkable tracking performance. However, challenges such as accurate feature representation of targets with different spatial regions and the utilization of diverse target temporal states still need to be addressed. Here, we proposed a Siamese network with spatio-temporal awareness called SiamST. The standard square convolutional kernels and the single feature matching operation hardly represent the targets with different shapes accurately. Therefore, we designed a refined region fusion module that combines multiple convolutional kernels to fit targets with different aspect ratios. Furthermore, we proposed a multi-granularity matching module to obtain more robust feature matching results by combining fine-grained and coarse-grained matching results. However, most existing Siamese trackers do not adequately employ target temporal states. They usually only update the templates, which automatically causes motion information loss. Therefore, we built dynamic templates by screening high-quality samples to describe the target appearance changes accurately. In addition, we designed a trend guidance module to adjust the location prior constraint appropriately to match the tracking results to the target's motion trajectory. Extensive experimental results on eight tracking benchmarks demonstrate the competitive performance of SiamST compared to many advanced trackers.

A Siamese tracker with "dynamic-static" dual-template fusion and dynamic template adaptive update.

In recent years, visual tracking algorithms based on Siamese networks have attracted attention for their desirable balance between speed and accuracy. The performance of such tracking methods relies heavily on target templates. Static templates cannot cope with the adverse effects of target appearance change. The dynamic template method, with a template update mechanism, can adapt to the change in target appearance well, but it also causes new problems, which may lead the template to be polluted by noise. Based on the DaSiamRPN and UpdateNet template update networks, a Siamese tracker with "dynamic-static" dual-template fusion and dynamic template adaptive update is proposed in this paper. The new method combines a static template and a dynamic template that is updated in real time for object tracking. An adaptive update strategy was adopted when updating the dynamic template, which can not only help adjust to the changes in the object appearance, but also suppress the adverse effects of noise interference and contamination of the template. The experimental results showed that the robustness and EAO of the proposed method were 23% and 9.0% higher than those of the basic algorithm on the VOT2016 dataset, respectively, and that the precision and success were increased by 0.8 and 0.4% on the OTB100 dataset, respectively. The most comprehensive real-time tracking performance was obtained for the above two large public datasets.

Learning a Deep Ensemble Network With Band Importance for Hyperspectral Object Tracking.

Attributing to material identification ability powered by a large number of spectral bands, hyperspectral videos (HSVs) have great potential for object tracking. Most hyperspectral trackers employ manually designed features rather than deeply learned features to describe objects due to limited available HSVs for training, leaving a huge gap to improve the tracking performance. In this paper, we propose an end-to-end deep ensemble network (SEE-Net) to address this challenge. Specifically, we first establish a spectral self-expressive model to learn the band correlation, indicating the importance of a single band in forming hyperspectral data. We parameterize the optimization of the model with a spectral self-expressive module to learn the nonlinear mapping from input hyperspectral frames to band importance. In this way, the prior knowledge of bands is transformed into a learnable network architecture, which has high computational efficiency and can fast adapt to the changes of target appearance because of no iterative optimization. The band importance is further exploited from two aspects. On the one hand, according to the band importance, each frame of HSVs is divided into several three-channel false-color images which are then used for deep feature extraction and location. On the other hand, based on the band importance, the importance of each false-color image is computed, which is then used to assemble the tracking results from individual false-color images. In this way, the unreliable tracking caused by false-color images of low importance can be suppressed to a large extent. Extensive experimental results show that SEE-Net performs favorably against the state-of-the-art approaches. The source code will be available at https://github.com/hscv/SEE-Net.

Visual Object Tracking Based on Modified LeNet-5 and RCCF

The field of object tracking has recently made significant progress. Particularly, the performance results in both deep learning and correlation filters, based trackers achieved effective tracking performance. Moreover, there are still some difficulties with object tracking for example illumination and deformation (DEF). The precision and accuracy of tracking algorithms suffer from the effects of such occurrences. For this situation, finding a solution is important. This research proposes a new tracking algorithm to handle this problem. The features are extracted by using Modified LeNet-5, and the precision and accuracy are improved by developing the Real-Time Cross-modality Correlation Filtering method (RCCF). In Modified LeNet-5, the visual tracking performance is improved by adjusting the number and size of the convolution kernels in the pooling and convolution layers. The high-level, middle-level, and handcraft features are extracted from the modified LeNet-5 network. The handcraft features are used to determine the specific location of the target because the handcraft features contain more spatial information regarding the visual object. The LeNet features are more suitable for a target appearance change in object tracking. Extensive experiments were conducted by the Object Tracking Benchmarking (OTB) databases like OTB50 and OTB100. The experimental results reveal that the proposed tracker outperforms other state-of-the-art trackers under different problems. The experimental simulation is carried out in python. The overall success rate and precision of the proposed algorithm are 93.8% and 92.5%. The average running frame rate reaches 42 frames per second, which can meet the real-time requirements.

Template pool updating and feature combination object tracking

In the target tracking task, most Siamese network target tracking algorithms only use the template information of the first frame and do not make good use of the feature information of the subsequent frames, which shows poor robustness and target drift in dealing with target occlusion and appearance change in complex scenes. In this regard, an online template pool update module is established based on the SiamFC++ algorithm to store the information of reliable template frames, and an accuracy and necessity judgment mechanism of template update is proposed to improve the reliability of the final template. In addition, a feature combination module is proposed to capture the information of different scale regions and obtain more scale information through an asymmetric dilated convolution layer. Experiments and analysis are conducted on OTB2015 and VOT2018 datasets, and the proposed algorithm improves the accuracy and success rate by 1.7% and 1.7%, respectively, on OTB2015 compared with the SiamFC++ algorithm; the accuracy and average overlap rate improve by 0.8% and 6.1%, respectively, on VOT2018, and the robustness decreases from 0.169 to 0.140, and the tracker more stable.

The ability to infer witness accuracy from high-confidence lineup identifications is undermined by the appearance-change instruction and target appearance change.

The ability to infer witness accuracy from high-confidence lineup identifications is undermined by the appearance-change instruction and target appearance change.

Research on Athlete Detection Method Based on Visual Image and Artificial Intelligence System.

Pedestrian detection and tracking based on computer vision has gradually become an international pattern recognition, which is one of the most active research topics in the field of computer vision and artificial intelligence. Using the theoretical results in the field of pattern recognition and computer vision technology, we are committed to detect and track pedestrians from video sequences. In addition to computer vision-based passer-by detection and tracking technology as the key, in the advanced computer vision action and analysis, it has a direct impact on the accuracy and robustness of its understanding. We analyzed various targets, such as subsequent recognition motion and pedestrian motion, and described them as high-level application processing, such as action understanding. In addition, because of the unique texture of human clothes compared with the surrounding natural landscape, they are highly “prominent” from the perspective of human visual system, and they are particularly prominent in the peripheral part of human contact with the background. In this paper, a binary function based on importance is proposed. As the space representation of image itself is not sensitive to noise and local signal, space representation is used. In addition, as an observation model, it can reduce the adverse effects of background noise and local noise on the tracking algorithm. Through the function block tracking, the pedestrian's body can be tracked in detail. At the same time, the color band learning method is used to update the target template online to deal with the changes of target appearance caused by sunshine, pedestrian posture, and other factors. According to the experimental results, even if the appearance and posture of pedestrians change greatly, it has a stable tracking effect.

Dual integration of multi‐model with spatial‐temporal occlusion‐awareness for visual object tracking

Visual object tracking is a complicated problem due to varied challenges such as occlusion, background clutter, target appearance changes, abrupt target motion, and illumination variations. Different tracking algorithms perform well on different challenges because of their unique strengths. In this paper, a dual-integration framework that integrates the strengths of multiple models while avoiding their weaknesses is proposed. In the proposed framework, several evaluation criteria and multiple models updated via different processes are combined. To distinguish the target from the distractors caused by different criteria, the motion dynamic model and the forward-backward analysis is introduced to provide information that is complementary to appearance information. In addition, for discriminating occlusion, a spatial-temporal occlusion-aware approach is further proposed. The detected occlusion results are applied in avoiding contamination of the appearance model. Extensive experiments on multiple benchmarks demonstrate that this proposed method improves the overlap rate of a hand-crafted feature-based tracker with relative gains of 4%, 1.6%, 1.9%, 2.2% and 8.5% on OTB-2015, OTB-2013, Temple-Color, UAV123 and UAV20L, respectively. Also, the experimental results demonstrate that our approach outperforms a deep feature-based tracker in overlap rate by 1.4%, 1.6%, 2.3%, 2.5% and 3.5% on OTB-2015, OTB-2013, Temple-Color, UAV123 and UAV20L, respectively.

Learning Temporal-Correlated and Channel- Decorrelated Siamese Networks for Visual Tracking

Recently, Siamese network based trackers have attracted growing popularity in visual tracking, which tackle the tracking by template matching between the initial template and successive search regions. The initial template patch is generally encoded into a convolutional feature for matching. However, the limited representational capability of the template feature limits the tracking accuracy. Besides, this fixed representation also fails to adapt to the target appearance changes. To alleviate these issues, we improve the Siamese trackers by introducing temporal correlation and channel decorrelation mechanisms. On the one hand, we consider the channel-wise correlations between the initial and historical template features to adaptively aggregate informative channel-wise representations for template update. On the other hand, we propose a decorrelation regularization to weaken the channel-wise correlations of individual template features. By end-to-end training, we learn a more complete and adaptive template for accurate object tracking. We demonstrate the generality of our approach by applying it to two prevalent Siamese trackers, i.e., SiamFC and SiamRPN. Extensive experiments on seven benchmark datasets verify the effectiveness of our method.

Discriminative Siamese Tracker Based on Multi-Channel-Aware and Adaptive Hierarchical Deep Features

Most existing Siamese trackers mainly use a pre-trained convolutional neural network to extract target features. However, due to the weak discrimination of the target and background information of pre-trained depth features, the performance of the Siamese tracker can be significantly degraded when facing similar targets or changes in target appearance. This paper proposes a multi-channel-aware and adaptive hierarchical deep features module to enhance the discriminative ability of the tracker. Firstly, through the multi-channel-aware deep features module, the importance values of feature channels are obtained from both the target details and overall information, to identify more important feature channels. Secondly, by introducing the adaptive hierarchical deep features module, the importance of each feature layer can be determined according to the response value of each frame, so that the hierarchical features can be integrated to represent the target, which can better adapt to changes in the appearance of the target. Finally, the proposed two modules are integrated into the Siamese framework for target tracking. The Siamese network used in this paper is a two-input branch symmetric neural network with two input branches, and they share the same weights, which are widely used in the field of target tracking. Experiments on some Benchmarks show that the proposed Siamese tracker has several points of improvement compared to the baseline tracker.

Learning target-aware background-suppressed correlation filters with dual regression for real-time UAV tracking

Traditional discriminative correlation filters have been rapidly developed in UAV tracking field due to their effectiveness and computational efficiency. Unfortunately, these methods are limited by unwanted boundary effects, which decrease the discriminative power between the target and background. Besides, due to the fast target appearance change and complex scenarios, such as background clutter and similar object, there often exists distractors around the response peak, which may cause tracking drift or even failure. In this paper, we propose a novel correlation filter method with dual regression, which aims at obtaining high-quality and reliable response maps for more robust tracking. Specifically, benefiting from a saliency detection method, the target feature is efficiently acquired from the global feature. Then, employing the dual regression strategy, these features are used to regress dual filters, that is, the target filter and the global filter. Response maps generated by dual filters, i.e., the target and global response maps, are merged in the detection phase to increase the response value of the target. Furthermore, a novel response-level background suppression regularization is proposed to solve boundary effects. Through the mutual restriction between the target and global response maps, the discrimination of dual filters can be promoted. We perform extensive and comprehensive analysis on three challenging UAV tracking benchmarks. Results confirm that the dual regression strategy and the background suppression regularization can facilitate the tracking accuracy improvement. The proposed tracker has comparable performance against other 27 state-of-the-art trackers while running at ∼34 FPS on a single CPU.

A Twofold Convolutional Regression Tracking Network With Temporal and Spatial Mechanism

In recent years, convolutional regression trackers have shown increasing attention for visual object tracking due to their favorable performance and easy implementation. However, most of them are restricted to features from a certain layer and hardly benefit from temporal spatial information, which limits the potential to significant appearance changes. In this work, we go beyond the traditional deep regression trackers and build a novel twofold tracking network, which exploits rich hierarchical features and incorporates both temporal and spatial information to boost the tracking performance. The proposed network is composed of two streams, i.e., an appearance stream and a semantic stream, each stream is independently learned from different convolutional layers. Specially, we propose temporal and spatial mechanism for robust target representation by considering historical information in previous frames as well as spatial information. By design, the proposed twofold convolutional regression tracking network with spatial and temporal mechanism can better tolerate the target appearance changes and improve the tracking accuracy. Extensive experimental results on the benchmarks OTB-2015, Temple-Color, UAV123, and VOT-2018 demonstrate the effectiveness of our method, as compared with a number of state-of-the-art trackers.