Visual Tracking Framework Research Articles

РОЗРОБКА ЕФЕКТИВНОГО ТА ТОЧНОГО МЕТОДУ ВІДСТЕЖЕННЯ ОБʼЄКТІВ НА ВІДЕО НА МОБІЛЬНИХ ПРИСТРОЯХ

One of the top research topics in computer vision area is visual object tracking. Main goal is to obtain the target object’s location in the first video frame of a given video sequence. The recent innovations of deep neural networks, specifically Siamese networks has significant impact on visual object tracking. In spite of high accuracy and high results in academic benchmarks, there are drawbacks in current state-of-the-art approaches in particular compute-intensive and large memory footprint that cannot satisfy the performance requirements of real-world applications. The aim of this paper is to design a new lightweight framework for resource-efficient and accurate visual object tracking. To add, a new tracker of efficiency benchmark and protocol were introduced. Efficiency is defined in terms of both energy consumption and execution speed on edge devices. New dual template representation for object model adaptation was developed. The first template, static, fixes the original visible appearance and thus prevents deviation and, as a result, failures caused by adaptation. The other is dynamic; the state reflects the current conditions of assembly and the appearance of its object. Unlike STARK, which incorporates additional timing information by introducing a separate estimation prediction head, we introduce parameter-free module similarity as a template update rule optimized from the latest network. We show that the learned convex combination of two patterns is effective for tracking on multiple tests. A lightweight tracker was proposed, which includes functions, dual representation of patterns and pixel-by-pixel merged blocks in its compact network. The resulting FEAR-XS tracker runs at 205 FPS on the iPhone 11, which is 4.2 times faster than LightTrack and 26.6 times faster than Ocean, with high accuracy on many tests – no state-of-the-art tracker is more accurate and faster than any FEAR tracker. In addition, the algorithm is highly energy efficient.

Target Focused Shallow Transformer Framework for Efficient Visual Tracking

Template learning transformer trackers have achieved significant performance improvement recently due to the longdependency learning using the self-attention (SA) mechanism. However, the typical SA mechanisms in transformers adopt a less discriminative design approach which is inadequate for focusing on the most important target information during tracking. Therefore, existing trackers are easily distracted by background information and have constraints in handling tracking challenges. The focus of our research is to develop a target-focused discriminative shallow transformer tracking framework that can learn to distinguish the target from the background and enable accurate tracking with fast speed. Extensive experiments will be performed on several popular benchmarks, including OTB100, UAV123, GOT10k, LaSOT, and TrackingNet, to demonstrate the effectiveness of the proposed framework.

Explicit Visual Prompts for Visual Object Tracking

How to effectively exploit spatio-temporal information is crucial to capture target appearance changes in visual tracking. However, most deep learning-based trackers mainly focus on designing a complicated appearance model or template updating strategy, while lacking the exploitation of context between consecutive frames and thus entailing the when-and-how-to-update dilemma. To address these issues, we propose a novel explicit visual prompts framework for visual tracking, dubbed EVPTrack. Specifically, we utilize spatio-temporal tokens to propagate information between consecutive frames without focusing on updating templates. As a result, we cannot only alleviate the challenge of when-to-update, but also avoid the hyper-parameters associated with updating strategies. Then, we utilize the spatio-temporal tokens to generate explicit visual prompts that facilitate inference in the current frame. The prompts are fed into a transformer encoder together with the image tokens without additional processing. Consequently, the efficiency of our model is improved by avoiding how-to-update. In addition, we consider multi-scale information as explicit visual prompts, providing multiscale template features to enhance the EVPTrack's ability to handle target scale changes. Extensive experimental results on six benchmarks (i.e., LaSOT, LaSOText, GOT-10k, UAV123, TrackingNet, and TNL2K.) validate that our EVPTrack can achieve competitive performance at a real-time speed by effectively exploiting both spatio-temporal and multi-scale information. Code and models are available at https://github.com/GXNU-ZhongLab/EVPTrack.

SiamFDA: feature dynamic activation siamese network for visual tracking.

In this paper, we present a novel anchor-free visual tracking framework, referred to as feature dynamic activation siamese network (SiamFDA), which addresses the issue of ignoring global spatial information in current Siamese network-based tracking algorithms. Our approach captures long-range dependencies between distant pixels in space, which enables robustness to unreliable regions. Additionally, we introduce a hierarchical feature selector that adaptively activates features at different layers, and an adaptive sample label assignment method to further improve tracking performance. Our extensive evaluations on six benchmark datasets, including VOT-2018, VOT-2019, GOT10k, LaSOT, OTB-2015, and OTB-2013, demonstrate that SiamFDA outperforms several state-of-the-art trackers in various challenging scenarios, with a real-time frame rate of 40 frames per second.

Advection-based tracking and analysis of salinity movement in the Indian Ocean

The Bay of Bengal (BoB) has maintained its salinity distribution over the years despite a continuous flow of fresh water entering it through rivers on the northern coast, which is capable of diluting the salinity. This can be attributed to the cyclic flow of high salinity water (≥35 psu), coming from Arabian sea and entering BoB from the south, which moves northward and mixes with this fresh water. The movement of this high salinity water has been studied and analyzed in previous work (Singh et al., 2022). This paper extends the computational methods and analysis of salinity movement. Specifically, we introduce an advection based feature definition that represents the movement of high salinity water, and describe algorithms to track their evolution over time. This method allows us to trace the movement of high salinity water caused due to ocean currents. The method is validated via comparison with established observations on the flow of high salinity water in the BoB, including its entry from the Arabian Sea and its movement near Sri Lanka. Further, the visual analysis and tracking framework enables us to compare it with previous work and analyze the contribution of advection to salinity transport.

Target-Cognisant Siamese Network for Robust Visual Object Tracking

• A new target-cognisant siamese-based anchor-free tracker. • The proposed method computes cross-spatial attention for refining the measurement of spatial similarity. • Two tracking mechanisms are used to promote the precision of bounding box prediction. • A max filtering module is proposed to filter out similar distractors. • Our method achieves competitive performance on several tracking datasets. Siamese trackers have become the mainstream framework for visual object tracking in recent years. However, the extraction of the template and search space features is disjoint for a Siamese tracker, resulting in a limited interaction between its classification and regression branches. This degrades the model capacity accurately to estimate the target, especially when it exhibits severe appearance variations. To address this problem, this paper presents a target-cognisant Siamese network for robust visual tracking. First, we introduce a new target-cognisant attention block that computes spatial cross-attention between the template and search branches to convey the relevant appearance information before correlation. Second, we advocate two mechanisms to promote the precision of obtained bounding boxes under complex tracking scenarios. Last, we propose a max filtering module to utilise the guidance of the regression branch to filter out potential interfering predictions in the classification map. The experimental results obtained on challenging benchmarks demonstrate the competitive performance of the proposed method.

Disentangling classification and regression in Siamese‐based network for visual tracking

SummarySiamese‐based trackers have made great progress in visual tracking community, however, the shared structure of network between classification and regression tasks limits the ability of the trackers to obtain more robust classification prediction and more accurate regression prediction. In this paper, we propose an effective visual tracking framework (named Siamese Disentangled Tracking‐Head, SiamDTH), which disentangles classification and regression in Siamese‐based network for visual tracking from two aspects: feature decoupling and differentiated tracking‐head. First of all, we gather the features of receptive fields with different scales and ratios, and decouple the correlation features through two different styles of feature fusion mode for classification and regression respectively. Moreover, we design the differentiated tracking‐head structure in the sibling head for discriminately handling the parallel classification and regression tasks on visual tracking. Extensive experiments on visual tracking benchmarks including VOT2018, VOT2019 and OTB100 demonstrate that our proposed SiamDTH achieves state‐of‐the‐art performance with a considerable real‐time speed. Our source code is available at:https://github.com/xl0312/SiamDTH.

Siamese Attention Networks with Adaptive Templates for Visual Tracking

Visual object tracking takes an important role in realistic applications, such as video understanding, unmanned auto vehicles, and autonomous robots. Although the Siamese-based tracker has achieved good performance in tracking tasks, the existing methods using initial template or updating template with simple strategy result in the performance degradation of the model when the target varies in realistic scenarios such as target occlusion, scale variation, and deformation. In this paper, we propose a visual tracking framework with adaptive template update and spatiotemporal attention, named SiamAttnAT. Specially, we propose a historical template selecting strategy and a template adaptively generating method for robust tracking. In addition, we apply the proposed mechanisms to the employed baseline SiamRPN++. Extensive experiments and comparisons with state-of-the-art trackers on short-term and long-term visual tracking benchmarks including VOT2018, OTB-100, UAV123, NFS, and LaSOT show that the proposed framework achieves the outstanding performance with a considerable real-time speed, verifying its efficiency and effectiveness.

Deep convolutional correlation iterative particle filter for visual tracking

This work proposes a novel framework for visual tracking based on the integration of an iterative particle filter, a deep convolutional neural network, and a correlation filter. The iterative particle filter enables the particles to correct themselves and converge to the correct target position. We employ a novel strategy to assess the likelihood of the particles after the iterations by applying K-means clustering. Our approach ensures a consistent support for the posterior distribution. Thus, we do not need to perform resampling at every video frame, improving the utilization of prior distribution information. Experimental results on three different benchmark datasets show that our tracker performs favorably against state-of-the-art methods.

Improved Siamese classification and regression adaptive network for visual tracking

ABSTRACT To solve the problems of the complex multi-scale searching and the low correlation between classification confidence and location accuracy in the visual tracking process, we propose a novel visual tracking framework named ISiamCRAN. ISiamCRAN uses modified ResNet-50 as the backbone network to extract depth features. Then, these extracted features are fed into an improved classification-regression adaptive head for depth feature cross-correlation operation. Different from existing trackers, in order to remove low-quality prediction bounding boxes, we integrate a quality assessment branch to the classification-regression adaptive head. Moreover, we use an elliptical sample label assignment strategy to replace traditional strategy, by this way our tracker can more accurately distinguish the foreground and background. Finally, for these feature response maps obtained by depth feature cross-correlation operation, the position and scale of the target are predicted directly in a unified fully convolutional network by an anchor-free manner. Extensive experiments on OTB100 and VOT2018 benchmarks indicate that our ISiamCRAN outperforms other traditional trackers, and is robust to motion blur and scale variation. Our tracker runs at approximatively 35 fps on GPU.

Object-Aware Adaptive Convolution Kernel Attention Mechanism in Siamese Network for Visual Tracking

As a classic framework for visual object tracking, the Siamese convolutional neural network has received widespread attention from the research community. This method uses a convolutional neural network to obtain the object features and to match them with the search area features to achieve object tracking. In this work, we observe that the contribution of each convolution kernel in the convolutional neural network for object tracking tasks is different. We propose an object-aware convolution kernel attention mechanism. Based on the characteristics of each object, the convolution kernel features are dynamically weighted to improve the expression ability of object features. The experiments performed using OTB and VOT benchmark datasets show that the performance of the tracking method fused with the convolution kernel attention mechanism is significantly better compared with the original method. Moreover, the attention mechanism can also be integrated with other tracking frameworks as an independent module to improve the performance.

Front and skeleton features based methods for tracking salinity propagation in the ocean

The Bay of Bengal (BoB) fosters several monsoon depressions and cyclones, playing a crucial role in the Asian summer and winter monsoons. The capacity of the bay to remain warm and energize such weather systems is attributed to its strong vertical stratification sustained by the large freshwater input into the bay. River runoff and rainfall into the northern bay in contrast to the high salinity water intrusion in the south creates a strong north–south salinity gradient. Here, we present a visual analysis tool to trace the path of the high salinity core (HSC) entering into the BoB from the Arabian Sea. We introduce two feature definitions that represent the movement and shape of the HSC, and algorithms to track their evolution over time. The two feature representations, namely fronts and skeletons, are based on geometric and topological analysis of the HSC. The method is validated via comparison with well established observations on the flow of the HSC in the BoB, including its entry from the Arabian Sea and its movement near Sri Lanka. Further, the visual analysis and tracking framework enable new detailed observations on forking behavior near the center of the BoB and subsequent northward movement of the HSC. The tools that we have developed offer new perspectives on the propagation of high salinity water and its mixing with the ambient low salinity waters.

Residual attention-based tracking-by-detection network with attention-driven data augmentation

Tracking-by-detection (TBD) is a significant framework for visual object tracking. However, current trackers are usually updated online based on random sampling with a probability distribution. The performance of the learning-based TBD trackers is limited by the lack of discriminative features, especially when the background is full of semantic distractors. We propose an attention-driven data augmentation method, in which a residual attention mechanism is integrated into the TBD tracking network as supplementary references to identify discriminative image features. A mask generating network is used to simulate changes in target appearances to obtain positive samples, where attention information and image features are combined to identify discriminative features. In addition, we propose a method for mining hard negative samples, which searches for semantic distractors with the response of the attention module. The experiments on the OTB2015, UAV123, and LaSOT benchmarks show that this method achieves competitive performance in terms of accuracy and robustness.

Online parallel framework for real-time visual tracking

Discriminative correlation filter (DCF) based tracking algorithms have recently achieved excellent performance in challenging factors such as rotations and distractors. However, in case with fast motions and full occlusion, these trackers are not always effective and easily drift. In contrast, Siamese matching networks are insensitive to fast motions, but suffer from distractors. In this work, we propose an efficient real-time parallel framework for robust visual tracking. By incorporating DCF and Siamese network into the tracking framework to complement each other and better distinguish target objects from background clutters. Additionally, to improve tracking results and prevent target drift, we introduce an effective tracker switch method to select suitable tracker in each frame by considering the confidence estimation as well as motion smoothness, and examines the impact among all trackers over time. To validate the effectiveness of our method, we perform comprehensive experiments on three popular benchmark datasets, namely OTB2013, OTB2015, VOT2016, and TC128. The experimental results demonstrate that the proposed algorithm exhibits favorably results and real-time tracking speed against state-of-art trackers.

Reliable part-based long-term tracking using multiple correlation filters

Most visual trackers focus on short-term tracking. The target is always in the camera field of view or slight occlusion (OCC). Compared with short-term tracking, long-term tracking is a more challenging task. It requires the ability to capture the target in long-term sequences and undergo frequent disappearances and reappearances of target. Therefore, long-term tracking is much closer to a realistic tracking system. However, few long-term tracking algorithms have been developed and few promising performances have been shown until now. We focus on a long-term visual tracking framework based on parts correlation filters (CFs). Our long-term tracking framework is composed of a part-based short-term tracker and a re-detection module. First, multiple CFs have been applied to locate the target collaboratively and address the partial OCC issue. Second, our method updates the part adaptively based on its motion similarity and reliability score to retain its robustness. Third, a switching strategy has been designed to dynamically activate the re-detection module and interact the search mode between local and global search. In addition, our re-detector is trained by sampling positive and negative samples around the reliable tracking target to adapt to the appearance changes. To evaluate the candidates from the re-detection module, verification has been carried out, which could ensure the precision of recovery. Numerous experimental results demonstrate that our proposed tracking method performs favorably against state-of-the-art methods in terms of accuracy and robustness.

Robust Visual Tracking with Reliable Object Information and Kalman Filter.

Object information significantly affects the performance of visual tracking. However, it is difficult to obtain accurate target foreground information because of the existence of challenging scenarios, such as occlusion, background clutter, drastic change of appearance, and so forth. Traditional correlation filter methods roughly use linear interpolation to update the model, which may lead to the introduction of noise and the loss of reliable target information, resulting in the degradation of tracking performance. In this paper, we propose a novel robust visual tracking framework with reliable object information and Kalman filter (KF). Firstly, we analyze the reliability of the tracking process, calculate the confidence of the target information at the current estimated location, and determine whether it is necessary to carry out the online training and update step. Secondly, we also model the target motion between frames with a KF module, and use it to supplement the correlation filter estimation. Finally, in order to keep the most reliable target information of the first frame in the whole tracking process, we propose a new online training method, which can improve the robustness of the tracker. Extensive experiments on several benchmarks demonstrate the effectiveness and robustness of our proposed method, and our method achieves a comparable or better performance compared with several other state-of-the-art trackers.

Improved Hierarchical Convolutional Features for Robust Visual Object Tracking

The target and background will change continuously in the long-term tracking process, which brings great challenges to the accurate prediction of targets. The correlation filter algorithm based on manual features is difficult to meet the actual needs due to its limited feature representation ability. Thus, to improve the tracking performance and robustness, an improved hierarchical convolutional features model is proposed into a correlation filter framework for visual object tracking. First, the objective function is designed by lasso regression modeling, and a sparse, time-series low-rank filter is learned to increase the interpretability of the model. Second, the features of the last layer and the second pool layer of the convolutional neural network are extracted to realize the target position prediction from coarse to fine. In addition, using the filters learned from the first frame and the current frame to calculate the response maps, respectively, the target position is obtained by finding the maximum response value in the response map. The filter model is updated only when these two maximum responses meet the threshold condition. The proposed tracker is evaluated by simulation analysis on TC-128/OTB2015 benchmarks including more than 100 video sequences. Extensive experiments demonstrate that the proposed tracker achieves competitive performance against state-of-the-art trackers. The distance precision rate and overlap success rate of the proposed algorithm on OTB2015 are 0.829 and 0.695, respectively. The proposed algorithm effectively solves the long-term object tracking problem in complex scenes.

Re-identification framework for long term visual object tracking based on object detection and classification

In this paper, we address the problem of long-term visual object tracking and we present an efficient real-time single object tracking system suitable for integration in autonomous platforms that need to encompass intelligent capabilities. We propose a novel long-term tracking framework for classification based re-detection and tracking, that incorporates state estimation, object re-identification and automated management of tracking and detection results. Our method integrates a novel object re-identification technique which efficiently filters a number of detection candidates and systematically corrects the tracking results. Through extensive experimental validation on the UAV123, UAV20L and TLP datasets, we demonstrate the effectiveness of the proposed system and its advantage over several state-of-the art trackers. The results furthermore highlight the proposed tracker’s ability to handle challenges arising from real-world and long-term scenarios, such as variations in pose, scale, occlusions and out-of-view situations. Furthermore, we propose a variant that is suitable for deployment on autonomous robots, such as Unmanned Aerial Vehicles.

Noise-Aware Framework for Robust Visual Tracking.

Both siamese network and correlation filter (CF)-based trackers have exhibited superior performance by formulating tracking as a similarity measure problem, where a similarity map is learned by the correlation between a target template and a region of interest (ROI) with a cosine window. Nevertheless, this window function is usually fixed for various targets and not changed, undergoing significant noise variations during tracking, which easily makes model drift. In this article, we focus on the study of a noise-aware (NA) framework for robust visual tracking. To this end, the impact of various window functions is first investigated in visual tracking. We identify that the low signal-to-noise ratio (SNR) of windowed ROIs makes the above trackers degenerate. At the prediction phase, a novel NA window customized for visual tracking is introduced to improve the SNR of windowed ROIs by adaptively suppressing the variable noise according to the observation of similarity maps. In addition, to further optimize the SNR of windowed pyramid ROIs for scale estimation, we propose to use the particle filter to dynamically sample several windowed ROIs with more favorable signals in temporal domains instead of this pyramid ROIs extracted in spatial domains. Extensive experiments on the popular OTB-2013, OTB-50, OTB-2015, VOT2017, TC128, UAV123, UAV123@10fps, UAV20L, and LaSOT datasets show that our NA framework can be extended to many siamese and CF trackers and our variants obtain superior performance than baseline trackers with a modest impact on efficiency.

An efficient hybrid framework for visual tracking using Exponential Quantum Particle Filter and Mean Shift optimization

Visual object tracking is a key component in many computer vision applications. In real time visual tracking, abrupt changes in speed and direction of the object are demanding challenges. In this paper, we present an efficient visual tracking framework which can efficiently handle the above challenge, i.e., abrupt motion of the object. The framework is formulated by hybridizing the proposed Exponential Quantum Particle Filter (EQPF) with the traditional Mean-Shift (MS) optimization for efficient computation in object tracking task. The efficacy of EQPF in estimating the abrupt changes in functions, is tested on the standard Quail function, and then it has been successfully applied in object tracking algorithm. The effective multi-modal propagation strategies of Quantum Particle Filter (QPF) enables the tracker to handle the abrupt changes in speed and direction, whereas, the hybridization with MS enhances the computational efficiency by reducing the number of particles. Performance of the proposed method is assessed by experimenting on different publicly available challenging sequences. Both the subjective and objective evaluations are carried out to validate the superiority of the proposed tracking method over other state-of-the-art methods.