Visual Object Tracking Algorithm Research Articles

Visual tracking in camera-switching outdoor sport videos: Benchmark and baselines for skiing

Skiing is a globally popular winter sport discipline with a rich history of competitive events. This domain offers ample opportunities for the application of computer vision to enhance the understanding of athletes’ performances. However, this potential has remained relatively untapped in comparison to other sports, primarily due to the limited availability of dedicated research studies and datasets. The present paper takes a significant stride towards bridging these gaps. It conducts a comprehensive examination of skier appearance tracking in videos capturing their entire performance—an essential step for more advanced performance analyses. To implement this investigation, we introduce SkiTB, the largest and most annotated dataset tailored for computer vision applications in skiing. We subject a range of visual object tracking algorithms to rigorous testing, including both well-established methodologies and a novel skier-specific baseline algorithm. The results yield valuable insights into the suitability of various tracking techniques for vision-based skiing analysis and into the generalization of state-of-the-art algorithms to complex target behaviors and conditions set by winter outdoor environments. To foster further development, we make SkiTB, the associated code, and the obtained results accessible through https://machinelearning.uniud.it/datasets/skitb.

Transformer-Based Visual Object Tracking with Global Feature Enhancement

With the rise of general models, transformers have been adopted in visual object tracking algorithms as feature fusion networks. In these trackers, self-attention is used for global feature enhancement. Cross-attention is applied to fuse the features of the template and the search regions to capture the global information of the object. However, studies have found that the feature information fused by cross-attention does not pay enough attention to the object region. In order to enhance cross-attention for the object region, an enhanced cross-attention (ECA) module is proposed for global feature enhancement. By calculating the average attention score for each position in the fused feature sequence and assigning higher weights to the positions with higher attention scores, the proposed ECA module can improve the feature information in the object region and further enhance the matching accuracy. In addition, to reduce the computational complexity of self-attention, orthogonal random features are introduced to implement a fast attention operation. This decomposes the attention matrix into the product of a random non-linear function between the original query and key. This module can reduce the spatial complexity and improve the inference speed by avoiding the explicit construction of a quadratic attention matrix. Finally, a tracking method named GFETrack is proposed, which comprises a Siamese backbone network and an enhanced attention mechanism. Experimental results show that the proposed GFETrack achieves competitive results on four challenging datasets.

Visual–inertial object tracking: Incorporating camera pose into motion models

Visual object tracking for autonomy of aerial robots could become challenging especially in the presence of target or camera fast motions and long-term occlusions. This paper presents a visual-inertial tracking paradigm by incorporating the camera kinematics states into the visual object tracking pipelines. We gathered a dataset of image sequences with the addition of camera’s position and orientation measurements as well as the object’s position measurement. For the cases of long-term object occlusion, we provide ground-truth boxes derived from mapping the measured object position onto the image frame. A search zone proposal method is developed based on the estimation of object future position represented in the inertial frame and projected back onto the image frame using the camera states. This search zone, which is robust to fast camera/target motions, is fused into the original search zone settings of the base tracker. Also proposed is a measure indicating the confidence of a tracking structure in keeping track of a correct target and reporting the tracking failure in-time. Accordingly, the model updating mechanism of base tracker is modulated to avoid recovering of wrong objects as the target. The proposed modifications are benchmarked on nine visual object tracking algorithms including five state-of-art deep learning structures , namely DiMP, PrDiMP, KYS, ToMP, and MixFormer. Most of the trackers are remarkably improved by the modifications with up to 8% increase in precision. Modified PrDiMP tracker yields the best precision of 68.4%, more than all considered original (and modified) trackers. Source code and dataset are made available online.1

Robust Visual Object Tracking Based on Feature Channel Weighting and Game Theory

Although the discriminative correlation filter- (DCF)-based tracker improves tracking performance, some object representation issues can still be further optimized. On the one hand, the DCF tracker’s deep convolutional features contain many noisy channels, and assigning the same weights to multiple channels cannot distinguish the importance of different channels. On the other hand, a simple weighted fusion approach cannot fully utilize the benefits of different feature types. We propose a visual object tracking algorithm based on adaptive channel weighting and feature game fusion to solve these problems. In this study, an adaptive channel weighting strategy is designed to assign suitable weights to each channel based on the average energy ratio of the target and background regions in the feature channels and prune the channels with low weights to improve feature robustness and reduce computational complexity. Simultaneously, the game theory concept is introduced in the multifeature fusion. The handcrafted features are combined with shallow and deep convolutional features according to feature complementarity. Then, the two combined features are seen as two sides of the game, continuously gamed during the tracking process to generate a feature model with a higher representation capacity. Extensive experiments are conducted on four mainstream visual tracking benchmark datasets, including OTB2015, VOT2018, LaSOT, and UAV123. The experimental results show that the proposed algorithm performs outstandingly compared to the state-of-the-art trackers.

Learning spatial variance-key surrounding-aware tracking via multi-expert deep feature fusion

The continuous advancement of visual object tracking (VOT) algorithms has made significant contributions to computer vision and video processing. Correlation filter (CF)-based trackers have achieved promising tracking performance in VOT, but still face some problems under scale changes, occlusion, and boundary effects. To deal with these constraints, the learning spatial variance-key surrounding-aware correlation filter (LSVKSCF) via multi-expert deep feature fusion is proposed in this work. Initially, we extract the surrounding samples from the input based on the target size and shape, which integrates the context information and maintains the integrity of the target. Then, we introduce selective spatial information to the filter for penalizing the background and preserving the valid information of the target. Additionally, the spatial variance information is utilized to find the second-order difference of the filter which avoids over-fitting problems. Furthermore, we introduce the keyframes by periodically selecting the surrounding patches which handle the background distractions and improve the tracking speed. Finally, a multi-expert strategy is used in the filter learning stage which shows the reliability of the proposed method. Extensive experiments on OTB2013, OTB2015, TempleColor128, UAV123, UAVDT and DTB70 datasets prove that our method performs well against several existing trackers.

Domain Adaptive Transformer Tracking Under Occlusions

Due to their excellent performance on aggregating global features, Transformer structures are being widely employed in deep learning-based visual object tracking algorithms, recently. Nevertheless, existing Transformer-based trackers still fail to handle occlusion problems due to drift in feature distributions. To address this issue, we introduce domain adaptation techniques into a novel object tracking framework, DATransT, including feature extraction, domain adaptive Transformer module and prediction head. The domain adaptive Transformer module consists of three weight-sharing branches with self and cross attention mechanisms: the source, the target and the source-target branches. Specifically, the source-target branch employs cross-attention to effectively align the feature distributions of the source and target branches. Meanwhile, we present a pseudo-labeling strategy to generate high-quality training samples. Extensive experiments show that DATransT obtains promising results on several popular datasets, containing LaSOT, TrackingNet, GOT-10k, NfS, OTB2015 and UAV123. Moreover, our method outperforms existing state-of-the-art trackers under full occlusions and partial occlusions.

3D-SiamMask: Vision-Based Multi-Rotor Aerial-Vehicle Tracking for a Moving Object

This paper aims to develop a multi-rotor-based visual tracker for a specified moving object. Visual object-tracking algorithms for multi-rotors are challenging due to multiple issues such as occlusion, quick camera motion, and out-of-view scenarios. Hence, algorithmic changes are required for dealing with images or video sequences obtained by multi-rotors. Therefore, we propose two approaches: a generic object tracker and a class-specific tracker. Both tracking settings require the object bounding box to be selected in the first frame. As part of the later steps, the object tracker uses the updated template set and the calibrated RGBD sensor data as inputs to track the target object using a Siamese network and a machine-learning model for depth estimation. The class-specific tracker is quite similar to the generic object tracker but has an additional auxiliary object classifier. The experimental study and validation were carried out in a robot simulation environment. The simulation environment was designed to serve multiple case scenarios using Gazebo. According to the experiment results, the class-specific object tracker performed better than the generic object tracker in terms of stability and accuracy. Experiments show that the proposed generic tracker achieves promising results on three challenging datasets. Our tracker runs at approximately 36 fps on GPU.

An Accurate Refinement Pathway for Visual Tracking

Recently, in the field of visual object tracking, visual object tracking algorithms combined with visual object segmentation have achieved impressive results while using mask to label targets in the VOT2020 dataset. Most of the trackers get the object mask by increasing the resolution through multiple upsampling modules and gradually get the mask by summing with the features in the backbone network. However, this refinement pathway does not fully consider the spatial information of the backbone features, and therefore, the segmentation results are not perfect. In this paper, the cross-stage and cross-resolution (CSCR) module is proposed for optimizing the segmentation effect. This module makes full use of the semantic information of high-level features and the spatial information of low-level features, and fuses them by skip connections to achieve a very accurate segmentation effect. Experiments were conducted on the VOT dataset, and the experimental results outperformed other excellent trackers and verified the effectiveness of the algorithm in this paper.

Visual Object Tracking Algorithm Based on Biological Visual Information Features and Few-Shot Learning.

Eye tracking is currently a research hotspot in the territory of service robotics. There is an urgent need for machine vision technique in the territory of video surveillance, and biological visual object following is one of the important basic research problems. By tracking the object of interest and recording the tracking trajectory, we can extract a structure from a video. It can also analyze the abnormal behavior of groups or individuals in the video or assist the public security organs in inquiring and searching for evidence of criminal suspects, etc. Moving object following has always been one of the frontier topics in the territory of machine vision, and it has very important appliances in mobile robot positioning and navigation, multirobot formation, lunar exploration, and intelligent monitoring. Moving object following has always been one of the frontier topics in the territory of machine vision, and it has very important appliances in mobile robot positioning and navigation, multirobot formation, lunar exploration, and intelligent monitoring. Moving object following in visual surveillance is easily affected by factors such as occlusion, rapid object movement, and appearance changes, and it is difficult to solve these problems effectively with single-layer features. This paper adopts a visual object following algorithm based on visual information features and few-shot learning, which effectively improves the accuracy and robustness of tracking.

An intelligent method for dairy goat tracking based on Siamese network

Tracking livestock can obtain their behavioral information, position information, activity data, and health status. Research on a robust, high-precision, non-contact, and real-time visual object tracking algorithm has important practical value for the management of livestock farming. Since dairy goats are collectively raised livestock, the difficulty in accurately monitoring large-scale goats farms lies in automatically tracking the individual. In this study, a novel Siamese Network Guided by Attention Mechanism (AMTracker) was proposed to track single dairy goat in the real farm scene. First of all, through analysis, we found that there were lots of similar goats raised in the same sheepfold, which led to similarity interference challenge when tracking someone. Secondly, the EfficientNet was employed to map features and the BiFPN model was employed to fuse the features at different levels. Next, we introduced the attention mechanism to improve the correlation between template frame and search frame to deal with the problem of similar target interference. Finally, we used an Anchor-free network to predict the position of the dairy goat in the search frame. The experimental results showed that AMTracker was superior to four state-of-the-art methods in terms of four evaluation indicators, and the Expected Average Overlap (EAO), Robustness (R), Precision (Prec) and Success (Succ) of AMTracker was 0.340, 0.455, 0.835 and 0.657, respectively. The tracker ran in a real-time manner with an average analysis speed of 30 fps. Hence, it was demonstrated that the proposed approach could offer one effective way for automatically tracking a dairy goat in real farms with complex conditions.

Retraction Note to: The visual object tracking algorithm research based on adaptive combination kernel

Retraction Note to: The visual object tracking algorithm research based on adaptive combination kernel

Research on visual object tracking algorithm based on improved twin network

The tracking accuracy of traditional algorithm decreases when the target is in a complex environment or the target itself changes, and an improved correlation filtering twin network algorithm combining the attention mechanism is proposed. Fast wavelet is used for filtering and multi-scale feature signal extraction, and loss function is used to minimize the squared error between sample and label. Then correlation filtering algorithm is used to calculate the similarity of image between target template and candidate region. Attention mechanism is introduced to improve correlation filtering twin network structure, and the GloU’s online learning is advanced as an aid to determine template updating. The experimental results show that the algorithm have certain improvement effect for target shade, quick movement and deformation problems, so performance and accuracy of target tracking, the algorithm has better competitiveness.

Adaptive Gaussian Incremental Expectation Stadium Parameter Estimation Algorithm for Sports Video Analysis

In this paper, we propose an adaptive Gaussian incremental expectation stadium parameter estimation algorithm for sports video analysis and prediction through the study and analysis of sports videos. The features with more discriminative power are selected from the set of positive and negative templates using a feature selection mechanism, and a sparse discriminative model is constructed by combining a confidence value metric strategy. The sparse generative model is constructed by combining L1 regularization and subspace representation, which retains sufficient representational power while dealing with outliers. To overcome the shortcomings of the traditional multiplicative fusion mechanism, this paper proposes an adaptive selection mechanism based on Euclidean distance, which aims to detect deteriorating models in time during the dynamic tracking process and adopt corresponding strategies to construct more reasonable likelihood functions. Based on the Bayesian citation framework, the adaptive selection mechanism is used to combine the sparse discriminative model and the sparse generative model. Also, different online updating strategies are adopted for the template set and Principal Component Analysis (PCA) subspace to alleviate the drift problem while ensuring that the algorithm can adapt to the changes of target appearance in the dynamic tracking environment. Through quantitative and qualitative evaluation of the experimental results, it is verified that the algorithm proposed in this paper has stronger robustness compared with other classical algorithms. Our proposed visual object tracking algorithm not only outperforms existing visual object tracking algorithms in terms of accuracy, success rate, accuracy, and robustness but also achieve the performance required for real-time tracking in terms of execution speed on the central processing unit (CPU). This paper provides an in-depth analysis and discussion of the adaptive Gaussian incremental expectation stadium parameter estimation algorithm for sports video analysis. Using a variety of county-level algorithms for analysis and multiple solutions to improve the accuracy of the results, we obtain a more efficient and accurate algorithm.

Efficient visual tracking approach via whale optimizer and corrected background weighted histogram

There are mainly two components in almost every visual object tracking algorithm, which are the object presentation and the searching mechanism. In the literature, metaheuristic algorithms have been recently, widely used as searching method in visual tracking. In fact, the more discriminative is their combined object presentation the better is their precision. Salient background information is often inside the first detected region of the object, which may decrease considerably the tracking performance. In this work, we combine a powerful metaheuristic searching method, whale optimization algorithm (WOA), with the discriminative presentation named as corrected background weighted histogram (CBWH) to improve the tracking accuracy in difficult environments. WOA algorithm is used because of its strong searching ability, which enable it avoid being trapped in local optimum, and also its lowest computation cost compared to other metaheuristic algorithms. The WOA sensitivity parameters are studied experimentally to be adjusted in visual tracking. For a fair assessment of the proposed approach, its performance has been evaluated on 20 video sequences, from the famous object tracking benchmark OTB15, and compared with six (6) other state-of-art trackers. The evaluations were carried quantitatively and qualitatively on challenging situations and have provided satisfying results. The experimental results shows that WOA combined with CBWH can lead to further improvement, and better success rate in tracking.

Research on the improvement of vision target tracking algorithm for Internet of things technology and Simple extended application in pellet ore phase

With the extensive application of Internet of things technology and visual object tracking technology in various neighborhoods, the traditional visual object tracking algorithm can no longer meet the needs of Internet of things. The Internet of things with its powerful perception, multi-information transmission ability and super intelligent processing ability force the traditional visual object tracking algorithm to innovate and improve. In order to improve the tracking algorithm’s ability to cope with complex and drastic changes in the target’s appearance, in this paper, an accurate and robust visual object tracking algorithm (visual object tracking) based on K-means clustering algorithm on the integrated model ENS is studied. The tracking target is divided into several sample packets, and the size of the sample packet is compressed to match different weights of the filter in frequency domain. Finally, a more accurate and robust filter is built to realize the tracking of the target. Experimental simulation results show that the improved visual object tracking algorithm ENS-CSK approximately conforms to two-dimensional gaussian distribution in tracking effect, and the tracking success rate and accuracy are higher than the AdaBoost algorithm and KCF algorithm mentioned in the paper. Therefore, the improved visual object tracking algorithm ENS-CSK algorithm in this paper can better match the visual object tracking under the Internet of things technology. Therefore, it can be extended to the process of ore phase transformation of pellet under continuous calcination with varying temperature. Thus, the pressure resistance of pellets at different calcining temperatures can be predicted more accurately.

A Visual Object Tracking Algorithm Based on Improved TLD

Visual object tracking is an important research topic in the field of computer vision. Tracking–learning–detection (TLD) decomposes the tracking problem into three modules—tracking, learning, and detection—which provides effective ideas for solving the tracking problem. In order to improve the tracking performance of the TLD tracker, three improvements are proposed in this paper. The built-in tracking module is replaced with a kernelized correlation filter (KCF) algorithm based on the histogram of oriented gradient (HOG) descriptor in the tracking module. Failure detection is added for the response of KCF to identify whether KCF loses the target. A more specific detection area of the detection module is obtained through the estimated location provided by the tracking module. With the above operations, the scanning area of object detection is reduced, and a full frame search is required in the detection module if objects fails to be tracked in the tracking module. Comparative experiments were conducted on the object tracking benchmark (OTB) and the results showed that the tracking speed and accuracy was improved. Further, the TLD tracker performed better in different challenging scenarios with the proposed method, such as motion blur, occlusion, and environmental changes. Moreover, the improved TLD achieved outstanding tracking performance compared with common tracking algorithms.

A Visual Object Tracking Algorithm Based on Dynamics Pattern and Convolutional Feature

Deep visual feature-based method has demonstrated impressive performance in visual tracking attributing to its powerful capability of visual feature representation. However, in some complex environments such as dramatic change of appearance, illumination variation and rotation, the extracted deep visual feature is insufficient for accurately characterizing the target. To solve this problem, we present an integrated tracking framework which combines a Long Short-Term Memory (LSTM) network and a Convolutional Neural Network (CNN). Firstly, the LSTM extracted dynamics feature of target on time sequence, resulting the state of target at present time step. With that state, the accurate preprocessed bounding box was obtained. Then, deep convolutional feature of the target was extracted using a CNN, based on the processed bounding box. Finally, the position of the target was determined based on the score of the feature. During tracking stage, in order to improve the adaptation of the network, the parameters of the network were updated using samples of the target captured while successful tracking. The experiment shows that the proposed method achieves outstanding tracking performance and robustness in cases of partial occlusion, out-of-view, motion blur and fast motion.

Fully Convolutional Single-Crop Siamese Networks for Real-Time Visual Object Tracking

The visual object tracking problem seeks to track an arbitrary object in a video, and many deep convolutional neural network-based algorithms have achieved significant performance improvements in recent years. However, most of them do not guarantee real-time operation due to the large computation overhead for deep feature extraction. This paper presents a single-crop visual object tracking algorithm based on a fully convolutional Siamese network (SiamFC). The proposed algorithm significantly reduces the computation burden by extracting multiple scale feature maps from a single image crop. Experimental results show that the proposed algorithm demonstrates superior speed performance in comparison with that of SiamFC.

An adaptive template matching-based single object tracking algorithm with parallel acceleration

Existing template matching based visual object tracking algorithms usually require to manually update the template and have high execution cost on general embedded systems. To address these issues, an adaptive template matching-based single object tracking algorithm with parallel acceleration is proposed in this paper. In this algorithm, we propose an adaptive single object tracking algorithm framework to achieve template update online. Based on the Faster-RCNN model, we design a single object capture method to update the template. Meanwhile, we present a parallel strategy to accelerate the process of template matching. To evaluate the proposed algorithm, we use OTB benchmark to compare the performance with several state-of-the-art trackers on TX2 embedded platform. Experimental results show that the proposed method achieves a 5.9 times execution speed and 71.9% accuracy improvement over the comparison methods.

Detection based visual tracking with convolutional neural network

In this paper, we propose a detection strategy based visual object tracking algorithm. We employ multiple trackers using layers of deep convolutional neural network (CNN) features. Each tracker which is correlation filter based tracking framework tracks an object forwardly and then backwardly. By analyzing the forward and backward trajectories, we measure the robustness of tracking results. A detection strategy which is based on locally adaptive regression kernels (LARK) feature is carried out according to the robustness of tracking result. Target can be located from the provided candidates. Extensive experimental results show that the proposed method improves the accuracy and robustness of tracking, achieving state-of-the-art results on several recent benchmark datasets.