Object Tracking Benchmark Research Articles

Multi-object tracking using score-driven hierarchical association strategy between predicted tracklets and objects

Machine vision is one of the major technologies to guarantee intelligent robots’ human-centered embodied intelligence. Especially in the complex dynamic scene involving multi-person, Multi-Object Tracking (MOT), which can accurately identify and track specific targets, significantly influences intelligent robots’ performance regarding behavior perception and monitoring, autonomous decision-making, and providing personalized humanoid services. In order to solve the problem of targets lost and identity switches caused by the scale variations of objects and frequent overlaps during the tracking process, this paper presents a multi-object tracking method using score-driven hierarchical association strategy between predicted tracklets and objects (ScoreMOT). Firstly, a motion prediction of occluded objects based on bounding box variation (MPOBV) is proposed to estimate the position of occluded objects. MPOBV models the motion state of the object using the bounding box and confidence score. Then, a score-driven hierarchical association strategy between predicted tracklets and objects (SHAS) is proposed to correctly associate them in frequently overlapping scenarios. SHAS associates the predicted tracklets and detected objects with different confidence in different stages. The comparison results with 16 state-of-the-art methods on Multiple Object Tracking Benchmark 20 (MOT20) and DanceTrack datasets are conducted, and ScoreMOT outperforms the compared methods.

Scene-aware classifier and re-detector for thermal infrared tracking

Compared with common visible light scenes, the target of infrared scenes lacks information such as the color, texture. Infrared images have low contrast, which not only lead to interference between targets, but also interference between the target and the background. In addition, most infrared tracking algorithms lack a redetection mechanism after lost target, resulting in poor tracking effect after occlusion or blurring. To solve these problems, we propose a scene-aware classifier to dynamically adjust low, middle, and high level features, improving the ability to utilize features in different infrared scenes. Besides, we designed an infrared target re-detector based on multi-domain convolutional network to learn from the tracked target samples and background samples, improving the ability to identify the differences between the target and the background. The experimental results on VOT-TIR2015, VOT-TIR2017 and LSOTB-TIR show that the proposed algorithm achieves the most advanced results in the three infrared object tracking benchmark.

AMTrack:Transformer tracking via action information and mix-frequency features

Nowadays, Transformer-based visual tracking algorithms have been developing quickly because of the self-attention mechanism of Transformer, which has the capability to model global information. Although the self-attention mechanism in Transformer can effectively capture long-range dependencies in feature space, they only use flattened two-dimensional features and are unable to capture long-range temporal dependencies. Furthermore, since the self-attention in Transformer functions as a low-pass filter, it picks up on low-frequency features of the target while ignoring high-frequency features. This research suggests a Transformer tracker based on action information and mix-frequency features (AMTrack) to address these problems. Specifically, to address the lack of temporal remote dependencies, we introduce the target action aware module and the target action offset module. The target action aware module sets up several pathways to extract spatio-temporal, channel, and motion feature independently. In contrast, the target action offset module computes the target’s offset information by computing relative feature maps. Furthermore, in order to address the imbalance between high and low frequency features, we propose a mix-frequency attention and multi-frequency self-attention convolutional block. The mix-frequency attention uses high-frequency features within partitioned local windows as input for the high-frequency branch and average-pooled low-frequency features as the input for the low-frequency branch, calculating attention scores in both branches respectively. The multi-frequency self-attention convolutional block uses self-attention to capture low-frequency features and convolution to capture high-frequency features. Extensive experiments are carried out on eight challenging tracking datasets (e.g., OTB100 (Object Tracking Benchmark 100), NFS (Need For Speed), UAV123 (Unmanned Aerial Vehicles 123), TC128 (Temple Color 128), VOT2018 (Visual Object Tracking 2018), LaSOT (Large-scale Single Object Tracking), TrackingNet (Tracking Network), GOT-10k (Generic Object Tracking-10k)), and the experimental results show that our tracker achieves excellent tracking performance when compared with several state-of-the-art tracking algorithms. The experimental results show that on LaSOT, the success rates AUC (Area Under Curve), PNorm, and P reach 65.8%, 69.2%, and 68.0%, respectively, where the AUC value is 2.1% higher than the baseline algorithm TrDiMP (Transformer Discriminative Model Prediction). On other datasets, our tracker also achieves excellent tracking performance.

Fusion of Multiple Attention Mechanisms and Background Feature Adaptive Update Strategies in Siamese Networks for Single-Object Tracking

Single-object tracking algorithms based on Siamese full convolutional networks have attracted much attention from researchers owing to their improvement in precision and speed. Since this tracking model only learns a similarity model offline, it is not able to obtain more useful feature discrimination information to adapt to the various variations of targets in complex scenes. To improve the performance of this tracking model, we propose a Siamese network tracking algorithm that incorporates multiple attention mechanisms and an adaptive updating strategy for background features. First, a backbone feature extraction network is proposed that utilizes a small convolutional kernel to fuse jump-layer connectivity features, thereby improving the feature representation capability of the network. Second, an adaptive update strategy for background features is proposed to improve the model’s ability to discriminate between the object and background features. Third, the fusion of multiple attention mechanisms is proposed so that the model learns to focus on the channel, spatial, and coordinate features. Fourth, the response fusion operation is proposed after the inter-correlation operation to enrich the output response of the model. Finally, our algorithm is trained using the GOT-10K dataset and evaluated by testing on the object tracking benchmark datasets OTB100 and VOT2018. The test results show that compared with other algorithms, our algorithm can effectively cope with the problem of degradation of the tracking performance in complex environments, and it can further improve the tracking precision and precision under the premise of ensuring the tracking speed.

LSOTB-TIR: A Large-Scale High-Diversity Thermal Infrared Single Object Tracking Benchmark.

Unlike visual object tracking, thermal infrared (TIR) object tracking methods can track the target of interest in poor visibility such as rain, snow, and fog, or even in total darkness. This feature brings a wide range of application prospects for TIR object-tracking methods. However, this field lacks a unified and large-scale training and evaluation benchmark, which has severely hindered its development. To this end, we present a large-scale and high-diversity unified TIR single object tracking benchmark, called LSOTB-TIR, which consists of a tracking evaluation dataset and a general training dataset with a total of 1416 TIR sequences and more than 643 K frames. We annotate the bounding box of objects in every frame of all sequences and generate over 770 K bounding boxes in total. To the best of our knowledge, LSOTB-TIR is the largest and most diverse TIR object tracking benchmark to date. We spilt the evaluation dataset into a short-term tracking subset and a long-term tracking subset to evaluate trackers using different paradigms. What's more, to evaluate a tracker on different attributes, we also define four scenario attributes and 12 challenge attributes in the short-term tracking evaluation subset. By releasing LSOTB-TIR, we encourage the community to develop deep learning-based TIR trackers and evaluate them fairly and comprehensively. We evaluate and analyze 40 trackers on LSOTB-TIR to provide a series of baselines and give some insights and future research directions in TIR object tracking. Furthermore, we retrain several representative deep trackers on LSOTB-TIR, and their results demonstrate that the proposed training dataset significantly improves the performance of deep TIR trackers. Codes and dataset are available at https://github.com/QiaoLiuHit/LSOTB-TIR.

AODiMP‐TIR: Anti‐occlusion thermal infrared targets tracker based on SuperDiMP

AbstractTo address the issue of tracking drift and failures in thermal infrared (TIR) tracking tasks caused by target occlusion, this study proposes an anti‐occlusion TIR target tracker named AODiMP‐TIR. This approach involves an anti‐occlusion strategy that relies on target occlusion status determination and trajectory prediction. This enables the prediction of the target's current position when it is identified as occluded, ensuring swift recapture upon reappearance. A criterion is introduced for occlusion status determination based on the classification response map of SuperDiMP. Additionally, a trajectory mapping module designed to decouple target motion from camera motion is presented, enhancing the precision of trajectory prediction. Comparative experiments with other state‐of‐the‐art trackers are conducted on the large‐scale high‐diversity thermal infrared object tracking benchmark (LSOTB‐TIR), LSOTB‐TIR100, and thermal infrared pedestrian tracking benchmark (PTB‐TIR) datasets. The results indicate that the AODiMP‐TIR performs well across all three datasets, particularly exhibiting outstanding performance in occlusion sequences. Furthermore, ablation study experiments confirm the effectiveness of the anti‐occlusion strategy, occlusion determination criterion and trajectory mapping module.

Siamese Trackers Based on Deep Features for Visual Tracking

Visual object tracking poses challenges due to deformation of target object appearance, fast motion, brightness change, blocking due to obstacles, etc. In this paper, a Siamese network that is configured using a convolutional neural network is proposed to improve tracking accuracy and robustness. Object tracking accuracy is dependent on features that can well represent objects. Thus, we designed a convolutional neural network structure that can preserve feature information that is produced in the previous layer to extract spatial and semantic information. Features are extracted from the target object and search area using a Siamese network, and the extracted feature map is input into the region proposal network, where fast Fourier-transform convolution is applied. The feature map produces a probability score for the presence of an object region and an object in a region, where the similarities are high to search the target. The network was trained with a video dataset called ImageNet Large Scale Visual Recognition Challenge. In the experiment, quantitative and qualitative evaluations were conducted using the object-tracking benchmark dataset. The evaluation results indicated competitive results for some video attributes through various experiments. By conducting experiments, the proposed method achieved competitive results for some video attributes, with a success metric of 0.632 and a precision metric of 0.856 as quantitative values.

Online object tracking based interactive attention

By embedding Transformer into the Siamese tracking framework, some Transformer-based Siamese tracking network are proposed, such as TransT and SwinTrack. Nevertheless, the existing Transformer-based Siamese tracking networks do not fully utilize the object information of template branch, and their position encoding methods cannot accurately perceive the object position. Aiming at these problems, a novel Transformer-based Siamese tracking network, which includes feature extraction, feature fusion and prediction head, is proposed in this work. Firstly, an interactive attention calculation module for template branch and search branch is designed to enhance the feature extraction capability of the network for the object region and suppress the background interference. In addition, to address the problem that the existing Transformer-based feature fusion network is not sensitive enough to the object location region, a position encoding method that characterizes the relative distance is proposed to enhance the perception ability of the object location and reduce network parameters. Then, the contrastive loss is introduced to enhance the discriminative ability of the classification layer between foreground and background, and to effectively deal with the interference of similar objects in the background. Extensive experiments with state-of-the-art trackers are carried out on four challenging visual object tracking benchmarks: GOT-10k, LaSOText, TLP, and TrackingNet. Experimental results demonstrate the proposed method is more robust on multiple challenges and can achieve considerable performances with AO of 70.1% on GOT-10k, SUC score of 45.5% on LaSOT ext, 56.9% on TLP, and 79.7% on TrackingNet datasets. It runs faster (7.1G MACs) and occupies less memory (21M), making it more suitable for practical applications.

A cloud-oriented siamese network object tracking algorithm with attention network and adaptive loss function

Aiming at solving the problems of low success rate and weak robustness of object tracking algorithms based on siamese network in complex scenes with occlusion, deformation, and rotation, a siamese network object tracking algorithm with attention network and adaptive loss function (SiamANAL) is proposed. Firstly, the multi-layer feature fusion module for template branch (MFFMT) and the multi-layer feature fusion module for search branch (MFFMS) are designed. The modified convolutional neural networks (CNN) are used for feature extraction through the fusion module to solve the problem of features loss caused by too deep network. Secondly, an attention network is introduced into the SiamANAL algorithm to calculate the attention of template map features and search map features, which enhances the features of object region, reduces the interference of background region, and improves the accuracy of the algorithm. Finally, an adaptive loss function combined with pairwise Gaussian loss function and cross entropy loss function is designed to increase inter-class separation and intra-class compactness of classification branches and improve the accuracy rate of classification and the success rate of regression. The effectiveness of the proposed algorithm is verified by comparing it with other popular algorithms on two popular benchmarks, the visual object tracking 2018 (VOT2018) and the object tracking benchmark 100 (OTB100). Extensive experiments demonstrate that the proposed tracker achieves competitive performance against state-of-the-art trackers. The success rate and precision rate of the proposed algorithm SiamANAL on OTB100 are 0.709 and 0.883, respectively. With the help of cloud computing services and data storage, the processing performance of the proposed algorithm can be further improved.

UAV Chasing Based on YOLOv3 and Object Tracker for Counter UAV Systems

This paper proposes a vision-based unmanned aerial vehicle (UAV) chasing system that can be embedded in a pursuer UAV (pUAV) to protect the attack from an evader UAV (eUAV). The proposed UAV chasing system consists of two parts, i.e., the UAV tracking and control signal generation parts. Combining a deep learning-based object detector, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">you only look once</i> version three (YOLOv3), and the existing object trackers, the proposed UAV tracking algorithm can improve the tracking performance of pUAV within affordable computational complexity. Then, the control signals of the pUAV are generated by utilizing the predicted bounding box area of the eUAV and the proportional-derivative control method. Following the object tracking benchmark performance criteria, various combinations of YOLOv3 and object trackers are examined and compared. From the evaluation results, the UAV tracking algorithm with the highest performance is chosen, whose average success rate and average precision rate for object tracking are 2.86% and 5.61% higher than YOLOv3, respectively. Through the field test, it is verified that the proposed UAV chasing system outperforms the YOLOv3 system in terms of bounding box misalignment (33% accuracy improvement) and computational complexity (71% reduction).

Tracking System for a Coal Mine Drilling Robot for Low-Illumination Environments

In recent years, discriminative correlation filters (DCF) based trackers have been widely used in mobile robots due to their efficiency. However, underground coal mines are typically a low illumination environment, and tracking in this environment is a challenging problem that has not been adequately addressed in the literature. Thus, this paper proposes a Low-illumination Long-term Correlation Tracker (LLCT) and designs a visual tracking system for coal mine drilling robots. A low-illumination tracking framework combining image enhancement strategies and long-time tracking is proposed. A long-term memory correlation filter tracker with an interval update strategy is utilized. In addition, a local area illumination detection method is proposed to prevent the failure of the enhancement algorithm due to local over-exposure. A convenient image enhancement method is proposed to boost efficiency. Extensive experiments on popular object tracking benchmark datasets demonstrate that the proposed tracker significantly outperforms the baseline trackers, achieving high real-time performance. The tracker’s performance is verified on an underground drilling robot in a coal mine. The results of the field experiment demonstrate that the performance of the novel tracking framework is better than that of state-of-the-art trackers in low-illumination environments.

Edge-aware object pixel-level representation tracking

Recently, there has been a trend in tracking to use more refined segmentation mask instead of coarse bounding box to represent the target object. Some trackers proposed segmentation branches based on the tracking framework and maintain real-time speed. However, those trackers use a simple FCNs structure and lack of the edge information modeling. This makes performance quite unsatisfactory. In this paper, we propose an edge-aware segmentation network, which uses the complementarity between target information and edge information to provide a more refined representation of the target. Firstly, We use the high-level features of the tracking backbone network and the correlation features of the classification branch of the tracking framework to fuse, and use the target edge and target segmentation mask for simultaneous supervision to obtain an optimized high-level feature with rough edge information and target information. Secondly, we use the optimized high-level features to guide the low-level features of the tracking backbone network to generate more refined edge features. Finally, we use the refined edge features to fuse with the target features of each layer to generate the final mask. Our approach has achieved leading performance on recent pixel-wise object tracking benchmark VOT2020 and segmentation datasets DAVIS2016 and DAVIS2017 while running on 47 fps. Code is available at https://github.com/TJUMMG/EATtracker.

Detection and Tracking Meet Drones Challenge.

Drones, or general UAVs, equipped with cameras have been fast deployed with a wide range of applications, including agriculture, aerial photography, and surveillance. Consequently, automatic understanding of visual data collected from drones becomes highly demanding, bringing computer vision and drones more and more closely. To promote and track the developments of object detection and tracking algorithms, we have organized three challenge workshops in conjunction with ECCV 2018, ICCV 2019 and ECCV 2020, attracting more than 100 teams around the world. We provide a large-scale drone captured dataset, VisDrone, which includes four tracks, i.e., (1) image object detection, (2) video object detection, (3) single object tracking, and (4) multi-object tracking. In this paper, we first present a thorough review of object detection and tracking datasets and benchmarks, and discuss the challenges of collecting large-scale drone-based object detection and tracking datasets with fully manual annotations. After that, we describe our VisDrone dataset, which is captured over various urban/suburban areas of 14 different cities across China from North to South. Being the largest such dataset ever published, VisDrone enables extensive evaluation and investigation of visual analysis algorithms for the drone platform. We provide a detailed analysis of the current state of the field of large-scale object detection and tracking on drones, and conclude the challenge as well as propose future directions. We expect the benchmark largely boost the research and development in video analysis on drone platforms. All the datasets and experimental results can be downloaded from https://github.com/VisDrone/VisDrone-Dataset.

Siamese visual tracking with multilayer feature fusion and corner distance IoU loss

The tracker based on the Siamese network regards tracking tasks as solving a similarity problem between the target template and search area. Using shallow networks and offline training, these trackers perform well in simple scenarios. However, due to the lack of semantic information, they have difficulty meeting the accuracy requirements of the task when faced with complex backgrounds and other challenging scenarios. In response to this problem, we propose a new model, which uses the improved ResNet-22 network to extract deep features with more semantic information. Multilayer feature fusion is used to obtain a high-quality score map to reduce the influence of interference factors in the complex background on the tracker. In addition, we propose a more powerful Corner Distance IoU (intersection over union) loss function so that the algorithm can better regression to the bounding box. In the experiments, the tracker was extensively evaluated on the object tracking benchmark data sets, OTB2013 and OTB2015, and the visual object tracking data sets, VOT2016 and VOT2017, and achieved competitive performance, proving the effectiveness of this method.

A Novel Target Tracking Scheme Based on Attention Mechanism in Complex Scenes

In recent years, target tracking algorithms based on deep learning have realized significant progress, especially the Siamese neural network structure, which has a simple structure and excellent scalability. Although these methods provide excellent generalization capabilities, they fail to perform the task of learning target information discrimination smoothly due to being affected by distractors such as background clutter, occlusion, and target size. To solve this problem, in this paper we propose a newly improved Siamese network target tracking algorithm based on an attention mechanism. We introduce a channel attention module and a spatial attention module into the original network to improve the problem of insufficient semantic extraction ability of the convolutional layer of the tracking algorithm in complex environments. A channel attention mechanism enhances the feature extraction ability by using the network to learn the importance of each channel and establish the relationship between channels, while a spatial attention mechanism strengthens the feature extraction ability by establishing the importance of spatial position in locating the target or carrying out a certain degree of deformation. In this paper, the above two models are combined to improve the robustness of trackers without sacrificing tracking speed. We conduct a comprehensive experiment on the Object Tracking Benchmark dataset. The experimental results show that our algorithm outperforms other real-time trackers in both accuracy and robustness in most complex environments.

Deep Feature Based Siamese Network for Visual Object Tracking

One of the most important and challenging research subjects in computer vision is visual object tracking. The information obtained from the first frame consists of limited and insufficient information to represent an object. If prior information about robust representation that can represent an object well is not sufficient, object tracking fails when not robustly responding to changes in features of the target object according to various factors, namely shape, illumination variation, and scene distortion. In this paper, a real-time single object tracking algorithm is proposed based on a Siamese network to solve this problem. For the object feature extraction, we designed a fully convolutional neural network that removes a fully connected layer and configured a convolution block consisting of a bottleneck structure that preserves the information in a previous layer. This network was designed as a Siamese network, while a regional proposal network was combined at the end of the network for object tracking. The ImageNet Large-Scale Visual Recognition Challenge 2017 dataset was used to train the network in the pre-training phase. Then, in the experimental phase, the object tracking benchmark dataset was used to quantitatively evaluate the network. The experimental results revealed that the proposed tracking algorithm produced more competitive results compared to other tracking algorithms.

SatSOT: A Benchmark Dataset for Satellite Video Single Object Tracking

By imaging a specific area continuously, satellite video shows excellent capability in various applications such as surveillance and traffic management. Although object tracking has made significant progress in recent years, development in satellite object tracking is limited by the lack of open-source satellite datasets. It is thus essential to establish a satellite video object-tracking benchmark to fill the gap and advance the research. In this work, we present SatSOT, the first densely annotated satellite video single object-tracking benchmark dataset. SatSOT consists of 105 sequences with 27664 frames, 11 attributes, and four categories of typical moving targets in satellite videos: car, plane, ship, and train. Based on the proposed dataset and the significant challenges in satellite video object tracking, such as small targets, background interference, and severe occlusion, detailed evaluation and analysis are performed on 15 among the best and most representative tracking algorithms, which provides a basis for further research on satellite video object tracking.

Comprehensive Underwater Object Tracking Benchmark Dataset and Underwater Image Enhancement With GAN

Current state-of-the-art object tracking methods have largely benefited from the public availability of numerous benchmark datasets. However, the focus has been on open-air imagery and much less on underwater visual data. Inherent underwater distortions, such as color loss, poor contrast, and underexposure, caused by attenuation of light, refraction, and scattering, greatly affect the visual quality of underwater data, and as such, existing open-air trackers perform less efficiently on such data. To help bridge this gap, this article proposes a first comprehensive underwater object tracking (UOT100) benchmark dataset to facilitate the development of tracking algorithms well-suited for underwater environments. The proposed dataset consists of 104 underwater video sequences and more than 74 000 annotated frames derived from both natural and artificial underwater videos, with great varieties of distortions. We benchmark the performance of 20 state-of-the-art object tracking algorithms and further introduce a cascaded residual network for underwater image enhancement model to improve tracking accuracy and success rate of trackers. Our experimental results demonstrate the shortcomings of existing tracking algorithms on underwater data and how our generative adversarial network (GAN)-based enhancement model can be used to improve tracking performance. We also evaluate the visual quality of our model's output against existing GAN-based methods using well-accepted quality metrics and demonstrate that our model yields better visual data.

Methods and Means for Small Dynamic Objects Recognition and Tracking

A literature analysis has shown that object search, recognition, and tracking systems are becoming increasingly popular. However, such systems do not achieve high practical results in analyzing small moving living objects ranging from 8 to 14 mm. This article examines methods and tools for recognizing and tracking the class of small moving objects, such as ants. To fulfill those aims, a customized You Only Look Once Ants Recognition (YOLO_AR) Convolutional Neural Network (CNN) has been trained to recognize Messor Structor ants in the laboratory using the LabelImg object marker tool. The proposed model is an extension of the You Only Look Once v4 (Yolov4) 512 × 512 model with an additional Self Regularized Non–Monotonic (Mish) activation function. Additionally, the scalable solution for continuous object recognizing and tracking was implemented. This solution is based on the OpenDatacam system, with extended Object Tracking modules that allow for tracking and counting objects that have crossed the custom boundary line. During the study, the methods of the alignment algorithm for finding the trajectory of moving objects were modified. I discovered that the Hungarian algorithm showed better results in tracking small objects than the K–D dimensional tree (k-d tree) matching algorithm used in OpenDataCam. Remarkably, such an algorithm showed better results with the implemented YOLO_AR model due to the lack of False Positives (FP). Therefore, I provided a new tracker module with a Hungarian matching algorithm verified on the Multiple Object Tracking (MOT) benchmark. Furthermore, additional customization parameters for object recognition and tracking results parsing and filtering were added, like boundary angle threshold (BAT) and past frames trajectory prediction (PFTP). Experimental tests confirmed the results of the study on a mobile device. During the experiment, parameters such as the quality of recognition and tracking of moving objects, the PFTP and BAT, and the configuration parameters of the neural network and boundary line model were analyzed. The results showed an increased tracking accuracy with the proposed methods by 50%. The study results confirmed the relevance of the topic and the effectiveness of the implemented methods and tools.

Criss-Cross Attentional Siamese Networks for Object Tracking

Visual object tracking is a hot topic in recent years. In the meanwhile, Siamese networks have attracted extensive attention in this field because of its balanced precision and speed. However, most of the Siamese network methods can only distinguish foreground from the non-semantic background. The fine-tuning and retraining of fully-convolutional Siamese networks for object tracking(SiamFC) can achieve higher precision under interferences, but the tracking accuracy is still not ideal, especially in the environment with more target interferences, dim light, and shadows. In this paper, we propose criss-cross attentional Siamese networks for object tracking (SiamCC). To solve the imbalance between foreground and non-semantic background, we use the feature enhancement module of criss-cross attention to greatly improve the accuracy of video object tracking in dim light and shadow environments. Experimental results show that the maximum running speed of SiamCC in the object tracking benchmark dataset is 90 frames/second. In terms of detection accuracy, the accuracy of shadow sequences is greatly improved, especially the accuracy score of sequence HUMAN8 is improved from 0.09 to 0.89 compared with the original SiamFC, and the success rate score is improved from 0.07 to 0.55.