Single Object Tracking Research Articles

Deep Feature Based Siamese Network for Visual Object Tracking

One of the most important and challenging research subjects in computer vision is visual object tracking. The information obtained from the first frame consists of limited and insufficient information to represent an object. If prior information about robust representation that can represent an object well is not sufficient, object tracking fails when not robustly responding to changes in features of the target object according to various factors, namely shape, illumination variation, and scene distortion. In this paper, a real-time single object tracking algorithm is proposed based on a Siamese network to solve this problem. For the object feature extraction, we designed a fully convolutional neural network that removes a fully connected layer and configured a convolution block consisting of a bottleneck structure that preserves the information in a previous layer. This network was designed as a Siamese network, while a regional proposal network was combined at the end of the network for object tracking. The ImageNet Large-Scale Visual Recognition Challenge 2017 dataset was used to train the network in the pre-training phase. Then, in the experimental phase, the object tracking benchmark dataset was used to quantitatively evaluate the network. The experimental results revealed that the proposed tracking algorithm produced more competitive results compared to other tracking algorithms.

Complexity Metric Methodology of Infrared Image Sequence for Single-Object Tracking

Complexity Metric Methodology of Infrared Image Sequence for Single-Object Tracking

RAMC: A Rotation Adaptive Tracker with Motion Constraint for Satellite Video Single-Object Tracking

Single-object tracking (SOT) in satellite videos (SVs) is a promising and challenging task in the remote sensing community. In terms of the object itself and the tracking algorithm, the rotation of small-sized objects and tracking drift are common problems due to the nadir view coupled with a complex background. This article proposes a novel rotation adaptive tracker with motion constraint (RAMC) to explore how the hybridization of angle and motion information can be utilized to boost SV object tracking from two branches: rotation and translation. We decouple the rotation and translation motion patterns. The rotation phenomenon is decomposed into the translation solution to achieve adaptive rotation estimation in the rotation branch. In the translation branch, the appearance and motion information are synergized to enhance the object representations and address the tracking drift issue. Moreover, an internal shrinkage (IS) strategy is proposed to optimize the evaluation process of trackers. Extensive experiments on space-born SV datasets captured from the Jilin-1 satellite constellation and International Space Station (ISS) are conducted. The results demonstrate the superiority of the proposed method over other algorithms. With an area under the curve (AUC) of 0.785 and 0.946 in the success and precision plots, respectively, the proposed RAMC achieves optimal performance while running at real-time speed.

A novel apple fruit detection and counting methodology based on deep learning and trunk tracking in modern orchard

Accurate count of fruits is important for producers to make adequate decisions in production management. Although some algorithms based on machine vision have been developed to count fruits which were all implemented by tracking fruits themselves, those algorithms often make mismatches or even lose targets during the tracking process due to the large number of highly similar fruits in appearance. This study aims to develop an automated video processing method for improving the counting accuracy of apple fruits in orchard environment with modern vertical fruiting-wall architecture. As the trunk is normally larger than fruits and appears clearly in the video, the trunk is thus selected as a single-object tracking target to reach a higher accuracy and higher speed tracking than the commonly used method of fruit-based multi-object tracking. This method was trained using a YOLOv4-tiny network integrated with a CSR-DCF (channel spatial reliability-discriminative correlation filter) algorithm. Reference displacement between consecutive frames was calculated according to the frame motion trajectory for predicting possible fruit locations in terms of previously detected positions. The minimum Euclidean distance of detected fruit position and the predicted fruit position was calculated to match the same fruits between consecutive video frames. Finally, a unique ID was assigned to each fruit for counting. Results showed that mean average precision of 99.35% for fruit and trunk detection was achieved in this study, which could provide a good basis for fruit accurate counting. A counting accuracy of 91.49% and a correlation coefficient R2 of 0.9875 with counting performed by manual counting were reached in orchard videos. Besides, proposed counting method can be implemented on CPU at 2 ∼ 5 frames per second (fps). These promising results demonstrate the potential of this method to provide yield data for apple fruits or even other types of fruits.

Accurate visual representation learning for single object tracking

Accurate visual representation learning for single object tracking

Local to global Tracker: A Siamese Network for Long-term Tracking

Visual object tracking is a basic task in computer vision, which can be applied to different applications. Former methods of single object tracking problem demands to perform target state estimation according to object position of previous frame, but these methods are poorly capable of handling the fast movement of the target. In this paper, we introduce a Siamese network for single object tracking task. The network consists of two branches, one of which is the classification branch used to predict positive or negative samples, and the other is the regression branch used to predict the specific location of the object in sequence. In addition, we propose a local to global algorithm for long-term tracking to expand the search region when the target is out-of-view or full occlusion. Experiments show that our tracker achieves very promising results on VOT2018LT datasets while running at over 89 FPS.

Multi-Object Tracking and Segmentation With Embedding Mask-Based Affinity Fusion in Hierarchical Data Association

In this paper, we propose a highly feasible fully online multi-object tracking and segmentation (MOTS) method that uses instance segmentation results as an input. The proposed method is based on the Gaussian mixture probability hypothesis density (GMPHD) filter, a hierarchical data association (HDA), and a mask-based affinity fusion (MAF) model to achieve high-performance online tracking. The HDA consists of two associations: segment-to-track and track-to-track associations. One affinity, for position and motion, is computed by using the GMPHD filter, and the other affinity, for appearance is computed by using the responses from single object trackers such as kernalized correlation filter, SiamRPN, and DaSiamRPN. These two affinities are simply fused by using a score-level fusion method such as min-max normalization referred to as MAF. In addition, to reduce the number of false positive segments, we adopt mask IoU-based merging (mask merging). The proposed MOTS framework with the key modules: HDA, MAF, and mask merging, is easily extensible to simultaneously track multiple types of objects with CPU-only execution in parallel processing. In addition, the developed framework only requires simple parameter tuning unlike many existing MOTS methods that need intensive hyperparameter optimization. In the experiments on the two popular MOTS datasets, the key modules show some improvements. For instance, ID-switch decreases by more than half compared to a baseline method in the training sets. In conclusion, our tracker achieves state-of-the-art MOTS performance in the test sets.

Single Object Tracking in Satellite Videos: A Correlation Filter-Based Dual-Flow Tracker

Satellite video (SV) can acquire rich spatiotemporal information on the Earth. Single object tracking (SOT) in SVs enables the continuous acquisition of the position and range of a specific object, expanding the field of remote-sensing applications. In SVs, objects are small with limited features and vulnerable to tracking drift. In this paper, a correlation filter-based dual-flow (DF) tracker is proposed to explore how the hybridization of spatial-spectral feature fusion and motion model can boost tracking. To represent small objects, the DF adaptively fuses complementary features using a state-aware indicator in feature flow. In motion flow, the indicator perceives the confidence of the feature flow. A dual-mode prediction model is then constructed to simulate the object's motion pattern, and cooperate linear and non-linear motion patterns to implement SOT in SVs. The ablation experiments demonstrate the dual-flow contributes to tracking. Experimental comparisons on 14 real SVs captured by the Jilin-1 satellite constellation show that DF achieves optimal performance with an area under the curve of 0.912 in the precision plot, 0.700 in the success plot, and a speed of 155.2 frames per second. This work would encourage the development of remote-sensing ground surveillance.

Research on Specific Long-term Single Object Tracking Algorithms in the Context of Traffic

A robust object tracking algorithm based on a three-channel Siamese network is proposed for the visual object tracking problem in the context of traffic. By adding the prediction box in the previous frame as the second template, our network obtained the second template to perform secondary prediction on the current frame. Then, in order to make sure whether the model to accurately regress the correct prediction box by the template frame or the previous frame, we design a similarity matching method based on Structure Similarity Index Measure(SSIM). We have also designed a template update method to ensure that the network selects the correct matching box. Our algorithm has a major advantage in robustness over other real-time target tracking algorithms such as SiamRPN.

Fusion-Enhanced Feature Deblurring for 3d Single Object Tracking

Fusion-Enhanced Feature Deblurring for 3d Single Object Tracking

Quality Aware Features for Performance Prediction and Time Reduction in Video Object Tracking

The existing body of work on video object tracking (VOT) algorithms has studied various image conditions such as occlusion, clutter, and object shape, which influence video quality and affect tracking performance. Nonetheless, there is no clear distinction between the performance reduction caused by scene-dependent challenges such as occlusion and clutter, and the effect of authentic in-capture and post-capture distortions. Despite the plethora of VOT methods in the literature, there is a lack of detailed studies analyzing the performance of videos with authentic in-capture and post-capture distortions. We introduced a new dataset of authentically distorted videos (AD-SVD) to address this issue. This dataset contains 4476 videos with different authentic distortions and surveillance activities. Furthermore, it provides benchmarking results for evaluating ten state-of-the-art visual object trackers (from VOT 2017–2018 challenges) based on the proposed dataset. In addition, this study develops an approach for performance prediction and quality-aware feature selection for single-object tracking in authentically distorted surveillance videos. The method predicts the performance of a VOT algorithm with high accuracy. Then, the probability of obtaining the reference output is maximized without executing the tracking algorithms. We also propose a framework to reduce video tracker computation resources (time and video storage space). We achieve this by balancing processing time and tracking accuracy by predicting the performance in a range of spatial resolutions. This approach can reduce the execution time by up to 34% with a slight decrease in performance of 3%.

Single object tracking algorithm in occlusion scene based on improved ECO

To alleviate the occlusion problem in a single object tracking scene, this paper proposes an ECO-MHDU object tracking algorithm with a more powerful anti-occlusion performance based on the ECO tracker. The algorithm first uses the pre-trained MobileNetV3 lightweight backbone network on the ImageNet dataset to replace the ResNet network in the ECO to increase the speed of the algorithm to obtain the shallow and deep feature information of the image, while effectively using the attention mechanism in the MobileNetV3 network to strengthen the algorithm’s ability to extract target features; secondly, use the DropBlock operation on the acquired feature map to generate a random continuous mask on the feature map channel to improve the algorithm’s learning of the global robust spatial structure information; finally, a confidence update strategy is introduced into the GMM sample generation space. To improve the quality of training samples, unreliable tracking states such as confidence detection and occlusion are designed to avoid updating the sample space with damaging information. Compared with the ECO algorithm, the ECO-MHDU algorithm proposed in this paper has a success rate of 68.0% on the occlusion attributes of the OTB100 dataset, which is 2.3% higher than the ECO algorithm, and the ECO-MHDU algorithm also showed the best performance on the entire dataset sequence, with a success rate of 69.3%.

Automatic multi-plaque tracking and segmentation in ultrasonic videos.

Carotid plaque tracking and segmentation in ultrasound videos is the premise for subsequent plaque property evaluation and treatment plan development. However, the task is quite challenging, as it needs to address the problems of poor image quality, plaque shape variations among frames, the existence of multiple plaques, etc. To overcome these challenges, we propose a new automatic multi-plaque tracking and segmentation (AMPTS) framework. AMPTS consists of three modules. The first module is a multi-object detector, in which a Dual Attention U-Net is proposed to detect multiple plaques and vessels simultaneously. The second module is a set of single-object trackers that can utilize the previous tracking results efficiently and achieve stable tracking of the current target by using channel attention and a ranking strategy. To make the first module and the second module work together, a parallel tracking module based on a simplified 'tracking-by-detection' mechanism is proposed to solve the challenge of tracking object variation. Extensive experiments are conducted to compare the proposed method with several state-of-the-art deep learning based methods. The experimental results demonstrate that the proposed method has high accuracy and generalizability with a Dice similarity coefficient of 0.83 which is 0.16, 0.06 and 0.27 greater than MAST (Lai etal., 2020), Track R-CNN (Voigtlaender etal., 2019) and VSD (Yang etal., 2019) respectively and has made significant improvements on seven other indicators. In the additional Testing set 2, our method achieved a Dice similarity coefficient of 0.80, an accuracy of 0.79, a precision of 0.91, a Recall 0.70, a F1 score of 0.79, an AP@0.5 of 0.92, an AP@0.7 of 0.74, and an expected average overlap of 0.79. Numerous ablation studies suggest the effectiveness of each proposed component and the great potential for multiple carotid plaques tracking and segmentation in clinical practice.

A Novel Anti-Drift Visual Object Tracking Algorithm Based on Sparse Response and Adaptive Spatial-Temporal Context-Aware

Correlation filter (CF) based trackers have gained significant attention in the field of visual single-object tracking, owing to their favorable performance and high efficiency; however, existing trackers still suffer from model drift caused by boundary effects and filter degradation. In visual tracking, long-term occlusion and large appearance variations easily cause model degradation. To remedy these drawbacks, we propose a sparse adaptive spatial-temporal context-aware method that effectively avoids model drift. Specifically, a global context is explicitly incorporated into the correlation filter to mitigate boundary effects. Subsequently, an adaptive temporal regularization constraint is adopted in the filter training stage to avoid model degradation. Meanwhile, a sparse response constraint is introduced to reduce the risk of further model drift. Furthermore, we apply the alternating direction multiplier method (ADMM) to derive a closed-solution of the object function with a low computational cost. In addition, an updating scheme based on the APCE-pool and Peak-pool is proposed to reveal the tracking condition and ensure updates of the target’s appearance model with high-confidence. The Kalam filter is adopted to track the target when the appearance model is persistently unreliable and abnormality occurs. Finally, extensive experimental results on OTB-2013, OTB-2015 and VOT2018 datasets show that our proposed tracker performs favorably against several state-of-the-art trackers.

Multi-Drone-Based Single Object Tracking With Agent Sharing Network

Drones equipped with cameras (UAVs) can dynamically track the target in the air from a broader view compared with static cameras or moving sensors over the ground. However, it is still challenging to accurately track the target using a single drone due to several factors such as appearance variations and severe occlusions. To this end, we collect a new <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">M</b> ulti- <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">D</b> rone single <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">O</b> bject <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">T</b> racking ( <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">MDOT</b> ) dataset that consists of 92 groups of video clips with 113, 918 high resolution frames taken by two drones and 63 groups of video clips with 145, 875 high resolution frames taken by three drones. Besides, two evaluation metrics are specially designed for multi-drone single object tracking, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">i.e.</i> , automatic fusion score (AFS) and ideal fusion score (IFS). Moreover, the agent sharing network (ASNet) is proposed by integrating self-supervised template sharing, target re-detection, and view-aware fusion of the target from multiple drones into a unified framework, which can improve the tracking accuracy significantly compared with single drone tracking. Extensive experiments on MDOT show that our ASNet significantly outperforms recent state-of-the-art trackers. The dataset can be found in <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/VisDrone/MultiDrone</uri> .

Tracking more than 100 arbitrary objects at 25 FPS through deep learning

Most video analytics applications rely on object detectors to localize objects in frames. However, when real-time is a requirement, running the detector at all the frames is usually not possible. This is somewhat circumvented by instantiating visual object trackers between detector calls, but this does not scale with the number of objects. To tackle this problem, we present SiamMT, a new deep learning multiple visual object tracking solution that applies single-object tracking principles to multiple arbitrary objects in real-time. To achieve this, SiamMT reuses feature computations, implements a novel crop-and-resize operator, and defines a new and efficient pairwise similarity operator. SiamMT naturally scales up to several dozens of targets, reaching 25 fps with 122 simultaneous objects for VGA videos, or up to 100 simultaneous objects in HD720 video. SiamMT has been validated on five large real-time benchmarks, achieving leading performance against current state-of-the-art trackers.

Structure aware 3D single object tracking of point cloud

Existing 3D single object trackers (SOTs) of a point cloud all apply downscaling when extracting features from points. This operation leads to a loss of spatial and structural information, degrading tracking performance of sparsely distributed and small-scale objects. To address this problem, a structure aware SOT of a point cloud is proposed. Specifically, the backbone network is combined with the auxiliary network to learn point-wise representations. During the training stage, the subsidiary network is used to perform additional tasks and supervisions, which guides the backbone network to extract discriminative structural features. During the inference stage, this network part is detached to meet a real-time requirement as well as to ensure the tracking accuracy. In addition, the impacts of the quantity setting of the input point cloud and re-initiation strategy are discussed; these are significant to the performance but have been ignored by former works. The experimental results show that the proposed method has a distinct improvement even if the tracked object is sparse and small scale.

Siamese single object tracking algorithm with natural language prior

Siamese single object tracking algorithm with natural language prior

Coarse-to-Fine Siamese Network for Tooth Object Tracking

At present, object tracking algorithms based on deep learning have been widely used in the computer vision industry and has great application value. In this paper, we track the tooth, which has the problem of poor texture and high similarity, with the object tracking algorithm. In general, previously existing single object tracking algorithms are derived from the siamese network architecture. These algorithms have the problem of losing a tracked object with poor textures and high similarities. In order to address the issue of object loss, we put forward a novel coarse-to-fine siamese network (CFSN) for tooth object tracking. The coarse detection network is based on the features extracted by the Feature Pyramid Networks (FPN) and the Region Proposal Network (RPN). The fine detection network has a similar structure to the coarse detection network, and the main purpose is to fine-tune the results of the previous stage. Based on the coarse-to-fine network, a framework of multi-task is proposed, that is, the mask prediction branch is added to the fine detection network, which lead to significant improvements in tracking efficiency. Through the comprehensive experimental evaluation of the tooth dataset, we find that the CFSN model is more effective in tracking tooth objects than the existing methods.

Siamese pyramid residual module with local binary convolution network for single object tracking

Visual object tracking has made rapid progress in the past few years, which has received more and more attention. It is known that the texture information plays an essential role in numerous applications, for instance, in the person reidentification, image classification and so on. In this work, we propose a kind of new framework based on Siamese networks for single object tracking, called Siamese pyramid residual module with local binary convolution network for single object tracking (SPLBCT). A local binary convolution (LBC) module is proposed for the extraction of the texture feature of the tracked objective, which can improve in-variance to lighting. A pyramid residual Module (PRM) is used to study different levels of features and achieve the fusion of multi-scale features. The combination of the LBC and PRM enhances the feature extraction of the image data. According to two object tracking benchmarks (VOT2015 and VOT2016) and online tracking benchmark (OTB100), the final experimental results show that the proposed method performs better compared with some previous trackers.