Object Tracking Algorithm Research Articles

Learning spatial variance-key surrounding-aware tracking via multi-expert deep feature fusion

The continuous advancement of visual object tracking (VOT) algorithms has made significant contributions to computer vision and video processing. Correlation filter (CF)-based trackers have achieved promising tracking performance in VOT, but still face some problems under scale changes, occlusion, and boundary effects. To deal with these constraints, the learning spatial variance-key surrounding-aware correlation filter (LSVKSCF) via multi-expert deep feature fusion is proposed in this work. Initially, we extract the surrounding samples from the input based on the target size and shape, which integrates the context information and maintains the integrity of the target. Then, we introduce selective spatial information to the filter for penalizing the background and preserving the valid information of the target. Additionally, the spatial variance information is utilized to find the second-order difference of the filter which avoids over-fitting problems. Furthermore, we introduce the keyframes by periodically selecting the surrounding patches which handle the background distractions and improve the tracking speed. Finally, a multi-expert strategy is used in the filter learning stage which shows the reliability of the proposed method. Extensive experiments on OTB2013, OTB2015, TempleColor128, UAV123, UAVDT and DTB70 datasets prove that our method performs well against several existing trackers.

Extended Object Tracking in Curvilinear Road Coordinates for Autonomous Driving

In literature, Extended Object Tracking (EOT) algorithms developed for autonomous driving predominantly provide obstacles state estimation in cartesian coordinates in the Vehicle Reference Frame. However, in many scenarios, state representation in road-aligned curvilinear coordinates is preferred when implementing autonomous driving subsystems like cruise control, lane-keeping assist, platooning, etc. This paper proposes a Gaussian Mixture Probability Hypothesis Density~(GM-PHD) filter with an Unscented Kalman Filter~(UKF) estimator that provides obstacle state estimates in curvilinear road coordinates. We employ a hybrid sensor fusion architecture between Lidar and Radar sensors to obtain rich measurement point representations for EOT. The measurement model for the UKF estimator is developed with the integration of coordinate conversion from curvilinear road coordinates to cartesian coordinates by using cubic hermit spline road model. The proposed algorithm is validated through Matlab Driving Scenario Designer simulation and experimental data collected at Monza Eni Circuit.

Autonomous Cargo Bike Fleets – Approaches for AI-Based Trajectory Forecasts of Road Users

Abstract Automated cargo bikes are intended to complement public transportation in a sharing concept and provide an alternative transportation option for people and goods. In highly automated driving without a seated user, real-time trajectory prediction of other road users is crucial for collision avoidance with other motor vehicles or vulnerable road users (VRU). For this purpose, moving obstacles are detected by environmental sensors and classified and tracked using object detection and tracking algorithms. The current and past position data as well as environmental information are used to predict future positions. In this paper, we present several AI-based trajectory prediction models that are specifically suited for this use case. Our focus is not only on the accuracy of trajectory prediction, but additionally on a robust, real-time and practical application. We consider models that can predict the trajectories with position estimation or distributions for position estimation for each time step in the future. For this aim, we present generative network structures based on Conditional Variational Autoencoder (CVAE) in different variants. After training, the models are integrated into our production system and their computation time is determined on the hardware we use.

CNN과 학습 가능한 상관필터를 결합한 객체 추적 알고리즘

Object tracking is considered a challenging problem due to various environmental changes contained in the video sequence. Object tracking is the use of information given in the first frame to estimate the area and trajectory of a target object in a video sequence. In this paper, we propose an algorithm for object tracking using multi-scale feature maps of Resnet-50 and correlation filters. To accommodate changes in object scale, we create an appearance model using different sized feature maps extracted from each block in the network. In order to maintain the robustness of the appearance model, it is adaptively updated according to the peak value of the response map when occlusion occurs due to obstacles. Experiments result using the OTB2015 dataset showed that the proposed algorithm achieved competitive results on several image attribution such as low resolution, scale variation, occlusion and out of view.

Real-Time Computer Vision for Tree Stem Detection and Tracking

Object detection and tracking are tasks that humans can perform effortlessly in most environments. Humans can readily recognize individual trees in forests and maintain unique identifiers during occlusion. For computers, on the other hand, this is a complex problem that decades of research have been dedicated to solving. This paper presents a computer vision approach to object detection and tracking tasks in forested environments. We use a state-of-the-art neural network-based detection algorithm to fit bounding boxes around individual tree stems and a simple, efficient, and deterministic multiple object tracking algorithm to maintain unique identities for stems through video frames. We trained the neural network object detector on approximately 3000 ground-truth bounding boxes of ponderosa pine trees. We show that tree stem detection can achieve an average precision of 87% using a Jaccard overlap index of 0.5. We also demonstrate the robustness of the tracking algorithm in occlusion and enter–exit–re-enter scenarios. The presented algorithms can perform object detection and tracking at 49 frames per second on a consumer-grade graphics processing unit.

Lane-specific speed analysis in urban work zones with computer vision

Objective Work zone speed is one of the most important factors in road construction safety management. This work presents a computer vision based technique designed to measure lane-specific individual vehicle speed using existing traffic monitoring cameras and computers. The resulted speeds support the influence analysis of factors including traffic control, lane positions, and construction activity. Methods Object detection (YOLOv5) and tracking (Deep-SORT) algorithms are combined to track the vehicles. In particular, 21 days’ worth of road construction videos are collected from a pole-mounted traffic monitoring camera operated by the Texas A&M University Transportation Services. Based on the object detection results, a novel construction activity inference technique is developed to approximate the times when construction workers are present. Based on this time separation, the vehicle speeds with and without the presence of construction activity are compared. Results The proposed framework is able to measure speeds with an error ranging from 0 to 6.4 kilometers per hour (KPH). Detailed analysis of this video data suggests that traffic control with barrels in the median work zone lowers the average speed (for all vehicles) by 15 KPH. The lane adjacent to the work zone also has higher speed variation than the other lanes. The construction activity speed comparisons show when the traffic is slow (possibly traffic after a red light), the difference is statistically significant with a p-value ranging from 0.01 to 0.03. When the traffic is fast (possibly traffic encountering a green signal as they approached the nearby intersection) construction activity has no significant effect on the work zone speeds. Conclusions The proposed CV technique is a reliable and cheap method to measure lane-specific work zone speeds. The derived measurements support detailed safety analysis. Other than work zone speeds, the proposed technique can also be used for regular traffic speed monitoring.

A novel Siamese network object tracking algorithm based on tensor space mapping and memory-learning mechanism

The tracker is a core component of the tracking algorithm, but it is difficult to identify the object, which is a challenge to improve the tracking accuracy. This paper proposes a Siamese network-based tracking algorithm based on tensor space mapping and memory-learning mechanisms. Firstly, the source image is mapped to the tensor space to serialize the feature distributions. Then the gating mechanism is used to extract the association information about the adjacent state, which guides the update of the subsequent state, and the interactive information on the objects is used to locate the object. On this basis, a memory-learning module is built to traverse and extract the fine-grained features, which can filter the semantic information of the object learned by the tracker. As a result, the tracking accuracy is enhanced. The experiments show that the proposed algorithm has better performance than that of the comparison methods in the OTB100 data set and the VOT data set.

Digital Twin and Metaverse Technologies, Cognitive Mapping and Navigation Tools, and Image Processing Computational and Object Tracking Algorithms in Immersive 3D Virtual Reality Environments

Digital Twin and Metaverse Technologies, Cognitive Mapping and Navigation Tools, and Image Processing Computational and Object Tracking Algorithms in Immersive 3D Virtual Reality Environments

Spike-Event Object Detection for Neuromorphic Vision

Neuromorphic vision is one of the novel research fields that study neuromorphic cameras and spiking neural networks (SNNs) for computer vision. Instead of computing on frame-based images, spike events are streamed from neuromorphic cameras, and novel object detection algorithms have to deal with spike events to achieve detection tasks. In this paper, we propose a solution to the novel object detection method with spike events. Spike events are first encoded to event images according to the computational methodology of neuromorphic theory. The event images can be realized as change-detected images of moving objects with a high frame rate. A redesigned deep learning framework is proposed for object detection to process the event images. We propose a deep SNN method that is achieved by the conversion of successful convolution neural networks but trained by event images. The networks with multiscale representation are discussed and designed in our method. We also design a semi-automatic data labeling method to build event-image datasets by object tracking algorithms. The proposed solution, therefore, includes spike event encoding, a redesigned deep SNN, and an event-image data augmentation algorithm. Experiments are conducted not only on the MNIST-DVS dataset, which is a benchmark dataset for the study of neuromorphic vision but also on our event pedestrian detection dataset. The experimental results show that the performance of the deep SNN trained with our augmented data is close to the model trained on manually labeled data. A performance comparison based on the PAFBenchmark dataset shows that our proposed method has higher accuracy than existing SNN methods, and better energy efficiency and lower energy consumption than existing CNN methods. It demonstrates that our deep SNN method is a feasible solution for the study of neuromorphic vision. The intuition that deep SNN trained with more learning data can achieve better accuracy is also confirmed for this brand-new research field.

A Multitask Benchmark Dataset for Satellite Video: Object Detection, Tracking, and Segmentation

Video satellites can continuously image large areas and provide dynamic, real-time monitoring of hotspots and objects. The intelligent processing and analysis of satellite video have become a research hotspot in the field of remote sensing. However, the lack of high-quality satellite video datasets limits the development of relevant object detection, object tracking, and object segmentation. In this paper, we build the largest scale satellite video dataset with the most task types supported and object categories, named Satellite Video Multi-Mission Benchmark (SAT-MTB). First, multi-task annotation of aircraft, ships, cars, trains, and their corresponding 14 categories of fine-grained objects in 249 satellite videos is performed based on horizontal bounding boxes (HBB), oriented bounding boxes (OBB), masks, which cover more than 50,000 frames and 1,033,511 annotated object instances. Then, we review the tasks of object detection, object tracking, and object segmentation based on satellite videos, providing a comprehensive overview of progress in related datasets and algorithm research. Finally, we establish the first public benchmark of multi-task algorithms for satellite video object detection, object tracking, and object segmentation, evaluating and analyzing the performance of a total of 47 representative algorithms under different tasks on the constructed dataset. The proposed SAT-MTB will significantly advance research in intelligent processing and analysis of satellite video and related applications.

Robust Deep Transfer Learning Based Object Detection and Tracking Approach

At present days, object detection and tracking concepts have gained more importance among researchers and business people. Presently, deep learning (DL) approaches have been used for object tracking as it increases the performance and speed of the tracking process. This paper presents a novel robust DL based object detection and tracking algorithm using Automated Image Annotation with ResNet based Faster regional convolutional neural network (R-CNN) named (AIA-FRCNN) model. The AIA-RFRCNN method performs image annotation using a Discriminative Correlation Filter (DCF) with Channel and Spatial Reliability tracker (CSR) called DCF-CSRT model. The AIA-RFRCNN model makes use of Faster RCNN as an object detector and tracker, which involves region proposal network (RPN) and Fast R-CNN. The RPN is a full convolution network that concurrently predicts the bounding box and score of different objects. The RPN is a trained model used for the generation of the high-quality region proposals, which are utilized by Fast R-CNN for detection process. Besides, Residual Network (ResNet 101) model is used as a shared convolutional neural network (CNN) for the generation of feature maps. The performance of the ResNet 101 model is further improved by the use of Adam optimizer, which tunes the hyperparameters namely learning rate, batch size, momentum, and weight decay. Finally, softmax layer is applied to classify the images. The performance of the AIA-RFRCNN method has been assessed using a benchmark dataset and a detailed comparative analysis of the results takes place. The outcome of the experiments indicated the superior characteristics of the AIA-RFRCNN model under diverse aspects.

RelationTrack: Relation-Aware Multiple Object Tracking With Decoupled Representation

Existing online multiple object tracking (MOT) algorithms often consist of two subtasks, detection and re-identification (ReID). In order to enhance the inference speed and reduce the complexity, current methods commonly integrate these double subtasks into a unified framework. Nevertheless, detection and ReID demand diverse features. This issue results in an optimization contradiction during the training procedure. With the target of alleviating this contradiction, we devise a module named Global Context Disentangling (GCD) that decouples the learned representation into detection-specific and ReID-specific embeddings. As such, this module provides an implicit manner to balance the different requirements of these two subtasks. Moreover, we observe that preceding MOT methods typically leverage local information to associate the detected targets and neglect to consider the global semantic relation. To resolve this limitation, we develop a module, referred to as Guided Transformer Encoder (GTE), by combining the powerful reasoning ability of Transformer encoder and deformable attention. Unlike previous works, GTE avoids analyzing all the pixels and only attends to capture the relation between query nodes and a few self-adaptively selected key samples. Therefore, it is computationally efficient. Extensive experiments have been conducted on the MOT16, MOT17 and MOT20 benchmarks to demonstrate the superiority of the proposed MOT framework, namely RelationTrack. The experimental results indicate that RelationTrack has surpassed preceding methods significantly and established a new state-of-the-art performance, e.g., IDF1 of 70.5% and MOTA of 67.2% on MOT20.

A Visual Tracking Algorithm Combining Parallel Network and Dual Attention-Aware Mechanism

In order to solve the problems of semantic loss and inaccurate boundary detection in the process of object tracking, a visual tracking algorithm combining parallel structure and dual attention-attention mechanism is proposed in this paper. As backbone network, parallel structure is composed of Convolutional neural network and Attention Cooperative(CAC) processing module, which is used for feature extraction. Because this structure can capture the local and global information of the target at the same time, it can solve the problem of semantic information loss. Dual Attention-aware Network(DAN) is used for feature enhancement, which is composed of target-aware attention and boundary-aware attention, and the accuracy of boundary detection is enhanced by feature pyramid structure. Template online updating strategy is used to improve template quality, and an effective score prediction module-Template Elimination Mechanism(TEM) is designed in the CAC processing module to select high quality templates. This kind of object tracking algorithm which combines local and global information is called TrackCAC. The evaluation results on different datasets show that the algorithm can maintain high tracking precision and success in different scenarios. It shows good robustness and accuracy in the performance evaluation results on VOT datasets.

Real-Time Tracking Based on Improved YOLOv5 Detection in Orchard Environment for Dragon Fruit

Highlights This method has achieved faster detection speed while maintaining accuracy. It is a real-time tracking method that can track dragon fruits in orchard environments in real-time. The introduction of an attention mechanism in the network provides good robustness to changes in lighting and target scale. Abstract. This article addresses the issue of dragon fruit real-time detection in orchard environments and proposes a real-time detection and tracking model for dragon fruit using an improved YOLOv5 object detection algorithm and Deep-sort object tracking algorithm. By applying real-time tracking to dragon fruit harvesting, the tracking algorithm provides timely feedback on the fruit's location, allowing for prompt correction of environmental issues that may affect the accuracy of the harvesting process. This approach enhances the robustness of the target positioning algorithm. First,based on the YOLOv5 object detection algorithm, the Convolutional Block Attention Module and Transformer self-attention mechanism are introduced to construct a YOLOv5s-DFT object detection model that is more suitable for dragon fruit detection. Next, Combining the Deep-sort multi-object tracking algorithm, this article proposes a real-time detection and tracking method for dragon fruit in the orchard environment. The YOLOv5s-DFT model was trained and experimented with using a self-built dataset. The trained model weight is only 19.26% of YOLOv7. The experimental result shows that, while ensuring detection accuracy, YOLOv5s-DFT has a faster detection speed in dragon fruit detection, with an average frame time of 0.01673 s, which is 0.00422 s faster than the original YOLOv5s. When tracking dragon fruit using the Deep-sort tracking algorithm, it can track dragon fruit at a speed of 47.08 frames per second. When utilizing the Deep-sort tracking algorithm to track dragon fruit, it achieves a tracking speed of 47.08 frames per second, enabling real-time acquisition of the fruit's position information. This technology provides technical assistance for the intelligent harvesting of dragon fruit and the intelligent management of dragon fruit orchards. Keywords: Dragon fruit, Improved YOLOv5, Orchard environment, Real-time tracking.

Research on Siamese Object Tracking Algorithm Based on Knowledge Distillation in Marine Environment

Siamese networks have gained considerable attention for object tracking due to their balance of speed and accuracy. However, existing Siamese tracking algorithms have been too rigid in their predictions of bounding box tags and lack uncertainty estimation, resulting in poor tracking performance in marine environments, particularly those with waves. To improve the effectiveness of trackers in marine environments, this study proposes a Siamese distillation network. First, to address the issue that the presence of waves and other disturbances may result in target loss or inaccuracy when tracking the target, the concept of a probability distribution of the bounding box is introduced in this study, which transforms the standard Dirac delta distribution of the bounding box into a probability distribution of the bounding box, effectively reducing the impact of interference on tracking performance and improving target location accuracy. Second, we chose ResNet100 as the backbone network to obtain richer features for localization. Finally, this work offers a knowledge distillation approach to further enhance the tracking accuracy and model performance, while considering the impact of the model’s number of parameters and computational amount on tracking performance. This network outperforms most trackers in terms of accuracy, according to extensive experimental results, and performs well on the target tracking benchmark and marine dataset annotated in this study. Specifically, this network achieved the highest accuracy value of 0.612 compared to other Siamese networks, resulting in a 2.5% increase compared to original baseline network. This suggests that the proposed algorithm is practical.

Object Tracking Algorithm Based on Multi-Time-Space Perception and Instance-Specific Proposals

Object Tracking Algorithm Based on Multi-Time-Space Perception and Instance-Specific Proposals

Domain Adaptive Transformer Tracking Under Occlusions

Due to their excellent performance on aggregating global features, Transformer structures are being widely employed in deep learning-based visual object tracking algorithms, recently. Nevertheless, existing Transformer-based trackers still fail to handle occlusion problems due to drift in feature distributions. To address this issue, we introduce domain adaptation techniques into a novel object tracking framework, DATransT, including feature extraction, domain adaptive Transformer module and prediction head. The domain adaptive Transformer module consists of three weight-sharing branches with self and cross attention mechanisms: the source, the target and the source-target branches. Specifically, the source-target branch employs cross-attention to effectively align the feature distributions of the source and target branches. Meanwhile, we present a pseudo-labeling strategy to generate high-quality training samples. Extensive experiments show that DATransT obtains promising results on several popular datasets, containing LaSOT, TrackingNet, GOT-10k, NfS, OTB2015 and UAV123. Moreover, our method outperforms existing state-of-the-art trackers under full occlusions and partial occlusions.

Adaptive Spatial Regularization Target Tracking Algorithm Based on Multifeature Fusion

The accuracy and robustness of object-tracking algorithms are challenging tasks in the field of artificial intelligence. The discriminative correlation filter has fast tracking speed and target discrimination ability in visual target tracking, but it will be affected by unnecessary boundary effects. Although spatially regularized discriminative correlation filters (SRDCF) effectively solve this problem, it faces a slow tracking speed and cannot meet the real-time requirement. Moreover, the constraints added by this algorithm are fixed during the tracking process, which needs to reflect better the characteristics and appearance changes of a particular object. Therefore, we propose an adaptive spatial regularization target tracking model based on multifeature fusion and learn the spatial regularization weights by adaptive spatial regularization terms, so that the filter can adapt to the changes of the target, which can highlight the target area and suppress the background area more accurately. At the same time, in order to cope with the problem of target loss when the target is obscured by a large area or rotated to a large extent, texture features are introduced to improve the deficiencies of HOG features and CN features, so as to achieve the complementary advantages among different features. In addition, the benchmark algorithm SRDCF uses Gaussian–Seidel iterative method to solve the filter, which makes the tracking speed very slow and cannot be tracked in real time. Therefore, this paper proposes an ADMM (alternating direction method of multiplier) algorithm to optimize the solution of the filter and achieve the tracking real-time requirement. The experimental results on the OTB-2015 dataset show that the accuracy and success rate of the improved algorithm reach 86% and 81.9%, respectively. At the same time, in the OTB-2013 dataset, it has increased by 3.4% and 6.3%, respectively, on the basis of the benchmark algorithm. It is verified that the improved model not only improves the robustness of the algorithm but also can better adapt to the changes of the target, while effectively reducing the computational overhead and meeting the tracking real-time requirements.

Automatic detection of foreign body objects in neurosurgery using a deep learning approach on intraoperative ultrasound images: From animal models to first in-human testing.

Objects accidentally left behind in the brain following neurosurgical procedures may lead to life-threatening health complications and invasive reoperation. One of the most commonly retained surgical items is the cotton ball, which absorbs blood to clear the surgeon's field of view yet in the process becomes visually indistinguishable from the brain parenchyma. However, using ultrasound imaging, the different acoustic properties of cotton and brain tissue result in two discernible materials. In this study, we created a fully automated foreign body object tracking algorithm that integrates into the clinical workflow to detect and localize retained cotton balls in the brain. This deep learning algorithm uses a custom convolutional neural network and achieves 99% accuracy, sensitivity, and specificity, and surpasses other comparable algorithms. Furthermore, the trained algorithm was implemented into web and smartphone applications with the ability to detect one cotton ball in an uploaded ultrasound image in under half of a second. This study also highlights the first use of a foreign body object detection algorithm using real in-human datasets, showing its ability to prevent accidental foreign body retention in a translational setting.

3D-SiamMask: Vision-Based Multi-Rotor Aerial-Vehicle Tracking for a Moving Object

This paper aims to develop a multi-rotor-based visual tracker for a specified moving object. Visual object-tracking algorithms for multi-rotors are challenging due to multiple issues such as occlusion, quick camera motion, and out-of-view scenarios. Hence, algorithmic changes are required for dealing with images or video sequences obtained by multi-rotors. Therefore, we propose two approaches: a generic object tracker and a class-specific tracker. Both tracking settings require the object bounding box to be selected in the first frame. As part of the later steps, the object tracker uses the updated template set and the calibrated RGBD sensor data as inputs to track the target object using a Siamese network and a machine-learning model for depth estimation. The class-specific tracker is quite similar to the generic object tracker but has an additional auxiliary object classifier. The experimental study and validation were carried out in a robot simulation environment. The simulation environment was designed to serve multiple case scenarios using Gazebo. According to the experiment results, the class-specific object tracker performed better than the generic object tracker in terms of stability and accuracy. Experiments show that the proposed generic tracker achieves promising results on three challenging datasets. Our tracker runs at approximately 36 fps on GPU.