Video Object Tracking Research Articles

Detecting and tracking moving objects in defocus blur scenes

Object tracking stands as a cornerstone challenge within computer vision, with blurriness analysis representing a burgeoning field of interest. Among the various forms of blur encountered in natural scenes, defocus blur remains significantly underexplored. To bridge this gap, this article introduces the Defocus Blur Video Object Tracking (DBVOT) dataset, specifically crafted to facilitate research in visual object tracking under defocus blur conditions. We conduct a comprehensive performance analysis of 18 state-of-the-art object tracking methods on this unique dataset. Additionally, we propose a selective deblurring framework based on Deblurring Auxiliary Learning Net (DID-Anet), innovatively designed to tackle the complexities of defocus blur. This framework integrates a novel defocus blurriness metric for the smart deblurring of video frames, thereby enhancing the efficacy of tracking methods in defocus blur scenarios. Our extensive experimental evaluations underscore the significant advancements in tracking accuracy achieved by incorporating our proposed framework with leading tracking technologies.

Lester: Rotoscope Animation through Video Object Segmentation and Tracking

This article introduces Lester, a novel method to automatically synthesize retro-style 2D animations from videos. The method approaches the challenge mainly as an object segmentation and tracking problem. Video frames are processed with the Segment Anything Model (SAM) and the resulting masks are tracked through subsequent frames with DeAOT, a method of hierarchical propagation for semi-supervised video object segmentation. The geometry of the masks’ contours is simplified with the Douglas–Peucker algorithm. Finally, facial traits, pixelation and a basic rim light effect can be optionally added. The results show that the method exhibits an excellent temporal consistency and can correctly process videos with different poses and appearances, dynamic shots, partial shots and diverse backgrounds. The proposed method provides a more simple and deterministic approach than diffusion models based video-to-video translation pipelines, which suffer from temporal consistency problems and do not cope well with pixelated and schematic outputs. The method is also more feasible than techniques based on 3D human pose estimation, which require custom handcrafted 3D models and are very limited with respect to the type of scenes they can process.

Multiple Object Tracking in Drone Aerial Videos by a Holistic Transformer and Multiple Feature Trajectory Matching Pattern

Drone aerial videos have immense potential in surveillance, rescue, agriculture, and urban planning. However, accurately tracking multiple objects in drone aerial videos faces challenges like occlusion, scale variations, and rapid motion. Current joint detection and tracking methods often compromise accuracy. We propose a drone multiple object tracking algorithm based on a holistic transformer and multiple feature trajectory matching pattern to overcome these challenges. The holistic transformer captures local and global interaction information, providing precise detection and appearance features for tracking. The tracker includes three components: preprocessing, trajectory prediction, and matching. Preprocessing categorizes detection boxes based on scores, with each category adopting specific matching rules. Trajectory prediction employs the visual Gaussian mixture probability hypothesis density method to integrate visual detection results to forecast object motion accurately. The multiple feature pattern introduces Gaussian, Appearance, and Optimal subpattern assignment distances for different detection box types (GAO trajectory matching pattern) in the data association process, enhancing tracking robustness. We perform comparative validations on the vision-meets-drone (VisDrone) and the unmanned aerial vehicle benchmarks; the object detection and tracking (UAVDT) datasets affirm the algorithm’s effectiveness: it obtained 38.8% and 61.7% MOTA, respectively. Its potential for seamless integration into practical engineering applications offers enhanced situational awareness and operational efficiency in drone-based missions.

Method for noninvasive HV/MV switchgear motion analysis using kernel-based algorithm with adaptive feature extraction

When designing, developing, and testing medium-voltage (MV) and high-voltage (HV) switchgears, it is of utmost importance to analyze the movements of their mechanical parts, such as drive trains, contact nozzles, etc. This ensures the safety of switchgear operations and enables detecting and predicting issues that could lead to component damage, power interruptions, reduced efficiency, or even switch failure. Conventional invasive measuring setups, including encoders and laser distance measurements, are often difficult or expensive to use, due to undesirable environmental properties such as high temperatures and/or high voltages, or dimensional constraints often encountered in testing laboratories, compact substations or confined equipment rooms. Video object tracking can be a viable solution in such conditions, allowing for the extraction of the trajectory of mechanical parts of an object under analysis. This paper proposes a novel kernel-based algorithm with adaptive feature extraction for precise colour-based object tracking through video processing. The implemented method is based on the principles of the CamShift algorithm, augmented with fusion with the Kalman filter for continuous estimation and prediction of the object’s position based on the available measurements while reducing sensitivity to noise. Beyond precise tracking of the object of interest, the algorithm automatically adapts the mask used in the standard CamShift algorithm by extracting and processing the features of the selected object. This approach exhibits flexibility and robustness in adverse industrial environments. It supports modifications according to user preferences and represents a cost-effective alternative to conventional methods, while performing real-time processing. Experimental results underscore this noninvasive approach is flexible, highly robust, and enables tracking of coloured markers even in challenging conditions like poor lighting, significant blur, and low frame rates.

Satellite Video Object Tracking Based on Location Prompts

Satellite Video Object Tracking Based on Location Prompts

Deep feature based correlation filter for single object tracking in satellite videos

Deep feature based correlation filter for single object tracking in satellite videos

Block based motion estimation model using CNN with representative point matching algorithm for object tracking in videos

Motion estimation is considered significant for tracking the movement of an object in video sequences, and it is widely used in various video processing applications. Traditionally, many researchers focus on pixel-based motion estimation for object tracking, but it experienced increased computation time and cost. To reduce computation time, the utilization of a block-based motion estimation approach for object tracking is a recent trend. The existing block-based approach faces difficulty in finding representative points within the intensity domain. Therefore, this current research merged the deep learning approach with a block-matching algorithm for achieving efficient object tracking. In this proposed work, initially, video sequences are collected from a benchmark video dataset. Then, the acquired video sequences are segmented into frames. From the segmented frames, current and previous frames are considered for motion estimation. Frames are sent for the data augmentation process in which the process of flipping, cropping, and rotation is carried out. Then, the augmented frames are sent into Convolutional Neural Network (CNN) for feature extraction. Representative Point Matching (RPM) is used to estimate the motion vector based on the extracted features. After estimating the motion vector, the similarity between two consecutive frames is found using Structural Similarity Index (SSIM) technique. Finally, based on the similarity score, the movement of an object in the video is tracked effectively. Simulation analysis of the proposed block-based motion estimation model is done by evaluating some performance metrics. RMSE, PSNR, Execution Time, SSIM, and accuracy obtained for the proposed model are 27.5, 26.5 db, 31 sec, 0.91, and 94 %. This analysis suggested that the proposed CNN-RPM motion estimation model performs better in tracking the movement of the object.

Adaptive and Anti-Drift Motion Constraints for Object Tracking in Satellite Videos

Object tracking in satellite videos has garnered significant attention due to its increasing importance. However, several challenging attributes, such as the presence of tiny objects, occlusions, similar objects, and background clutter interference, make it a difficult task. Many recent tracking algorithms have been developed to tackle these challenges in tracking a single interested object, but they still have some limitations in addressing them effectively. This paper introduces a novel correlation filter-based tracker, which uniquely integrates attention-enhanced bounding box regression and motion constraints for improved single-object tracking performance. Initially, we address the regression-related interference issue by implementing a spatial and channel dual-attention mechanism within the search area’s region of interest. This enhancement not only boosts the network’s perception of the target but also improves corner localization. Furthermore, recognizing the limitations in small size and low resolution of target appearance features in satellite videos, we integrate motion features into our model. A long short-term memory (LSTM) network is utilized to create a motion model that can adaptively learn and predict the target’s future trajectory based on its historical movement patterns. To further refine tracking accuracy, especially in complex environments, an anti-drift module incorporating motion constraints is introduced. This module significantly boosts the tracker’s robustness. Experimental evaluations on the SatSOT and SatVideoDT datasets demonstrate that our proposed tracker exhibits significant advantages in satellite video scenes compared to other recent trackers for common scenes or satellite scenes.

A Quantum Evolutionary Learning Tracker for Video

Video object tracking has been a popular area in the field of computer vision. As video data evolves, more special perspectives and challenging video data are constantly kept up to date. This poses challenges for object tracking tasks and places higher demands on the generalization capabilities of the models. In this paper, we propose a novel quantum evolutionary learning tracker for video. The model combines quantum evolution with deep networks for tracking video objects. The model uses a quantum evolutionary learning tracker to generate a reliable population of candidate regions and a deep network for classification. In particular, the quantum evolutionary predictor predicts the object motion state through rotation operator and trajectory inference, and provides motion state information for the tracker. The predictor can incorporate object history contextual information and can provide stable candidate estimation populations for the model in case of failure of appearance features. Both quantum evolution and deep networks are combined to form an end-to-end online video object tracker. In addition, we propose a new video object tracking evaluation algorithm, Balanced Intersection over Union. The evaluation algorithm uses aspect ratios to balance the share of overlap and distance. Finally, we test the model on the OTB 2015 dataset for natural video and on the SV248A10-SOT dataset for satellite video. The performance of the proposed model is also analyzed and validated by comparing it with more than twenty classical tracker models. The experimental results show that our model has high generalization ability and robustness.

Multi-Scene Generalized Trajectory Global Graph Solver with Composite Nodes for Multiple Object Tracking

The global multi-object tracking (MOT) system can consider interaction, occlusion, and other ``visual blur'' scenarios to ensure effective object tracking in long videos. Among them, graph-based tracking-by-detection paradigms achieve surprising performance. However, their fully-connected nature poses storage space requirements that challenge algorithm handling long videos. Currently, commonly used methods are still generated trajectories by building one-forward associations across frames. Such matches produced under the guidance of first-order similarity information may not be optimal from a longer-time perspective. Moreover, they often lack an end-to-end scheme for correcting mismatches. This paper proposes the Composite Node Message Passing Network (CoNo-Link), a multi-scene generalized framework for modeling ultra-long frames information for association. CoNo-Link's solution is a low-storage overhead method for building constrained connected graphs. In addition to the previous method of treating objects as nodes, the network innovatively treats object trajectories as nodes for information interaction, improving the graph neural network's feature representation capability. Specifically, we formulate the graph-building problem as a top-k selection task for some reliable objects or trajectories. Our model can learn better predictions on longer-time scales by adding composite nodes. As a result, our method outperforms the state-of-the-art in several commonly used datasets.

Enhancing Embedded Object Tracking: A Hardware Acceleration Approach for Real-Time Predictability.

While Siamese object tracking has witnessed significant advancements, its hard real-time behaviour on embedded devices remains inadequately addressed. In many application cases, an embedded implementation should not only have a minimal execution latency, but this latency should ideally also have zero variance, i.e., be predictable. This study aims to address this issue by meticulously analysing real-time predictability across different components of a deep-learning-based video object tracking system. Our detailed experiments not only indicate the superiority of Field-Programmable Gate Array (FPGA) implementations in terms of hard real-time behaviour but also unveil important time predictability bottlenecks. We introduce dedicated hardware accelerators for key processes, focusing on depth-wise cross-correlation and padding operations, utilizing high-level synthesis (HLS). Implemented on a KV260 board, our enhanced tracker exhibits not only a speed up, with a factor of 6.6, in mean execution time but also significant improvements in hard real-time predictability by yielding 11 times less latency variation as compared to our baseline. A subsequent analysis of power consumption reveals our approach's contribution to enhanced power efficiency. These advancements underscore the crucial role of hardware acceleration in realizing time-predictable object tracking on embedded systems, setting new standards for future hardware-software co-design endeavours in this domain.

ResNet50-deep affinity network for object detection and tracking in videos

ResNet50-deep affinity network for object detection and tracking in videos

Deep-feature-based asymmetrical background-aware correlation filter for object tracking

Correlation filter-based video object-tracking algorithms have gained widespread attention due to their efficiency and excellent tracking performance. However, traditional correlation filtering tracking algorithms possess several limitations. (1) They extract image features only by using rule sampling, which ignores the shape information of the target, resulting in insufficient discriminative power of the features. (2) They focus only on the difference between the background and the object, ignoring the effects of more challenging intra-class interference. (3) They do not further evaluate the reliability of the optimal candidate samples, resulting in easy failure in occlusion scenarios. This paper proposes an asymmetric background-aware correlation filter method for object tracking with deep features to solve the above limitations. First, an asymmetrical background-aware sampling method based on the shape information of the object is proposed. This sampling method significantly differs from the symmetrical sampling method in the traditional correlation filter framework. By exploring the shape information of the object, improving the otherness between the background and object samples is easy, thus suppressing the intra-class interferences. Second, deep neural networks are introduced in the correlation filter framework to extract the deep object features and a spatio-temporal regularization factor is adaptively assigned to suppress the boundary effects, intra-class distractors and aberrance between frames. Finally, a multi-modal object pool is constructed to evaluate the optimal candidate sample in each frame. This template pool fully exploits object diversity and solves the tracking drift and failure caused by invalid appearance changes in scenes such as occlusion and vigorous motion scenarios. To validate the effectiveness of the proposed method, it was compared with the state-of-the-art methods on public datasets. The experimental results show that the tracking performance of the proposed method is competitive.

Multiple object detection and tracking from drone videos based on GM-YOLO and multi-tracker

Multiple object tracking in drone videos is a vital vision task with broad application prospects, but most trackers use spatial or appearance clues alone to correlate detections. Our proposed Multi-Tracker uses a novel similarity measure that combines position and appearance information. We designed the GM-YOLO network to provide high-quality detections as input to Multi-Tracker. Add a Coordinate Attention mechanism and a weighted Bidirectional Feature Pyramid Network structure to the Backbone, each feature point's effective receptive field is modeled as a Gaussian distribution. To accurately obtain the motion and appearance features of the object, the adaptive noise covariance Kalman filter is used to get the position information, MB-OSNet network is designed to use global features to learn contour information to retrieve images from a wider field of view while incorporating Part-Level elements that contain more fine-grained data. Finally, the motion and appearance features are jointly compared to realize multi object tracking. The performance of the GM-YOLO object detector and the Multi-Tracker was verified on the VisDrone MOT and UAVDT datasets.

Multibeam Forward-Looking Sonar Video Object Tracking Using Truncated - Sparsity and Aberrances Repression Regularization

Multibeam Forward-Looking Sonar Video Object Tracking Using Truncated - Sparsity and Aberrances Repression Regularization

STFormer: Spatial-Temporal-Aware Transformer for Video Instance Segmentation.

Video instance segmentation (VIS) is a challenging task, requiring handling object classification, segmentation, and tracking in videos. Existing Transformer-based VIS approaches have shown remarkable success, combining encoded features and instance queries as decoder inputs. However, their decoder inputs are low-resolution due to computational cost, resulting in a loss of fine-grained information, sensitivity to background interference, and poor handling of small objects. Moreover, the queries are randomly initialized without location information, hindering convergence efficiency and accurate object instance localization. To address these issues, we propose a novel VIS approach, STFormer, with a spatial-temporal feature aggregation (STFA) module and spatial-temporal-aware Transformer (STT). Specifically, STFA obtains robust high-resolution masked features efficiently for the decoder, while STT's location-guided instance query (LGIQ) improves initial instance queries. STFormer preserves more fine-grained information, improves convergence efficiency, and localizes object instance features accurately. Extensive experiments on YouTube-VIS 2019, YouTube-VIS 2021, and OVIS datasets show that STFormer outperforms mainstream VIS methods.

Heterogeneous Graph Transformer for Multiple Tiny Object Tracking in RGB-T Videos

Heterogeneous Graph Transformer for Multiple Tiny Object Tracking in RGB-T Videos

Hy-Tracker: A Novel Framework for Enhancing Efficiency and Accuracy of Object Tracking in Hyperspectral Videos

Hy-Tracker: A Novel Framework for Enhancing Efficiency and Accuracy of Object Tracking in Hyperspectral Videos

Video object tracking algorithm based on dual-branch online optimization and feature fusion

Video object tracking algorithm based on dual-branch online optimization and feature fusion

Particle Filter-Based Video Object Tracking Scheme With Target Remodeling and Reinitialization and Its Hardware Implementation Using Raspberry Pi

Particle Filter-Based Video Object Tracking Scheme With Target Remodeling and Reinitialization and Its Hardware Implementation Using Raspberry Pi