Object Tracking Research Articles

Clear Image Acquisition by 3D High-Speed Vibration Tracking of Object With Various Shapes Under High Magnification

Clear Image Acquisition by 3D High-Speed Vibration Tracking of Object With Various Shapes Under High Magnification

Learnable Graph Matching: A Practical Paradigm for Data Association.

Data association is at the core of many computer vision tasks, e.g., multiple object tracking, image matching, and point cloud registration. however, current data association solutions have some defects: they mostly ignore the intra-view context information; besides, they either train deep association models in an end-to-end way and hardly utilize the advantage of optimization-based assignment methods, or only use an off-the-shelf neural network to extract features. In this paper, we propose a general learnable graph matching method to address these issues. Especially, we model the intra-view relationships as an undirected graph. Then data association turns into a general graph matching problem between graphs. Furthermore, to make optimization end-to-end differentiable, we relax the original graph matching problem into continuous quadratic programming and then incorporate training into a deep graph neural network with KKT conditions and implicit function theorem. In MOT task, our method achieves state-of-the-art performance on several MOT datasets. For image matching, our method outperforms state-of-the-art methods on a popular indoor dataset, ScanNet. For point cloud registration, we also achieve competitive results.

Spatio-Temporal Contextual Learning for Single Object Tracking on Point Clouds.

Single object tracking (SOT) is one of the most active research directions in the field of computer vision. Compared with the 2-D image-based SOT which has already been well-studied, SOT on 3-D point clouds is a relatively emerging research field. In this article, a novel approach, namely, the contextual-aware tracker (CAT), is investigated to achieve a superior 3-D SOT through spatially and temporally contextual learning from the LiDAR sequence. More precisely, in contrast to the previous 3-D SOT methods merely exploiting point clouds in the target bounding box as the template, CAT generates templates by adaptively including the surroundings outside the target box to use available ambient cues. This template generation strategy is more effective and rational than the previous area-fixed one, especially when the object has only a small number of points. Moreover, it is deduced that LiDAR point clouds in 3-D scenes are often incomplete and significantly vary from frame to another, which makes the learning process more difficult. To this end, a novel cross-frame aggregation (CFA) module is proposed to enhance the feature representation of the template by aggregating the features from a historical reference frame. Leveraging such schemes enables CAT to achieve a robust performance, even in the case of extremely sparse point clouds. The experiments confirm that the proposed CAT outperforms the state-of-the-art methods on both the KITTI and NuScenes benchmarks, achieving 3.9% and 5.6% improvements in terms of precision.

Satellite Video Object Tracking Based on Location Prompts

Satellite Video Object Tracking Based on Location Prompts

TRAFFIC VIOLATION PREDICTION USING DEEP LEARNING BASED ON HELMETS WITH NUMBER PLATE RECOGNITION

At the moment, two-wheelers are the most widely used kind of transportation. It is highly advised that both bike riders and soldiers wear helmets. Many academics are interested in object tracking in video surveillance, which is an important application and burgeoning field of study in image processing and machine learning. Finding objects in an image with a bounding box and different categories or shapes of the objects placed is called object detection. This study reviews tracking techniques, classifies them into several categories, and concentrates on significant and practical tracking approaches. After reviewing broad strategies under a scan of the literature on various methodologies, we analyze potential research areas. This study uses the YOLOv8 algorithm, an image processing technique, to identify motorcycle riders who do not wear helmets. Additionally, put the Optical Character Recognition method into practice to identify the license plate in a picture and retrieve user information. Next, determine the fine amount. Eventually, SMS services will be able to notify consumers in order to prevent motorbike accidents. We evaluate the framework in terms of speed and accuracy.

An efficient object tracking based on multi‐head cross‐attention transformer

AbstractObject tracking is an essential component of computer vision and plays a significant role in various practical applications. Recently, transformer‐based trackers have become the predominant method for tracking due to their robustness and efficiency. However, existing transformer‐based trackers typically focus solely on the template features, neglecting the interactions between the search features and the template features during the tracking process. To address this issue, this article introduces a multi‐head cross‐attention transformer for visual tracking (MCTT), which effectively enhance the interaction between the template branch and the search branch, enabling the tracker to prioritize discriminative feature. Additionally, an auxiliary segmentation mask head has been designed to produce a pixel‐level feature representation, enhancing and tracking accuracy by predicting a set of binary masks. Comprehensive experiments have been performed on benchmark datasets, such as LaSOT, GOT‐10k, UAV123 and TrackingNet using various advanced methods, demonstrating that our approach achieves promising tracking performance. MCTT achieves an AO score of 72.8 on the GOT‐10k.

Peak response regularization for localization

Deep convolutional neural networks approaches often assume that the feature response has a Gaussian distribution with target-centered peak response, which can be used to guide the target location and classification. Nevertheless, such an assumption is implausible when there is progressive interference from other targets and/or background noise, which produces sub-peaks on the tracking response map and causes model drift. In this paper, we propose a feature response regularization approach for sub-peak response suppression and peak response enforcement and aim to handle progressive interference systematically. Our approach, referred to as Peak Response Regularization (PRR), applies simple-yet-efficient method to aggregate and align discriminative features, which convert local extremal response in discrete feature space to extremal response in continuous space, which enforces the localization and representation capability of convolutional features. Experiments on human pose detection, object detection, object tracking, and image classification demonstrate that PRR improves the performance of image tasks with a negligible computational cost.

Hunt-inspired Transformer for visual object tracking

This paper presents a hunt-inspired Transformer for visual object tracking, dubbed as HuntFormer. The HuntFormer focuses on robust target detection and identification, simulating natural hunting processes. Specifically, the HuntFormer comprises two essential module designs including a predictor for detection and a verifier for identification. The predictor emulates the detection stage by designing a motion trajectory guided particle filter, which identifies potential target locations by predicting the motion state within a particle filtering framework. The predictor utilizes spatio-temporal correlation scores between dynamic target templates and the search region to guide the learning process to generate a set of reliable particles. This enables the base tracker to narrow its search range to focus on the target, and swiftly re-detect the target in case of model drift. Once the target is re-detected, the verifier assesses the detection result as a reliable tracked item. The verifier initially maintains a dynamic memory that stores reliable target templates and their corresponding locations in the motion trajectory. It then models the uncertainty of appearance information within this memory probabilistically. The output uncertainty score determines whether the memory gets updated or not. Ultimately, the predictor and the verifier collaborate, ensuring a robust tracking outcome. Extensive evaluations on six challenging benchmark datasets demonstrate HuntFormer’s favorable performance against various state-of-the-art trackers. Notably, in the VOT-LT2022 tracking challenge, the HuntFormer won the third place with an F-score of 0.598, closely competing for the second place with an F-score of 0.600.

Reliable object tracking by multimodal hybrid feature extraction and transformer-based fusion

Visual object tracking, which is primarily based on visible light image sequences, encounters numerous challenges in complicated scenarios, such as low light conditions, high dynamic ranges, and background clutter. To address these challenges, incorporating the advantages of multiple visual modalities is a promising solution for achieving reliable object tracking. However, the existing approaches usually integrate multimodal inputs through adaptive local feature interactions, which cannot leverage the full potential of visual cues, thus resulting in insufficient feature modeling. In this study, we propose a novel multimodal hybrid tracker (MMHT) that utilizes frame-event-based data for reliable single object tracking. The MMHT model employs a hybrid backbone consisting of an artificial neural network (ANN) and a spiking neural network (SNN) to extract dominant features from different visual modalities and then uses a unified encoder to align the features across different domains. Moreover, we propose an enhanced transformer-based module to fuse multimodal features using attention mechanisms. With these methods, the MMHT model can effectively construct a multiscale and multidimensional visual feature space and achieve discriminative feature modeling. Extensive experiments demonstrate that the MMHT model exhibits competitive performance in comparison with that of other state-of-the-art methods. Overall, our results highlight the effectiveness of the MMHT model in terms of addressing the challenges faced in visual object tracking tasks.

Multi-Tracking Sensor Architectures for Reconstructing Autonomous Vehicle Crashes: An Exploratory Study.

With the continuous development of new sensor features and tracking algorithms for object tracking, researchers have opportunities to experiment using different combinations. However, there is no standard or agreed method for selecting an appropriate architecture for autonomous vehicle (AV) crash reconstruction using multi-sensor-based sensor fusion. This study proposes a novel simulation method for tracking performance evaluation (SMTPE) to solve this problem. The SMTPE helps select the best tracking architecture for AV crash reconstruction. This study reveals that a radar-camera-based centralized tracking architecture of multi-sensor fusion performed the best among three different architectures tested with varying sensor setups, sampling rates, and vehicle crash scenarios. We provide a brief guideline for the best practices in selecting appropriate sensor fusion and tracking architecture arrangements, which can be helpful for future vehicle crash reconstruction and other AV improvement research.

Motioninsights: real-time object tracking in streaming video

Motioninsights: real-time object tracking in streaming video

The Effects of High-Intensity Interval Training on Cognitive and Physical Skills in Basketball and Soccer Players.

As athletes pursue excellence, training techniques continue to advance, making structured physical activity an essential tool for enhancing performance. To optimize athletic performance in modern competitive sports, the balance of physical performance and mental clarity is required. This study seeks to examine the effects of High-Intensity Interval Training (HIIT) on cognitive and physical skills in basketball and soccer players. A 3-week HIIT protocol was incorporated based on the Wingate technique. This study included 10 soccer players and 10 basketball players with an average age of 22.79 ± 1.90 years. Participants performed pre- and post-intervention assessments. Physical proficiency was assessed using 20 m sprint, change-of-direction (COD) and dribbling tests, while cognitive skills were assessed using motion object tracking (MOT), working memory, perceptual load (PL), and attention window (AW) tests. The HIIT intervention significantly improved cognitive performance in particular; noteworthy observations were a 15% improvement in motion object tracking test scores and a 16% increase in working memory test scores for basketball players. The attention window test scores showed a 32% increase, and perceptual load test scores were 31% decreased for soccer players post-intervention. There were significant improvements in physical skills; for example, sprint times were decreased by 6%, and change-of-direction and dribbling times were reduced by 8% and 7%, respectively, indicating improved agility, speed, and ball control abilities. In conclusion, both groups performed significantly better on cognitive and physical skill tests post-HIIT intervention.

SGRTmreg: A Learning-Based Optimization Framework for Multiple Pairwise Registrations.

Point cloud registration is a fundamental task in computer vision and graphics, which is widely used in 3D reconstruction, object tracking, and atlas reconstruction. Learning-based optimization and deep learning methods have been widely developed in pairwise registration due to their own distinctive advantages. Deep learning methods offer greater flexibility and enable registering unseen point clouds that are not trained. Learning-based optimization methods exhibit enhanced robustness and stability when handling registration under various perturbations, such as noise, outliers, and occlusions. To leverage the strengths of both approaches to achieve a less time-consuming, robust, and stable registration for multiple instances, we propose a novel computational framework called SGRTmreg for multiple pairwise registrations in this paper. The SGRTmreg framework utilizes three components-a Searching scheme, a learning-based optimization method called Graph-based Reweighted discriminative optimization (GRDO), and a Transfer module to achieve multi-instance point cloud registration.Given a collection of instances to be matched, a template as a target point cloud, and an instance as a source point cloud, the searching scheme selects one point cloud from the collection that closely resembles the source. GRDO then learns a sequence of regressors by aligning the source to the target, while the transfer module stores and applies the learned regressors to align the selected point cloud to the target and estimate the transformation of the selected point cloud. In short, SGRTmreg harnesses a shared sequence of regressors to register multiple point clouds to a target point cloud. We conduct extensive registration experiments on various datasets to evaluate the proposed framework. The experimental results demonstrate that SGRTmreg achieves multiple pairwise registrations with higher accuracy, robustness, and stability than the state-of-the-art deep learning and traditional registration methods.

Kalman Filter for a Particular Class of Dynamic Object Images

We discuss the problem of estimating the state of a dynamic object by using observed images generated by an optical system. The work aims to implement a novel approach that would ensure improved accuracy of dynamic object tracking using a sequence of images. We utilize a vector model that describes the object image as a limited number of vertexes (reference points). Upon imaging, the object of interest is assumed to be retained at the center of each frame, so that the motion parameters can be considered as projections onto the axes of a coordinate system matched with the camera's optical axis. The novelty of the approach is that the observed parameters (the distance along the optical axis and angular attitude) of the object are calculated using the coordinates of specified points in the object images. For estimating the object condition, a Kalman-Bucy filter is constructed on the assumption that the dynamic object motion is described by a set of equations for the translational motion of the center of mass along the optical axis and variations in the angular attitude relative to the image plane. The efficiency of the proposed method is illustrated by an example of estimating the object's angular attitude.

Dynamic center point learning for multiple object tracking under Severe occlusions

Multiple Object Tracking (MOT) methods based on per-pixel prediction and association have achieved remarkable progress recently. These approaches prefer to select points in central region of the bounding box as positive samples and other points as negative samples. Under severe occlusions, these sample allocation methods might lead to the central region being contaminated by occluding samples, resulting in significant degradation of both detection and association performance. To address this issue, we propose a novel Dynamic-Center-Point-based Multiple Object Tracking method (DCP-MOT), which aims to self-identify the visible center region for occluded objects. Compared with the previous bounding box center point, our Dynamic-Center-Point (DCP) can better represent the visible region of the occluded object. Specifically, we design an Iterative Refinement Branch to generate dynamic center points for each occluded object. It includes two parts: Center Probability Predictor and Center Generator. Initially, we utilize the Center Probability Predictor to derive an accurate probability map for the occluded object. Subsequently, we use Center Generator to quantify the probability map by introducing a Mutual Exclusive Potential Function, yielding a dynamic center point for the occluded object that is distinct from its occluding counterpart. Finally, by joining the bounding box center points for unoccluded objects and our dynamic center points for occluded objects, our JDE branch can achieve better tracking performance. Extensive experiments demonstrate our DCP-based MOT method surpasses the bounding box center based SOTA in metrics MOTA (+0.7,+2.1,+1.3) and IDF1 (+0.7,+0.9,+1.5) on the challenging MOT16, MOT17, MOT20 datasets.

Entertainment robot based on IoT and VR interaction for motion training posture monitoring

With the development of artificial intelligence technology, the role of social robots in sports activities has gradually become prominent. Robots can help with intelligent control of sports venues, assist in motion guidance, and engage in emotional communication with users, helping athletes adjust their emotions. This article studies the motion training posture monitoring of entertainment robots based on the interaction between the Internet of Things and VR. A new high-intensity motion attitude monitoring system based on quantum dot photodetectors has been developed to improve monitoring accuracy and stability. Entertainment robots can simulate and analyze motion training postures, and provide relevant technical guidance. The quantum dot photodetectors were integrated into the sensor components, and the collected data was transmitted to the data acquisition and processing unit through signal transmission and processing modules. The collected data was processed and analyzed to achieve real-time monitoring and tracking of the target object’s attitude. Multiple experiments were conducted on the entire system, simulating various high-intensity exercise environments, and evaluating the performance parameters of the system. Through continuous optimization and improvement, the stability and accuracy of the system were ultimately ensured.

Telescopic Network of Zhulong for Orbit Determination and Prediction of Space Objects

The increasing proliferation of space debris, intermittent space incidents, and the rapid emergence of massive LEO satellite constellations pose significant threats to satellites in orbit. Ground-based optical observations play a crucial role in space surveillance and space situational awareness (SSA). The Zhulong telescopic observation network stands as a pivotal resource in the realm of space object tracking and prediction. This publicly available network plays a critical role in furnishing essential data for accurately delineating and forecasting the orbit of space objects in Earth orbit. Comprising a sophisticated array of hardware components including precise telescopes, optical sensors, and image sensors, the Zhulong network synergistically collaborates to achieve unparalleled levels of precision in tracking and observing space objects. Central to the network’s efficacy is its ability to extract positional information, referred to as angular data, from consecutive images. These angular data serve as the cornerstone for precise orbit determination and prediction. In this study, the CPF (Consolidated Prediction Format) orbit serves as the reference standard against which the accuracy of the angular data is evaluated. The findings reveal that the angular data error of the Zhulong network remains consistently below 3 arcseconds, attesting to its remarkable precision. Moreover, through the accumulation of angular data over time, coupled with the utilization of numerical integration and least squares methods, the Zhulong network facilitates highly accurate orbit determination and prediction for space objects. These methodologies leverage the wealth of data collected by the network to extrapolate trajectories with unprecedented accuracy, offering invaluable insights into the behavior and movement of celestial bodies. The results presented herein underscore the immense potential of electric optic telescopes in the realm of space surveillance. By harnessing the capabilities of the Zhulong network, researchers and astronomers can gain deeper insights into the dynamics of space objects, thereby advancing our understanding of the cosmos. Ultimately, the Zhulong telescopic observation network emerges as a pioneering tool in the quest to unravel the mysteries of the universe.

High-Magnification Object Tracking with Ultra-Fast View Adjustment and Continuous Autofocus Based on Dynamic-Range Focal Sweep.

Active vision systems (AVSs) have been widely used to obtain high-resolution images of objects of interest. However, tracking small objects in high-magnification scenes is challenging due to shallow depth of field (DoF) and narrow field of view (FoV). To address this, we introduce a novel high-speed AVS with a continuous autofocus (C-AF) approach based on dynamic-range focal sweep and a high-frame-rate (HFR) frame-by-frame tracking pipeline. Our AVS leverages an ultra-fast pan-tilt mechanism based on a Galvano mirror, enabling high-frequency view direction adjustment. Specifically, the proposed C-AF approach uses a 500 fps high-speed camera and a focus-tunable liquid lens operating at a sine wave, providing a 50 Hz focal sweep around the object's optimal focus. During each focal sweep, 10 images with varying focuses are captured, and the one with the highest focus value is selected, resulting in a stable output of well-focused images at 50 fps. Simultaneously, the object's depth is measured using the depth-from-focus (DFF) technique, allowing dynamic adjustment of the focal sweep range. Importantly, because the remaining images are only slightly less focused, all 500 fps images can be utilized for object tracking. The proposed tracking pipeline combines deep-learning-based object detection, K-means color clustering, and HFR tracking based on color filtering, achieving 500 fps frame-by-frame tracking. Experimental results demonstrate the effectiveness of the proposed C-AF approach and the advanced capabilities of the high-speed AVS for magnified object tracking.

Prior-free 3D tracking of a fast-moving object at 6667 frames per second with single-pixel detectors.

Real-time tracking and 3D trajectory computation of fast-moving objects is a promising technology, especially in the field of autonomous driving. However, existing image-based tracking methods face significant challenges when it comes to real-time tracking, primarily due to the limitation of storage space and computational resources. Here, we propose a novel approach that enables real-time 3D tracking of a fast-moving object without any prior motion information and at a very low computational cost. To enable 3D coordinate synthesis with a space-efficient optical setup, geometric moment patterns are projected on two non-orthogonal planes with a spatial resolution of 125 μm. Our experiment demonstrates an impressive tracking speed of 6667 frames per second (FPS) with a 20 kHz digital micromirror device (DMD), which is more than 200 times faster than the widely adopted video-based tracking methods. To the best of our knowledge, this is the highest tracking speed record in the field of single-pixel 3D trajectory tracking. This method promotes the development of real-time tracking techniques with single-pixel imaging (SPI).

A Transformer-based visual object tracker via learning immediate appearance change

Transformer has shown its great strength in visual object tracking due to its effective attention mechanism, but most prevailing transformer-based trackers only explore temporal information frame by frame, thus overlooking the rich context information inherent in videos. To alleviate this problem, we propose a transformer-based tracker via learning immediate appearance change information in videos, called IAC-tracker. The proposed tracker enhances the perception of the immediate motion state to improve the performance of single target tracking. IAC-tracker contains three key components: a spatial information extractor (SIE) with a superior attention mechanism to progressively extract spatial information, a temporal information extractor (TIE) with a designed temporal attention mechanism to progressively learn target immediate appearance change, and a novel spatial–temporal context enhanced fusion module integrating the information from SIE and TIE to prepare for the final prediction head. Comparison experiments with state-of-the-art trackers on six challenging datasets demonstrate the superior performance of IAC-tracker with real-time running speed.