Object Tracking Research Articles

TOSS: Deep Learning-Based Track Object Detection Using Smart Sensor

TOSS: Deep Learning-Based Track Object Detection Using Smart Sensor

Construction and Study of an Improved PP-Human Multi-Objective Tracking Method

Multiple objective tracking represents a crucial research focus within the realm of computer vision. It has a very broad application prospect in video surveillance, intelligent transportation, robot navigation and positioning and other fields.However, multi-target visual perception is affected by light, weather, occlusion and other factors, and is vulnerable to noise interference, faced with problems such as unstable image enhancement effect, target correlation accuracy and robustness.In this study, the PP-Human framework is applied to multi-target tracking. Through training and refinement of the PP-Human model, integration of a high-accuracy detector, enhancement of pedestrian re-identification (ReID) techniques, and optimization of the data association approach, the model's multi-object detection performance is enhanced, achieving efficient and precise multi-target tracking. Using the extended Market 150 dataset to comprehensively evaluate the performance of the proposed multi-objective tracking method, this paper improves the accuracy of multi-objective tracking by 5.0% and reaches 95.0% of MOTA through a series of optimization measures.To validate the efficacy and robustness of this approach, experimental evaluations were conducted. The refined PP-Human framework demonstrated strong performance in multi-target tracking tasks, establishing a reliable basis for practical applications such as pedestrian analysis, behavior recognition, and flow statistics.

Improving Visual Object Detection using General UFIR Filtering

Object detection is a fundamental task in computer vision, which involves the identification and localization of objects within image frames or video sequences. The problem is complicated by large variations in the video camera bounding box, which can be thought of as colored measurement noise (CMN). In this paper, we use the general unbiased finite impulse response (GUFIR) approach to improve detection performance under CMN. The results are compared to the general Kalman filter (GKF) and two detection methods: “Faster-RCNN” and “Tensorflow PASCAL Visual Object Classes (VOC)”. Experimental testing is carried out using the benchmark data ”Car4”. It is shown that GUFIR significantly improves the detection accuracy and demonstrates the properties of the effective tool for visual object tracking.

New Astronomical Observatory Design for the Detection and Tracking of Satellite Objects: The Satellite Robotic Observatory (SRO)

Robotic Astronomical Observatories (RAOs) have provided very good results in different research projects in astrophysics/astronomy. Their applications in the detection, tracking, and identification of near-Earth objects have contributed to the identification of potentially dangerous objects for our security, such as near-Earth Objects (NEOs), near-Earth Asteroids (NEAs), meteors, and comets, whose trajectory changes can cause an impact on our planet. If advances in astrometry techniques (measuring the position and trajectory of Earth-orbiting objects) and photometry (variation in light curves) are considered together with the new sensors that work in the optical and near-infrared spectral ranges, a new observatory system that allows for the detection of nearby satellite objects in different spectral ranges and with better-defined optics can be developed. The present paper describes the design of a new observatory applied to the surveillance and tracking of satellites and other debris objects, the Satellite Robotic Observatory (SRO). Starting from general constraints from astronomy observatories, the design process has been determined, considering the main objectives, the necessary sensors, and several technical improvements that have contributed to a final configuration proposed for the SRO. The result is the design of a portable observatory model that can host at least two sensors to track and monitor satellite objects simultaneously.

Evaluation of Cluster Algorithms for Radar-Based Object Recognition in Autonomous and Assisted Driving

Perception systems for assisted driving and autonomy enable the identification and classification of objects through a concentration of sensors installed in vehicles, including Radio Detection and Ranging (RADAR), camera, Light Detection and Ranging (LIDAR), ultrasound, and HD maps. These sensors ensure a reliable and robust navigation system. Radar, in particular, operates with electromagnetic waves and remains effective under a variety of weather conditions. It uses point cloud technology to map the objects in front of you, making it easy to group these points to associate them with real-world objects. Numerous clustering algorithms have been developed and can be integrated into radar systems to identify, investigate, and track objects. In this study, we evaluate several clustering algorithms to determine their suitability for application in automotive radar systems. Our analysis covered a variety of current methods, the mathematical process of these methods, and presented a comparison table between these algorithms, including Hierarchical Clustering, Affinity Propagation Balanced Iterative Reducing and Clustering using Hierarchies (BIRCH), Density-Based Spatial Clustering of Applications with Noise (DBSCAN), Mini-Batch K-Means, K-Means Mean Shift, OPTICS, Spectral Clustering, and Gaussian Mixture. We have found that K-Means, Mean Shift, and DBSCAN are particularly suitable for these applications, based on performance indicators that assess suitability and efficiency. However, DBSCAN shows better performance compared to others. Furthermore, our findings highlight that the choice of radar significantly impacts the effectiveness of these object recognition methods.

Object Detection and Tracking in Maritime Environments in Case of Person-Overboard Scenarios: An Overview

We examine the current state of the art and the related research on the automated detection and tracking of small objects—or persons—in the context of a person-overboard (POB) scenario and present the associated governing relationship between different technologies, platforms, and approaches as a system of systems. A novel phase model, structuring a POB scenario, comprises three phases: (1) detection, (2) search and track, and (3) rescue. Within these phases, we identify the central areas of responsibility and describe in detail the phases (1) and (2). We emphasize the importance of a high-level representation of different systems and their interactions to comprehensively represent the complexity and dynamics of POB scenarios. Our systematic classification and detailed description of the technologies and methods used provide valuable insights to support future regulatory and research activities. Our primary aim is to advance the development of corresponding technologies and standards.

The Future of Radar Space Observation in Europe—Major Upgrade of the Tracking and Imaging Radar (TIRA)

The use of near-Earth space has grown dramatically during the last decades, resulting in thousands of active and inactive satellites and a huge amount of space debris. To observe and monitor the near-Earth space environment, radar systems play a major role as they can be operated at any time and under any weather conditions. The Tracking and Imaging Radar (TIRA) is one of the largest space observation radars in the world. It consists of a 34m Cassegrain antenna, a precise tracking radar, and a high-resolution imaging radar. Since the 1990s, TIRA contributes to the field of space domain awareness by tracking and imaging space objects and by monitoring the debris population. Due to new technologies, modern satellites become smaller, and satellite extensions become more compact. Thus, sensitive high-resolution space observation systems are needed to detect, track, and image these space objects. To fulfill these requirements, TIRA is undergoing a major upgrade. The current imaging radar in the Ku band will be replaced by a new radar with improved geometrical and radiometric resolution operating in the Ka band. Due to its wideband fully polarimetric capability, the new imaging radar will increase the analysis and characterization of space objects. In addition, the tracking radar in the L band is also being currently refurbished. Through its novel modular structure and open design, highly flexible radar modes and precise tracking concepts can be efficiently implemented for enhanced space domain awareness. The new TIRA system will mark the start of a new era for space observation with radar in Europe.

Integrating computer vision and AI for interactive augmented reality experiences in new media

Abstract. Augmented reality (AR) is a groundbreaking technology that fully immerse the user in a mixed 'reality' where the real and virtual coexist in a unique manner. Integrating more artificial intelligence (AI) and computer vision into AR devices can greatly improve user input and provide a whole novel interface. Through AI technology, such as gesture recognition, object tracking, and face recognition, AR systems can offer more intuitive and engaging interactions. An AR system featuring such improved AI technologies can process real-time data while having the contextual awareness to respond to users' input and the surrounding environment on the fly. It can also provide narrative integration, character development, and dynamic environments to users, thereby enabling them to have a more personalised and meaningful experience. This paper examines the evolution of new media by discussing the possibilities of using AI and computer vision in AR devices to create personalised experiences, all the while critically looking at technical challenges and the opportunities they present to the field of future research and development. It also looks at case studies across different sectors, such as education, training, tourism, gaming, retail, and aviation to justify the potential of future development of AI-enhanced AR.

Advancements in deep learning for visual object tracking

Abstract. Since contemporary information-retrieval systems rely heavily on the content of titles and abstracts to identify relevant articles in literature searches, great care should be taken in constructing both. This comprehensive review delves into the transformative impact of deep learning on the domain of visual object tracking. Since the inception of AlexNet in 2012, deep learning has revolutionized feature extraction, leading to significant advancements in tracking accuracy and robustness. The article explores the integration of deep learning with various tracking algorithms, including deep correlation filters, classification-based approaches, Siamese networks, gradient-based methods, and the innovative application of Transformer architectures. Moreover, the role of tracking datasets in fostering algorithm development and innovation is highlighted, with an emphasis on the expansion in scale, diversity, and annotation quality. Furthermore, the article also examines the multifaceted evaluation metrics for tracking algorithms, encompassing precision, robustness, efficiency, generalization, and real-time capabilities. Looking ahead, the review outlines future research directions, such as algorithm optimization for lightweight and accelerated performance, enhancing generalizability, leveraging multimodal data fusion, and refining Transformer models for improved temporal information processing. The challenges of long-term tracking and the growing importance of algorithm interpretability and transparency are also discussed. In summary, the article underscores the promising trajectory of deep learning in visual object tracking, with ongoing research poised to make tracking technologies smarter, more efficient, and robust, catering to a wide array of practical applications and environments.

Fully integrated multi-mode optoelectronic memristor array for diversified in-sensor computing.

In-sensor computing, which integrates sensing, memory and processing functions, has shown substantial potential in artificial vision systems. However, large-scale monolithic integration of in-sensor computing based on emerging devices with complementary metal-oxide-semiconductor (CMOS) circuits remains challenging, lacking functional demonstrations at the hardware level. Here we report a fully integrated 1-kb array with 128 × 8 one-transistor one-optoelectronic memristor (OEM) cells and silicon CMOS circuits, which features configurable multi-mode functionality encompassing three different modes of electronic memristor, dynamic OEM and non-volatile OEM (NV-OEM). These modes are configured by modulating the charge density within the oxygen vacancies via synergistic optical and electrical operations, as confirmed by differential phase-contrast scanning transmission electron microscopy. Using this OEM system, three visual processing tasks are demonstrated: image sensory pre-processing with a recognition accuracy enhanced from 85.7% to 96.1% by the NV-OEM mode, more advanced object tracking with 96.1% accuracy using both dynamic OEM and NV-OEM modes and human motion recognition with a fully OEM-based in-sensor reservoir computing system achieving 91.2% accuracy. A system-level benchmark further shows that it consumes over 20 times less energy than graphics processing units. By monolithically integrating the multi-functional OEMs with Si CMOS, this work provides a cost-effective platform for diverse in-sensor computing applications.

Improved multi object tracking with locality sensitive hashing

Improved multi object tracking with locality sensitive hashing

An effective Key Frame Extraction technique based on Feature Fusion and Fuzzy-C means clustering with Artificial Hummingbird

Key frame extraction is very important in video summarization and content-based video analysis to address the problem of data redundancy in a video. Key frame extraction enables quick navigation and expert video arrangement in many applications. The visually impaired can benefit from the use of key frame extraction for rapid object recognition and tracking. Most key frame extraction techniques consider only a single visual feature instead of multiple features or full pictorial information of the video. This study proposes a key frame extraction method from a video that (i) first removes insignificant frames by pre-processing, (ii) second, four visual and structural feature differences among the consecutive frames are extracted and aggregated to identify informative frames, (iii) third, to cluster the obtained frames, a hybrid FCM-AHA method is proposed by combining Fuzzy C-means(FCM) with artificial hummingbird optimization algorithm (AHA) to circumvent the local minima trapping problem of FCM, and finally, from each cluster, the two frames having greatest Euclidean distance from all the other frames within a cluster is selected as key frames to remove redundant frames. Experimental results on the Open video and YouTube datasets show that the suggested method outperforms state-of-the-art methods both in terms of subjective qualitative analysis and objective quantitative evaluation, e.g., Precision, Recall, and F-score. Further, results are also taken on real video to demonstrate its applicability in real-life applications.

Object Tracking Algorithm Based on Integrated Multi-Scale Templates Guided by Judgment Mechanism

The object tracking algorithm TransT, based on Transformer, achieves significant improvements in accuracy and success rate by fusing the extracted features of convolutional neural networks with the structure of Transformer. However, when dealing with the deformation of the object’s appearance, the algorithm exhibits issues such as insufficient tracking accuracy and drift, which directly affect the stability of the algorithm. In order to overcome this problem, this paper demonstrates how to expand scene information at the scale level during the fusion process and, on this basis, achieve accurate recognition and positioning. The predicted results are promptly fed back to the subsequent tracking process, from which temporal templates are embedded. Starting from both location and time can effectively improve the adaptive ability of the tracking model. In the final experimental comparison results, the algorithm proposed in this paper can adapt well to the situation of object deformation, and the overall performance of the tracking model has also been improved.

Moving objects detection based on histogram of oriented gradient algorithm chip for hazy environment

<span>The most important aspects of computer vision are moving object detection (MOD) and tracking. Many signal-processing applications use regional image statistics. Compute-intensive video and image processing with low latency and high throughput is done with field programmable gate array (FPGA) image processing. Local image statistics are used for edge identification and filtering. The histogram of oriented gradients (HoG) algorithm extracts local shape characteristics by equalizing histograms. The objective of the work is to design the hardware chip of the algorithm and perform the simulation in the Xilinx ISE 14.7 simulation environment. The performance of the chip is evaluated in Modelsim 10.0 simulation software to check its feasibility. The performance of the chip design is estimated on Viretx-5 FPGA and compared with the MATLAB-2020 image processing tool-based response time. This form of tracking typically deals with identifying, anchoring, and tracking images and videos. A mask made from a cut-out of the object can then determine the plane's coordinates depending on its position. This type of object tracking is frequently utilized in the field of augmented reality (AR). The algorithm is most suited for object detection using hardware controllers in haze and foggy environments.</span>

A Comparison of Thunderstorm Identification Methods

Abstract Climatological analysis of thunderstorm occurrence is important since thunderstorms have far-reaching societal impacts. Historically, the majority of thunderstorm-related climatologies have focused on Eulerian frames of reference resulting in thunderstorm day or lightning density summaries. However, a Lagrangian frame of reference is required to answer more in-depth questions such as spatial distribution and variability in thunderstorm initiation. Lagrangian approaches generally utilize object identification and tracking algorithms with tracks then classified as being a thunderstorm or not. Prior methods for thunderstorm identification have been based on classification criteria such as radar reflectivity thresholds, a combination of radar reflectivity thresholds and temporal longevity thresholds, or a combination of radar reflectivity and cloud-to-ground lightning occurrence. However, a comparison of different methods and an evaluation of their accuracy have not been documented in the peer-reviewed literature. This study examines the impact of thunderstorm identification methodology choice. Differences in three thunderstorm identification methodologies are examined over the central plains of the United States during an 11-yr study period. Results show that there is a sensitivity to methodological choice with differences in the number of candidate thunderstorms classified as thunderstorms, as well as the resulting spatial pattern of thunderstorms. Additionally, this study provides insight into the accuracy of each method via a multiyear comparison of each method to regional lightning mapping array total lightning detections where significant differences were observed between methods with the method incorporating cloud-to-ground lightning data providing the best performance metrics. Significance Statement Previous research has utilized a variety of criteria to identify thunderstorms in order to generate datasets of thunderstorm occurrence. This study directly compares commonly used criteria, demonstrating that the choice of thunderstorm identification criteria impacts the composition and accuracy of resulting datasets of thunderstorm occurrence.

CTIFTrack: Continuous Temporal Information Fusion for object track

In visual tracking tasks, researchers usually focus on increasing the complexity of the model or only discretely focusing on the changes in the object itself to achieve accurate recognition and tracking of the moving object. However, they often overlook the significant contribution of video-level linear temporal information fusion and continuous spatiotemporal mapping to tracking tasks. This oversight may lead to poor tracking performance or insufficient real-time ability of the model in complex scenes. Therefore, this paper proposes a real-time tracker, namely Continuous Temporal Information Fusion Tracker (CTIFTrack). The key of CTIFTrack lies in its well-designed Temporal Information Fusion (TIF) module, which cleverly performs a linear fusion of the temporal information between the (t-1)-th and the t-th frames and completes the spatiotemporal mapping. This enables the tracker to better understand the overall spatiotemporal information and contextual spatiotemporal correlations within the video, thereby having a positive impact on the tracking task. In addition, this paper also proposes the Object Template Feature Refinement (OTFR) module, which effectively captures the global information and local details of the object, and further improves the tracker’s understanding of the object features. Extensive experiments are conducted on seven benchmarks, such as LaSOT, GOT-10K, UAV123, NFS, TrackingNet, VOT2018 and OTB-100. The experimental results validate the significant contribution of the TIF module and OTFR module to the tracking task, as well as the effectiveness of CTIFTrack. It is worth noting that while maintaining excellent tracking performance, CTIFTrack also shows outstanding real-time tracking speed. On the Nvidia Tesla T4-16GB GPU, the FPS of CTIFTrack reaches 71.98. The code and demo materials will be available at https://github.com/vpsg-research/CTIFTrack.

Automatic Apple Detection and Counting with AD-YOLO and MR-SORT.

In the production management of agriculture, accurate fruit counting plays a vital role in the orchard yield estimation and appropriate production decisions. Although recent tracking-by-detection algorithms have emerged as a promising fruit-counting method, they still cannot completely avoid fruit occlusion and light variations in complex orchard environments, and it is difficult to realize automatic and accurate apple counting. In this paper, a video-based multiple-object tracking method, MR-SORT (Multiple Rematching SORT), is proposed based on the improved YOLOv8 and BoT-SORT. First, we propose the AD-YOLO model, which aims to reduce the number of incorrect detections during object tracking. In the YOLOv8s backbone network, an Omni-dimensional Dynamic Convolution (ODConv) module is used to extract local feature information and enhance the model's ability better; a Global Attention Mechanism (GAM) is introduced to improve the detection ability of a foreground object (apple) in the whole image; a Soft Spatial Pyramid Pooling Layer (SSPPL) is designed to reduce the feature information dispersion and increase the sensory field of the network. Then, the improved BoT-SORT algorithm is proposed by fusing the verification mechanism, SURF feature descriptors, and the Vector of Local Aggregate Descriptors (VLAD) algorithm, which can match apples more accurately in adjacent video frames and reduce the probability of ID switching in the tracking process. The results show that the mAP metrics of the proposed AD-YOLO model are 3.1% higher than those of the YOLOv8 model, reaching 96.4%. The improved tracking algorithm has 297 fewer ID switches, which is 35.6% less than the original algorithm. The multiple-object tracking accuracy of the improved algorithm reached 85.6%, and the average counting error was reduced to 0.07. The coefficient of determination R2 between the ground truth and the predicted value reached 0.98. The above metrics show that our method can give more accurate counting results for apples and even other types of fruit.

Asymmetric Deformable Spatio-temporal Framework for Infrared Object Tracking

The Infrared Object Tracking (IOT) task aims to locate objects in infrared sequences. Since color and texture information is unavailable in infrared modality, most existing infrared trackers merely rely on capturing spatial contexts from the image to enhance feature representation, where other complementary information is rarely deployed. To fill this gap, we in this article propose a novel Asymmetric Deformable Spatio-Temporal Framework (ADSF) to fully exploit collaborative shape and temporal clues in terms of the objects. Firstly, an asymmetric deformable cross-attention module is designed to extract shape information, which attends to the deformable correlations between distinct frames in an asymmetric manner. Secondly, a spatio-temporal tracking framework is coined to learn the temporal variance trend of the object during the training process and store the template information closest to the tracking frame when testing. Comprehensive experiments demonstrate that ADSF outperforms state-of-the-art methods on three public datasets. Extensive ablation experiments further confirm the effectiveness of each component in ADSF. Furthermore, we conduct generalization validation to demonstrate that the proposed method also achieves performance gains in RGB-based tracking scenarios.

End-to-end multiple object tracking in high-resolution optical sensors of drones with transformer models

Drone aerial imaging has become increasingly important across numerous fields as drone optical sensor technology continues to advance. One critical challenge in this domain is achieving both accurate and efficient multi-object tracking. Traditional deep learning methods often separate object identification from tracking, leading to increased complexity and potential performance degradation. Conventional approaches rely heavily on manual feature engineering and intricate algorithms, which can further limit efficiency. To overcome these limitations, we propose a novel Transformer-based end-to-end multi-object tracking framework. This innovative method leverages self-attention mechanisms to capture complex inter-object relationships, seamlessly integrating object detection and tracking into a unified process. By utilizing end-to-end training, our approach simplifies the tracking pipeline, leading to significant performance improvements. A key innovation in our system is the introduction of a trajectory detection label matching technique. This technique assigns labels based on a comprehensive assessment of object appearance, spatial characteristics, and Gaussian features, ensuring more precise and logical label assignments. Additionally, we incorporate cross-frame self-attention mechanisms to extract long-term object properties, providing robust information for stable and consistent tracking. We further enhance tracking performance through a newly developed self-characteristics module, which extracts semantic features from trajectory information across both current and previous frames. This module ensures that the long-term interaction modules maintain semantic consistency, allowing for more accurate and continuous tracking over time. The refined data and stored trajectories are then used as input for subsequent frame processing, creating a feedback loop that sustains tracking accuracy. Extensive experiments conducted on the VisDrone and UAVDT datasets demonstrate the superior performance of our approach in drone-based multi-object tracking.

Stepwise virus assembly in the cell nucleus revealed by spatiotemporal click chemistry of DNA replication.

Biomolecular assemblies are fundamental to life and viral disease. The spatiotemporal coordination of viral replication and assembly is largely unknown. Here, we developed a dual-color click chemistry procedure for imaging adenovirus DNA (vDNA) replication in the cell nucleus. Late- but not early-replicated vDNA was packaged into virions. Early-replicated vDNA segregated from the viral replication compartment (VRC). Single object tracking, superresolution microscopy, fluorescence recovery after photobleaching, and correlative light-electron microscopy revealed a stepwise assembly program involving vDNA and capsid intermediates. Depending on replication and the scaffolding protein 52K, late-replicated vDNA with rapidly exchanging green fluorescent protein-tagged capsid linchpin protein V and incomplete virions emerged from the VRC periphery. These nanogel-like puncta exhibited restricted movements and were located with the capsid proteins hexon, VI, and virions in the nuclear periphery, suggestive of sites for virion formation. Our findings identify VRC dynamics and assembly intermediates, essential for stepwise productive adenovirus morphogenesis.