Multiple Object Tracking Research Articles

Extensive Familiarization Is Required Before Assessing Acute Changes in Multiple Object Tracking Performance.

The Neurotracker CORE assessment is an 8-minute multiple object tracking (MOT) program used in sport science research and clinical rehabilitation as a perceptual-cognitive training tool; however, it has garnered interest for its potential use as an acute assessment of cognitive performance. Although some data exist regarding the learning effect of repeated exposures, it is often overlooked with investigators focusing primarily on the presence of transfer effects to other cognitive realms. As a result, exclusive data on the effect of repeated testing, or subsequent periods of no testing (ie,detraining) on test-retest reliability, and on MOT performance are sparse. Repeated-measures/reliability. Twenty-three recreationally active men and women completed 15 training sessions consisting of 2 CORE assessments per session (30 assessments). Participants were randomized to either 1 or 2weeks of detraining prior to completing 15 retraining sessions (30 assessments). Training and retraining periods were divided into 10 blocks (3 assessments/block) for analysis. MOT speed threshold (MOT-ST), consistency, fastest trial score success speed, lowest trial score miss speed, the number of perfect, near misses, and significant miss trials within each block were used to determine performance. Intraclass correlation coefficient, standard error of measurement, and minimal detectable change were used to determine reliability. Significant improvements in MOT-ST and fastest trial score success speed were noted within training blocks 1 to 6 and 1 to 7, respectively (P < .05). MOT-ST and fastest trial score success speed demonstrated excellent test-retest reliability between blocks 8 and 9. There was no effect of detraining period on performance during retraining. Eighteen tests are necessary to overcome training effects and establish a reliable baseline when MOT-ST is used as the performance outcome. Detraining periods up to 2weeks did not impact performance. The average of 3 discrete tests should be used when assessing MOT-ST performance.

Automatic Apple Detection and Counting with AD-YOLO and MR-SORT.

In the production management of agriculture, accurate fruit counting plays a vital role in the orchard yield estimation and appropriate production decisions. Although recent tracking-by-detection algorithms have emerged as a promising fruit-counting method, they still cannot completely avoid fruit occlusion and light variations in complex orchard environments, and it is difficult to realize automatic and accurate apple counting. In this paper, a video-based multiple-object tracking method, MR-SORT (Multiple Rematching SORT), is proposed based on the improved YOLOv8 and BoT-SORT. First, we propose the AD-YOLO model, which aims to reduce the number of incorrect detections during object tracking. In the YOLOv8s backbone network, an Omni-dimensional Dynamic Convolution (ODConv) module is used to extract local feature information and enhance the model's ability better; a Global Attention Mechanism (GAM) is introduced to improve the detection ability of a foreground object (apple) in the whole image; a Soft Spatial Pyramid Pooling Layer (SSPPL) is designed to reduce the feature information dispersion and increase the sensory field of the network. Then, the improved BoT-SORT algorithm is proposed by fusing the verification mechanism, SURF feature descriptors, and the Vector of Local Aggregate Descriptors (VLAD) algorithm, which can match apples more accurately in adjacent video frames and reduce the probability of ID switching in the tracking process. The results show that the mAP metrics of the proposed AD-YOLO model are 3.1% higher than those of the YOLOv8 model, reaching 96.4%. The improved tracking algorithm has 297 fewer ID switches, which is 35.6% less than the original algorithm. The multiple-object tracking accuracy of the improved algorithm reached 85.6%, and the average counting error was reduced to 0.07. The coefficient of determination R2 between the ground truth and the predicted value reached 0.98. The above metrics show that our method can give more accurate counting results for apples and even other types of fruit.

End-to-end multiple object tracking in high-resolution optical sensors of drones with transformer models

Drone aerial imaging has become increasingly important across numerous fields as drone optical sensor technology continues to advance. One critical challenge in this domain is achieving both accurate and efficient multi-object tracking. Traditional deep learning methods often separate object identification from tracking, leading to increased complexity and potential performance degradation. Conventional approaches rely heavily on manual feature engineering and intricate algorithms, which can further limit efficiency. To overcome these limitations, we propose a novel Transformer-based end-to-end multi-object tracking framework. This innovative method leverages self-attention mechanisms to capture complex inter-object relationships, seamlessly integrating object detection and tracking into a unified process. By utilizing end-to-end training, our approach simplifies the tracking pipeline, leading to significant performance improvements. A key innovation in our system is the introduction of a trajectory detection label matching technique. This technique assigns labels based on a comprehensive assessment of object appearance, spatial characteristics, and Gaussian features, ensuring more precise and logical label assignments. Additionally, we incorporate cross-frame self-attention mechanisms to extract long-term object properties, providing robust information for stable and consistent tracking. We further enhance tracking performance through a newly developed self-characteristics module, which extracts semantic features from trajectory information across both current and previous frames. This module ensures that the long-term interaction modules maintain semantic consistency, allowing for more accurate and continuous tracking over time. The refined data and stored trajectories are then used as input for subsequent frame processing, creating a feedback loop that sustains tracking accuracy. Extensive experiments conducted on the VisDrone and UAVDT datasets demonstrate the superior performance of our approach in drone-based multi-object tracking.

Application and integration of computer vision technologies for automated recognition and recording of chemical experiments

Abstract Advancements in computer vision have significantly improved motion and object recognition accuracy. These advancements should aid the automatic recognition of chemical experiments, potentially contributing to the recording of experiments. Creating an electronic laboratory notebook from experiment filming enhances convenience and allows more detailed information storage compared to traditional manual recording methods. Our previous research focused on employing object detection and action recognition to automate the recognition of chemical experiments. This paper presents a novel system that combines object detection, action recognition, multiple object tracking, and barcode recognition to automatically generate experimental flowcharts. We implemented our system as a graphical user interface-based application for laboratory use that successfully constructs flowcharts from videos of chemical experiments, including simple chemical manipulations.

Multi-object tracking using score-driven hierarchical association strategy between predicted tracklets and objects

Machine vision is one of the major technologies to guarantee intelligent robots’ human-centered embodied intelligence. Especially in the complex dynamic scene involving multi-person, Multi-Object Tracking (MOT), which can accurately identify and track specific targets, significantly influences intelligent robots’ performance regarding behavior perception and monitoring, autonomous decision-making, and providing personalized humanoid services. In order to solve the problem of targets lost and identity switches caused by the scale variations of objects and frequent overlaps during the tracking process, this paper presents a multi-object tracking method using score-driven hierarchical association strategy between predicted tracklets and objects (ScoreMOT). Firstly, a motion prediction of occluded objects based on bounding box variation (MPOBV) is proposed to estimate the position of occluded objects. MPOBV models the motion state of the object using the bounding box and confidence score. Then, a score-driven hierarchical association strategy between predicted tracklets and objects (SHAS) is proposed to correctly associate them in frequently overlapping scenarios. SHAS associates the predicted tracklets and detected objects with different confidence in different stages. The comparison results with 16 state-of-the-art methods on Multiple Object Tracking Benchmark 20 (MOT20) and DanceTrack datasets are conducted, and ScoreMOT outperforms the compared methods.

P2FTrack: Multi-object tracking with motion Prior and Feature Posterior

Multiple object tracking (MOT) has emerged as a crucial component of the rapidly developing computer vision. However, existing multi-object tracking methods often overlook the relationship between features and motion, hindering the ability to strike a performance balance between coupled motion and complex scenes. In this work, we propose a novel end-to-end multi-object tracking method that integrates motion and feature information. To achieve this, we introduce a motion prior generator that transforms motion information into attention masks. Additionally, we leverage prior-posterior fusion multi-head attention to combine the motion-derived priors and attention-based posteriors. Our proposed method is extensively evaluated on MOT17 and DanceTrack datasets through comprehensive experiments and ablation studies, demonstrating state-of-the-art performance in the feature-based method with reasonable speed.

Comparative analysis of personal protective equipment nonadherence detection: computer vision versus human observers.

Human monitoring of personal protective equipment (PPE) adherence among healthcare providers has several limitations, including the need for additional personnel during staff shortages and decreased vigilance during prolonged tasks. To address these challenges, we developed an automated computer vision system for monitoring PPE adherence in healthcare settings. We assessed the system performance against human observers detecting nonadherence in a video surveillance experiment. The automated system was trained to detect 15 classes of eyewear, masks, gloves, and gowns using an object detector and tracker. To assess how the system performs compared to human observers in detecting nonadherence, we designed a video surveillance experiment under 2 conditions: variations in video durations (20, 40, and 60 seconds) and the number of individuals in the videos (3 versus 6). Twelve nurses participated as human observers. Performance was assessed based on the number of detections of nonadherence. Human observers detected fewer instances of nonadherence than the system (parameter estimate -0.3, 95% CI -0.4 to -0.2, P < .001). Human observers detected more nonadherence during longer video durations (parameter estimate 0.7, 95% CI 0.4-1.0, P < .001). The system achieved a sensitivity of 0.86, specificity of 1, and Matthew's correlation coefficient of 0.82 for detecting PPE nonadherence. An automated system simultaneously tracks multiple objects and individuals. The system performance is also independent of observation duration, an improvement over human monitoring. The automated system presents a potential solution for scalable monitoring of hospital-wide infection control practices and improving PPE usage in healthcare settings.

Real-Time Detection and Localization of Weeds in Dictamnus dasycarpus Fields for Laser-Based Weeding Control

Traditional Chinese medicinal herbs have strict environmental requirements and are highly susceptible to weed damage, while conventional herbicides can adversely affect their quality. Laser weeding has emerged as an effective method for managing weeds in precious medicinal herbs. This technique allows for precise weed removal without chemical residue and protects the surrounding ecosystem. To maximize the effectiveness of this technology, accurate detection and localization of weeds in the medicinal herb fields are crucial. This paper studied seven species of weeds in the field of Dictamnus dasycarpus, a traditional Chinese medicinal herb. We propose a lightweight YOLO-Riny weed-detection algorithm and develop a YOLO-Riny-ByteTrack Multiple Object Tracking method by combining it with the ByteTrack algorithm. This approach enables accurate detection and localization of weeds in medicinal fields. The YOLO-Riny weed-detection algorithm is based on the YOLOv7-tiny network, which utilizes the FasterNet lightweight structure as the backbone, incorporates a lightweight upsampling operator, and adds structure reparameterization to the detection network for precise and rapid weed detection. The YOLO-Riny-ByteTrack Multiple Object Tracking method provides quick and accurate feedback on weed identification and location, reducing redundant weeding and saving on laser weeding costs. The experimental results indicate that (1) YOLO-Riny improves detection accuracy for Digitaria sanguinalis and Acalypha australis, ultimately amounting to 5.4% and 10%, respectively, compared to the original network. It also diminishes the model size by 2 MB and inference time by 10 ms, making it more suitable for resource-constrained edge devices. (2) YOLO-Riny-ByteTrack enhances Multiple Object Tracking accuracy by 3%, reduces ID switching by 14 times, and improves overall tracking accuracy by 3.4%. The proposed weed-detection and localization method for Dictamnus dasycarpus offers fast detection speed, high localization accuracy, and stable tracking, supporting the implementation of laser weeding during the seedling stage of Dictamnus dasycarpus.

OptiRet-Net: An Optimized Low-Light Image Enhancement Technique for CV-based Applications in Resource-Constrained Environments

The illumination of images can significantly impact computer-vision applications such as image classification, multiple object detection, and tracking, leading to a significant decline in detection and tracking accuracy. Recent advancements in deep learning techniques have been applied to low-light image enhancement (LLIE) to combat this issue. Retinex theory-based methods following a decomposition-adjustment pipeline for LLIE have performed well in various aspects. Despite their success, current research on Retinex-based deep learning still needs to improve in terms of optimization techniques and complicated convolution connections, which can be computationally intensive for end-device deployment. We propose an Optimized Retinex-based CNN (OptiRet-Net) deep-learning framework to address these challenges for the low-light image enhancement problem. Our results demonstrate that the proposed method outperforms existing state-of-the-art models in terms of full reference metrics with a PSNR of 21.87, SSIM of 0.80, LPIPS of 0.16, and zero reference metrics with a NIQE of 3.4 and PIQE of 56.6. Additionally, we validate our approach using a comprehensive evaluation comprising five datasets and nine prior methods. Furthermore, we assess the efficacy of our proposed model combining low-light multiple object tracking applications using YOLOX and ByteTrack in versatile video coding (VVC/H.266) across various quantization parameters. Our findings reveal that LLIE-enhanced frames surpass their tracking results with a MOTA of 80.6% and a remarkable precision rate of 96%. Our model also achieves minimal file sizes by effectively compressing the enhanced low-light images while maintaining their quality, making it suitable for resource-constrained environments where storage or bandwidth limitations are a concern.

Efficient Real-Time Droplet Tracking in Crop-Spraying Systems

Spray systems in agriculture serve essential roles in the precision application of pesticides, fertilizers, and water, contributing to effective pest control, nutrient management, and irrigation. These systems enhance efficiency, reduce labor, and promote environmentally friendly practices by minimizing chemical waste and runoff. The efficacy of a spray is largely determined by the characteristics of its droplets, including their size and velocity. These parameters are not only pivotal in assessing spray retention, i.e., how much of the spray adheres to crops versus becoming environmental runoff, but also in understanding spray drift dynamics. This study introduces a real-time deep learning-based approach for droplet detection and tracking which significantly improves the accuracy and efficiency of measuring these droplet properties. Our methodology leverages advanced AI techniques to overcome the limitations of previous tracking frameworks, employing three novel deep learning-based tracking methods. These methods are adept at handling challenges such as droplet occlusion and varying velocities, ensuring precise tracking in real-time potentially on mobile platforms. The use of a high-speed camera operating at 2000 frames per second coupled with innovative automatic annotation tools enables the creation of a large and accurately labeled droplet dataset for training and evaluation. The core of our framework lies in the ability to track droplets across frames, associating them temporally despite changes in appearance or occlusions. We utilize metrics including Multiple Object Tracking Accuracy (MOTA) and Multiple Object Tracking Precision (MOTP) to quantify the tracking algorithm’s performance. Our approach is set to pave the way for innovations in agricultural spraying systems, offering a more efficient, accurate, and environmentally responsible method of applying sprays and representing a significant step toward sustainable agricultural practices.

NTrack: A Multiple-Object Tracker and Dataset for Infield Cotton Boll Counting

NTrack: A Multiple-Object Tracker and Dataset for Infield Cotton Boll Counting

Intelligent beehive monitoring system based on internet of things and colony state analysis

Bees play a crucial role in terrestrial ecosystems. However, beekeepers are unable to monitor the state of beehives (bees and environment) all the time, which often results in bees escaping or even dying. Currently, some researchers provided the scheme of intelligent beehive monitoring system equipped with the Internet of Things (IoT), There remain two challenges: accurately monitor the environmental status around the hive and accurately track and monitor bees in real time. With the development of the IoT and computer vision algorithms, we hope to provide an automated and efficient system to meet the above challenges. In this paper, we proposed a hive monitoring system, and build a visualization module in the cloud to monitor the activity of bee colonies and the environmental dynamic changes. (1) We proposed a multi-bee tracking algorithm to solve the problem of monitoring bees at the door of the hive; (2) we constructed a dataset containing various complex scenes, named BEE22, for training and testing the performance of our algorithm; (3) we designed a bee counting rule, based on results of multi-bee tracking algorithm, to reasonably count the bees entering or leaving the beehive; (4) we have deployed multiple sensors around(center, margin, door, and environment) the hive to accurately reflect the changes in the environment around the hive. Experimental results demonstrate the effectiveness and excellence of our system. In particular, the tracking performance of the multi-bee tracking algorithm reaches 83.5 % ± 0.7 % Multiple Object Tracking Accuracy (MOTA) and 77.3 %±0.2 % Multiple Object Tracking Precision (MOTP), speeds up to 16 frames per second, compared with other algorithms, MOTA and Identity F1 Score (IDF1) are improved by 5.4 % and 8.2 % respectively. Moreover, our counting algorithm also achieved excellent results, with root mean square error (RMSE) of 1.3 ± 0.1, 0.2 ± 0.0, and 1.6 ± 0.1 in counting the number of bees current, entry, and out scene in an episode, respectively. After that, the system will be deployed and monitored for a long time in the actual scenario, it was found that the activity of bees decreased significantly under heavy rainfall conditions. Additionally, the activity of the bee colony will also increase accordingly, when the amplitude is 500 dB to 2000 dB, the temperature of the center of the beehive is 25 °C to 37 °C, and the humidity is 48 % to 67 %. In summary, our system can provide valuable information for bee farmers to make control decisions on hives.

Detecting thyroid nodules along with surrounding tissues and tracking nodules using motion prior in ultrasound videos

Ultrasound examination plays a crucial role in the clinical diagnosis of thyroid nodules. Although deep learning technology has been applied to thyroid nodule examinations, the existing methods all overlook the prior knowledge of nodules moving along a straight line in the video. We propose a new detection model, DiffusionVID-Line, and design a novel tracking algorithm, ByteTrack-Line, both of which fully leverage the prior knowledge of linear motion of nodules in thyroid ultrasound videos. Among them, ByteTrack-Line groups detected nodules, further reducing the workload of doctors and significantly improving their diagnostic speed and accuracy. In DiffusionVID-Line, we propose two new modules: Freq-FPN and Attn-Line. Freq-FPN module is used to extract frequency features, taking advantage of these features to reduce the impact of image blur in ultrasound videos. Based on the standard practice of segmented scanning by doctors, Attn-Line module enhances the attention on targets moving along a straight line, thus improving the accuracy of detection. In ByteTrack-Line, considering the characteristic of linear motion of nodules, we propose the Match-Line association module, which reduces the number of nodule ID switches. In the testing of the detection and tracking datasets, DiffusionVID-Line achieved a mean Average Precision (mAP50) of 74.2 for multiple tissues and 85.6 for nodules, while ByteTrack-Line achieved a Multiple Object Tracking Accuracy (MOTA) of 83.4. Both nodule detection and tracking have achieved state-of-the-art performance.

CMTT-JTracker: a fully test-time adaptive framework serving automated cell lineage construction.

Cell tracking is an essential function needed in automated cellular activity monitoring. In practice, processing methods striking a balance between computational efficiency and accuracy as well as demonstrating robust generalizability across diverse cell datasets are highly desired. This paper develops a central-metric fully test-time adaptive framework for cell tracking (CMTT-JTracker). Firstly, a CMTT mechanism is designed for the pre-segmentation of cell images, which enables extracting target information at different resolutions without additional training. Next, a multi-task learning network with the spatial attention scheme is developed to simultaneously realize detection and re-identification tasks based on features extracted by CMTT. Experimental results demonstrate that the CMTT-JTracker exhibits remarkable biological and tracking performance compared with benchmarking tracking methods. It achieves a multiple object tracking accuracy (MOTA) of $0.894$ on Fluo-N2DH-SIM+ and a MOTA of $0.850$ on PhC-C2DL-PSC. Experimental results further confirm that the CMTT applied solely as a segmentation unit outperforms the SOTA segmentation benchmarks on various datasets, particularly excelling in scenarios with dense cells. The Dice coefficients of the CMTT range from a high of $0.928$ to a low of $0.758$ across different datasets.

Multiple object tracking as a measure of sustained attention and relation with fluid reasoning

Multiple object tracking as a measure of sustained attention and relation with fluid reasoning

The nature and computation of attentional effort: A peak/end rule integrating over moment-by-moment effort during multiple-object tracking

The nature and computation of attentional effort: A peak/end rule integrating over moment-by-moment effort during multiple-object tracking

Sometimes more (overlap) is better! Action plan overlap impacts the interference between visually-guided touch and multiple-object tracking (MOT)

Sometimes more (overlap) is better! Action plan overlap impacts the interference between visually-guided touch and multiple-object tracking (MOT)

Examining the influence of cognitive processing orientation on tracking performance using a modified multiple object tracking task

Examining the influence of cognitive processing orientation on tracking performance using a modified multiple object tracking task

A lightweight convolutional neural network-based feature extractor for visible images

Feature extraction networks (FENs), as the first stage in many computer vision tasks, play critical roles. Previous studies regarding FENs employed deeper and wider networks to attain higher accuracy, but their approaches were memory-inefficient and computationally intensive. Here, we present an accurate and lightweight feature extractor (RoShuNet) for visible images based on ShuffleNetV2. The provided improvements are threefold. To make ShuffleNetV2 compact without degrading its feature extraction ability, we propose an aggregated dual group convolutional module; to better aid the channel interflow process, we propose a γ-weighted shuffling module; to further reduce the complexity and size of the model, we introduce slimming strategies. Classification experiments demonstrate the state-of-the-art (SOTA) performance of RoShuNet, which yields an increase in accuracy and reduces the complexity and size of the model compared to those of ShuffleNetV2. Generalization experiments verify that the proposed method is also applicable to feature extraction tasks in semantic segmentation and multiple-object tracking scenarios, achieving comparable accuracy to that of other approaches with more memory and greater computational efficiency. Our method provides a novel perspective for designing lightweight models.

Maritime ISAR detection and tracking algorithm for multiple maneuvering extended vessels in heavy-tailed clutter using SK-MM-Sub-RMM-MB-TBD filter

Multiple extended objects tracking (EOT) tasks play an important role in computer vision and engineering applications of artificial intelligence. In particular, nonlinear marine object imaging and tracking has received an increasing amount of attention due to its enormous application potential in the field of marine engineering, Autonomous Underwater Vehicles, and Remotely Operated Vehicles. Under high sea state, ship-EOTs perform complex maneuvering movements due to strong disturbances such as sea winds and sea waves. In this paper, we exploit emergent maneuvering EOTs (M-EOTs) methodologies in heavy-tailed clutter. We propose a M-EOT procedure in real-time scenario based on the popular multi-Bernoulli (MB)-TBD filter in maritime inverse synthetic aperture radar (ISAR) systems, and in particular, we describe the extended ship target state through the random matrices model (RMM). In RMM, scatter centers are distributed symmetrically around the M-EOT's centroid. However, in ship M-EOT scenario, the distribution over the whole object is not symmetrical, but distributed and skewed in some portions while a target maneuvers. To solve this problem, a novel robustness observation model is represented by using skewed (SK) non-symmetrically normal distribution and multiple model (MM) MB-TBD with more than one ellipse. Simulation and experimental results illustrate that the proposed SK-MM-Sub-RMM-MB-TBD filter outperforms the existing filters for M-EOTs.