Multiple Object Tracking Research Articles

A meta-analysis of performance advantages on athletes in multiple object tracking tasks

This study compared the multiple object tracking (MOT) performance of athletes vs. non-athletes and expert athletes vs. novice athletes by systematically reviewing and meta-analyzing the literature. A systematic literature search was conducted using five databases for articles published until July 2024. Healthy people were included, specifically classified as athletes and non-athletes, or experts and novices. Potential sources of heterogeneity were selected using a random-effects model. Moderator analyses were also performed. A total of 23 studies were included in this review. Regarding the overall effect, athletes were significantly better at MOT tasks than non-athletes, and experts performed better than novices. Subgroup analyses showed that expert athletes had a significantly larger effect than novices, and that the type of sport significantly moderated the difference in MOT performance between the two groups. Meta-regression revealed that the number of targets and duration of tracking moderated the differences in performance between experts and novices, but did not affect the differences between athletes and non-athletes. This meta-analysis provides evidence of performance advantages for athletes compared with nonathletes, and experts compared with novices in MOT tasks. Moreover, the two effects were moderated by different factors; therefore, future studies should classify participants more specifically according to sports levels.

YOLODCC: Improved YOLOv8 combined with dynamic confidence compensation for lightweight moving object detection

AbstractMost multiple object tracking algorithms depend on the output of the detector. Aiming at the problem that the higher detection quality model is restricted by the computing power, and the robustness of the lightweight detection model is easily affected by motion blur, this paper proposes a lightweight moving object detector based on improved YOLOv8 combined with dynamic confidence compensation algorithm. The algorithm combines various technical means such as network structure optimization, lightweight design, self‐knowledge distillation, loss function improvement and dynamic confidence compensation. ByteTrack is used as a tracker to conduct experiments on PASCAL VOC07+12 data set and UA‐DETRAC test sequence. Compared with the baseline YOLOv8n+ByteTrack, the proposed algorithm improves the HOTA by 1.3% when the single frame tracking delay is reduced by 1.1%. Mostly tracked target is improved by 79.7%, mostly lost target is reduced by 10.9%, and the detection effect is better than the original detector and other popular object detectors. The YOLODCC model achieves a balance between lightweight and multi‐object motion blur.

On-Line Detection Method of Salted Egg Yolks with Impurities Based on Improved YOLOv7 Combined with DeepSORT.

Salted duck egg yolk, a key ingredient in various specialty foods in China, frequently contains broken eggshell fragments embedded in the yolk due to high-speed shell-breaking processes, which pose significant food safety risks. This paper presents an online detection method, YOLOv7-SEY-DeepSORT (salted egg yolk, SEY), designed to integrate an enhanced YOLOv7 with DeepSORT for real-time and accurate identification of salted egg yolks with impurities on production lines. The proposed method utilizes YOLOv7 as the core network, incorporating multiple Coordinate Attention (CA) modules in its Neck section to enhance the extraction of subtle eggshell impurities. To address the impact of imbalanced sample proportions on detection accuracy, the Focal-EIoU loss function is employed, adaptively adjusting bounding box loss values to ensure precise localization of yolks with impurities in images. The backbone network is replaced with the lightweight MobileOne neural network to reduce model parameters and improve real-time detection performance. DeepSORT is used for matching and tracking yolk targets across frames, accommodating rotational variations. Experimental results demonstrate that YOLOv7-SEY-DeepSORT achieves a mean average precision (mAP) of 0.931, reflecting a 0.53% improvement over the original YOLOv7. The method also shows enhanced tracking performance, with Multiple Object Tracking Accuracy (MOTA) and Multiple Object Tracking Precision (MOTP) scores of 87.9% and 73.8%, respectively, representing increases of 17.0% and 9.8% over SORT and 2.9% and 4.7% over Tracktor. Overall, the proposed method balances high detection accuracy with real-time performance, surpassing other mainstream object detection methods in comprehensive performance. Thus, it provides a robust solution for the rapid and accurate detection of defective salted egg yolks and offers a technical foundation and reference for future research on the automated and safe processing of egg products.

How does colour predict multiple object tracking performance? The role of surface feature in attentive tracking

ABSTRACT Simultaneously tracking multiple unique dynamic objects, such as watching soccer games, is common in everyday life. However, the role of intragroup and intergroup features in this cognitive process remains unclear. We set up different colour combinations to investigate the effect of distinctiveness between target and distractor sets (DTD), the complexity of the target set (CT) and distractor set (CD) on multiple object tracking (MOT) on PCs and iPads, and used multiple linear regression to fit the relationship between these factors and tracking performance. The models support our hypothesis, showing that DTD contributes most to the interpretation of tracking performance, followed by CT, while CD has no effect. Furthermore, CT effect is modulated by DTD level. This study demonstrates that DTD and CT jointly contribute to predicting MOT performance regardless of devices. Thus, it would provide the theoretical basis for the future development of MOT-based cognitive applications on mobile devices.

A dual-instrument Kalman-based tracker to enhance robustness of microsurgical tools tracking.

The integration of a surgical robotic instrument tracking module within optical microscopes holds the potential to advance microsurgery practices, as it facilitates automated camera movements, thereby augmenting the surgeon's capability in executing surgical procedures. In the present work, an innovative detection backbone based on spatial attention module is implemented to enhance the detection accuracy of small objects within the image. Additionally, we have introduced a robust data association technique, capable to re-track surgical instrument, mainly based on the knowledge of the dual-instrument robotics system, Intersection over Union metric and Kalman filter. The effectiveness of this pipeline was evaluated through testing on a dataset comprising ten manually annotated videos of anastomosis procedures involving either animal or phantom vessels, exploiting the Symani®Surgical System-a dedicated robotic platform designed for microsurgery. The multiple object tracking precision (MOTP) and the multiple object tracking accuracy (MOTA) are used to evaluate the performance of the proposed approach, and a new metric is computed to demonstrate the efficacy in stabilizing the tracking result along the video frames. An average MOTP of 74±0.06% and a MOTA of 99±0.03% over the test videos were found. These results confirm the potential of the proposed approach in enhancing precision and reliability in microsurgical instrument tracking. Thus, the integration of attention mechanisms and a tailored data association module could be a solid base for automatizing the motion of optical microscopes.

Optimized Deep LSTM-Based Multi-Object Tracking with Occlusion Handling Mechanism

The multi-object tracking is a basic computer vision process having a huge class of real-life tools that range from monitoring of medical video to surveillance. The goal of tracking numerous items is to place numerous objects in a scene, handle their identities throughout time, and construct trajectories for analysis. However, this is a complex task, because of certain issues like occlusions, complicated object dynamics, and variations in the appearance of objects. In this research, a new technique named TPRO-based Deep LSTM is developed for tracking multi-object with occlusion handling. Here, the videos are considered as input wherein the extraction of frames is done from each video. Each frame undergoes pre-processing with filtering to eliminate noise from frames. By using a sparse Fuzzy c-Means (FCM) and Local Optimal-Oriented Pattern (LOOP) features, the localization of objects is done. Moreover, the visual and spatial trackings are considered for hybrid tracking. The second derivative model and neighborhood search model are used to perform visual tracking. Then the occlusion handling is performed. Concurrently, with the use of Deep Long Short-Term Memory (Deep LSTM) the spatial tracking is performed and the Taylor Poor Rich Optimization (TPRO) algorithm assigns the weight and bias of the Deep LSTM. The TPRO is obtained by the unification of the Taylor series along with the Poor and Rich Optimization algorithm. By combining visual and spatial tracking, the final tracked output is generated. The devised method achieves a performance with the highest value of 88.9% for Multiple Object Tracking Precision (MOTP), smallest tracking distance (TD) of 4.185, average MOTP of 0.889, average TD of 4.201, and highest tracking number (TN) of 14.

Puzzle: taking livestock tracking to the next level

Animal behavior is a critical aspect for a better understanding and management of animal health and welfare. The combination of cameras with artificial intelligence holds significant potential, particularly as it eliminates the need to handle animals and allows for the simultaneous measurement of various traits, including activity, space utilization, and inter-individual distance. The primary challenge in using these techniques is dealing with the individualization of data, known as the multiple object tracking problem in computer science. In this article, we propose an original solution called “Puzzle.” Similar to solving a puzzle, where you start with the border pieces that are easy to position, our approach involves commencing with video sequences where tracking is straightforward. This initial phase aims to train a Convolutional Neural Network (CNN) capable of deriving the appearance clues of each animal. The CNN is then used on the entire video, together with distance-based metrics, in order to associate detections and animal id. We illustrated our method in the context of outdoor goat tracking, achieving a high percentage of good tracking, exceeding 90%. We discussed the impact of different criteria used for animal ID association, considering whether they are based solely on location, appearance, or a combination of both. Our findings indicate that, by adopting the puzzle paradigm and tailoring the appearance CNN to the specific video, relying solely on appearance can yield satisfactory results. Finally, we explored the influence of tracking efficacy on two behavioral studies, estimating space utilization and activity. The results demonstrated that the estimation error remained below 10%. The code is entirely open-source and extensively documented. Additionally, it is linked to a data-paper to facilitate the training of any automatic detection algorithm for goats, with the goal of fostering open access within the deep-learning livestock community.

Validity of a motor-cognitive dual-task agility test in elite youth football players.

Agility, as the ability to react rapidly to unforeseen events, is an essential component of football performance. However, existing agility diagnostics often do not reflect the complex motor-cognitive interaction required on the field. Therefore, this study evaluates the criterion and ecological validity of a newly developed motor-cognitive dual-task agility approach in elite youth football players and compare it to a traditional reactive agility test. Twenty-one male youth elite football players (age:17.4±0.6; BMI:23.2±1.8) performed two agility tests (reactive agility, reactive agility with integrated multiple-object-tracking (Dual-Task Agility)) on the SKILLCOURT system. Performance was correlated to motor (sprint, jump), cognitive (executive functions, attention, reaction speed) and football specific tests (Loughborough soccer passing test (LSPT)) as well as indirect game metrics (coaches' rating, playing time). Reactive agility performance showed moderate correlations to attention and choice reaction times (r=0.48-0.63), as well as to the LSPT (r=0.51). The dual-task agility test revealed moderate relationships with attention and reaction speed (r=0.47-0.58), executive functions (r=0.45-0.63), as well as the game metrics (r=0.51-0.61). Finally, the dual-task agility test significantly differentiated players based on their coaches' rating and playing time using a median split (p<0.05; d=0.8-1.28). Motor-cognitive agility performance in elite youth football players seems to be primarily determined by cognitive functions. The integration of multiple object tracking into reactive agility testing seems to be an ecologically valid approach for performance diagnostics in youth football.

Multiple object tracking training affects the executive function in basketball players: the role of instant feedback

BackgroundThe present study aims to investigate the potential impact of eight sessions of Multiple Object Tracking (MOT) training on the executive function in basketball players. The purpose of the study was primarily to observe the effects of MOT training with and without feedback on the executive function of basketball players.MethodsA sample of fifty-eight participants was selected from college students enrolled in a university basketball special selection class. The participants were divided into three equal groups. The first group received MOT training with instant feedback and was called feedback group, the second group received MOT training without instant feedback and was called no feedback group, and the third group did not receive any intervention and was called control group.ResultsAfter eight sessions of MOT training, feedback group demonstrated the best performance in the Go/No-go task and the 3-back task. After eight sessions of MOT training, there was no significant difference in test scores on the Stroop task between the feedback and no feedback groups. There was also no significant difference in test scores between the feedback and no feedback groups on the 2-back task after eight sessions of MOT training. The findings of this study suggest that MOT training can effectively enhance the executive function of basketball players.ConclusionsMOT training was found to enhance the executive function of basketball players, irrespective of whether they received instant feedback. However, the feedback group exhibited superior improvements in the Go/No-go task and the 3-back task.

A robust method for multi object tracking in autonomous ship navigation systems

A novel Maritime Multi-Object Tracking method is proposed, combining a deep learning-based object detector with target association algorithms to achieve robust sea-surface multi-object tracking. Specifically, the Multi-Object Tracking employs You Only Look Once version 7 detector for object detection. In the data association part, a module for onboard camera motion compensation is developed, a maritime dynamic spatial information-based intersection-over-union is presented as a similarity metric, and a progressive refinement cascade matching strategy is designed to enhance the tracker's sea-surface multi-object tracking capabilities. The Jari Maritime Tracking Dataset is utilised to validate the effectiveness performance of the proposed method. Experimental results demonstrate that compared to the earlier process, the proposed method exhibits a significant enhancement in multiple object tracking accuracy, with an increase of 27.8% and an achieved score of 81.3. In particular, it reduces the number of identifications switching and missed targets, achieving holistically preferable performance. Meanwhile, the speed of the proposed Multi-Object Tracking fulfils the engineering application requirements for an autonomous ship navigation system.

Multiple Object Tracking in Drone Aerial Videos by a Holistic Transformer and Multiple Feature Trajectory Matching Pattern

Drone aerial videos have immense potential in surveillance, rescue, agriculture, and urban planning. However, accurately tracking multiple objects in drone aerial videos faces challenges like occlusion, scale variations, and rapid motion. Current joint detection and tracking methods often compromise accuracy. We propose a drone multiple object tracking algorithm based on a holistic transformer and multiple feature trajectory matching pattern to overcome these challenges. The holistic transformer captures local and global interaction information, providing precise detection and appearance features for tracking. The tracker includes three components: preprocessing, trajectory prediction, and matching. Preprocessing categorizes detection boxes based on scores, with each category adopting specific matching rules. Trajectory prediction employs the visual Gaussian mixture probability hypothesis density method to integrate visual detection results to forecast object motion accurately. The multiple feature pattern introduces Gaussian, Appearance, and Optimal subpattern assignment distances for different detection box types (GAO trajectory matching pattern) in the data association process, enhancing tracking robustness. We perform comparative validations on the vision-meets-drone (VisDrone) and the unmanned aerial vehicle benchmarks; the object detection and tracking (UAVDT) datasets affirm the algorithm’s effectiveness: it obtained 38.8% and 61.7% MOTA, respectively. Its potential for seamless integration into practical engineering applications offers enhanced situational awareness and operational efficiency in drone-based missions.

Behavior recognition of non-motorized transport at intersections using dual-channel grid model based on disordered trajectory point data

Accurately identifying the passing and waiting behavior of pedestrians and non-motorized vehicles at intersection is essential to planning management measures for non-motorized transport. Much previous study in this field has focused on methods based on continuous individual trajectory detection, which are almost ineffective under the poor detection condition in dense traffic scenarios. To address the task, this paper establish a workable framework for inferring the probabilities of passing or waiting behavior of pedestrians and non-motorized vehicles in arbitrary space using disordered trajectory point data. First, a two-channel model is proposed to perform a formalized grid-level representation of an intersection, in which the functional attributes and occupancy characteristics of grids are comprehensively defined and quantified. Then, the quantitative characteristics of the grids are used to detect the real-world occurrence space of passing and waiting behaviors, by the clustering and expanding operations on grids. Finally, through feature transfer along path, characteristics is decomposed into passing and waiting occurrence characteristics for behavior probability computation. Results indicate that the method achieves over 91% accuracy of behavior recognition, which is better than compared methods in various Multiple Object Tracking Accuracy (MOTA). Although the method is sensitive to spatial detection conditions, it obtains steady accuracy under various target detection settings.

Multi-level temporal feature fusion with feature exchange strategy for multiple object tracking

Multi-level temporal feature fusion with feature exchange strategy for multiple object tracking

Monitoring the lactation-related behaviors of sows and their piglets in farrowing crates using deep learning

Pig farming is a major sector of livestock production. The preweaning stage is a critical period in the pig farming process, where lactation-related behaviors between sows and their piglets directly influence the preweaning survivability of the piglets. Lactation-related behaviors are mutual interactions that require the combined monitoring of both the sow and her piglets. Conventional naked-eye observation is discontinuous and labor-intensive and may result in undetected abnormal behavior and economic losses. Thus, this study proposed to monitor the lactation-related behaviors of sows and their piglets simultaneously and continuously using computer vision. Videos were recorded from farrowing crates using embedded systems equipped with regular RGB cameras. The sow posture recognition model (SPRM), comprising a convolutional neural network (CNN) of the architecture EfficientNet and a long short-term memory network, was trained to identify seven postures of sows. The piglet localization and tracking model (PLTM), comprising a CNN of the architecture YOLOv7 and a simple online and realtime tracking algorithm, was trained to localize and track piglets in the farrowing crate. The sow posture information was then combined with the piglet activity to detect unfed piglets. The trained SPRM and PLTM reached an accuracy of 91.36% and a multiple object tracking accuracy of 94.6%. The performance of the proposed unfed piglet detection achieved a precision of 98.4% and a recall of 90.7%. A long-term experiment was conducted to monitor lactation-related behaviors of sows and their piglets from the birth of the piglets to day 15. The overall mean daily percentages ± standard deviations (SDs) of sow postures were 6.8% ± 2.9% for feeding, 8.8% ± 6.6% for standing, 11.8% ± 4.5% for sitting, 20.6% ± 16.3% for recumbency, 14.1% ± 6.5% for lying, and 38.1% ± 7.5% for lactating. The overall mean daily percentages ± SDs of piglet activities were 38.1% ± 7.5% for suckling, 22.2% ± 5.4% for active, and 39.7% ± 10.5% for rest. The proposed approach provides a total solution for the automatic monitoring of sows and their piglets in the farrowing house. This automatic detection of abnormal lactation-related behaviors can help in preventing piglet preweaning mortality and therefore aid pig farming efficiency.

A New Multimodal Map Building Method Using Multiple Object Tracking and Gaussian Process Regression

Recent advancements in simultaneous localization and mapping (SLAM) have significantly improved the handling of dynamic objects. Traditionally, SLAM systems mitigate the impact of dynamic objects by extracting, matching, and tracking features. However, in real-world scenarios, dynamic object information critically influences decision-making processes in autonomous navigation. To address this, we present a novel approach for incorporating dynamic object information into map representations, providing valuable insights for understanding movement context and estimating collision risks. Our method leverages on-site mobile robots and multiple object tracking (MOT) to gather activation levels. We propose a multimodal map framework that integrates occupancy maps obtained through SLAM with Gaussian process (GP) modeling to quantify the activation levels of dynamic objects. The Gaussian process method utilizes a map-based grid cell algorithm that distinguishes regions with varying activation levels while providing confidence measures. To validate the practical effectiveness of our approach, we also propose a method to calculate additional costs from the generated maps for global path planning. This results in path generation through less congested areas, enabling more informative navigation compared to traditional methods. Our approach is validated using a diverse dataset collected from crowded environments such as a library and public square and is demonstrated to be intuitive and to accurately provide activation levels.

Deep Efficient Data Association for Multi-Object Tracking: Augmented with SSIM-Based Ambiguity Elimination.

Recently, to address the multiple object tracking (MOT) problem, we harnessed the power of deep learning-based methods. The tracking-by-detection approach to multiple object tracking (MOT) involves two primary steps: object detection and data association. In the first step, objects of interest are detected in each frame of a video. The second step establishes the correspondence between these detected objects across different frames to track their trajectories. This paper proposes an efficient and unified data association method that utilizes a deep feature association network (deepFAN) to learn the associations. Additionally, the Structural Similarity Index Metric (SSIM) is employed to address uncertainties in the data association, complementing the deep feature association network. These combined association computations effectively link the current detections with the previous tracks, enhancing the overall tracking performance. To evaluate the efficiency of the proposed MOT framework, we conducted a comprehensive analysis of the popular MOT datasets, such as the MOT challenge and UA-DETRAC. The results showed that our technique performed substantially better than the current state-of-the-art methods in terms of standard MOT metrics.

Relationship between visual ability assessment and punch performance in competition in male amateur boxers.

Sport visual ability plays an important role in the performance of elite athletes in competition. However, its relationship with boxers' performance has not been fully understood. This study investigated the relationship between sports visual abilities and in-competition punching performance in 26 highly trained male amateur boxers. Ten visual abilities of the boxers were tested using the Senaptec Sensory Station (Senaptec, Beaverton, OR, United States), including Visual Clarity (VC), Contrast Sensitivity (CS), Depth Perception (DP), Near Far Quickness (NFQ), Target Capture (TC), Perception Span (PS), Multiple Object Tracking (MOT), Eye-Hand Coordination (EHC), Go/No Go (GNG), and Reaction Time (RT). Performance analyses were conducted on national boxing championships conducted by the boxers to analyze their punching accuracy. Correlation and regression analyses showed that punch performance %Hit was very strong correlated with DP, EHC, GNG, and RT, and showed a strong correlation with VC, CS, and PS. %Hit was moderate correlated with MOT, while there was no correlation with NFQ and TC. In addition, RT, EHC, and DP are important visual ability variables for boxers. The results of this study indicate that there is an overall relationship between sports visual ability and boxing performance, but there is also a specific relationship between variables in visual ability and boxing performance, especially faster reaction times, better processing of visual information, and decision-making abilities, and the ability to accurately recognize the distance and position of an opponent and their punches are essential for enhancing boxing performance. Further studies are needed to investigate the relationship between sport visual ability and more comprehensive performance in boxers, and the possibility of enhancing performance through specific visual training.

Image hashing-based shallow object tracker for construction worker monitoring in scaffolding scenes

Multiple Object Tracking (MOT) has potential applications in construction site safety man- agement, particularly for the individualized assessment of scaffold workers' safety status. However, its accuracy can be degraded due to inconsistent detection results and insuffi- cient object association capabilities. This paper addresses these challenges by proposing an object tracking method, named as the Shallow Cascaded Buffered Intersection over Union (Shallow C-BIoU). This non-machine learning-based tracker employs a color hash- ing technique to enhance tracking performance. Employing rigorous evaluation metrics, the experimental results underscore the efficacy of the proposed method. Compared to state-of- the-art algorithms, the Shallow C-BIoU method improved association performance by 7.56%, reducing 52.44% of falsely assigned tracking IDs. Consequently, this paper contributes to the development of reliable object trackers, thereby advancing monitoring technologies for construction management purposes.

Learnable Graph Matching: A Practical Paradigm for Data Association.

Data association is at the core of many computer vision tasks, e.g., multiple object tracking, image matching, and point cloud registration. however, current data association solutions have some defects: they mostly ignore the intra-view context information; besides, they either train deep association models in an end-to-end way and hardly utilize the advantage of optimization-based assignment methods, or only use an off-the-shelf neural network to extract features. In this paper, we propose a general learnable graph matching method to address these issues. Especially, we model the intra-view relationships as an undirected graph. Then data association turns into a general graph matching problem between graphs. Furthermore, to make optimization end-to-end differentiable, we relax the original graph matching problem into continuous quadratic programming and then incorporate training into a deep graph neural network with KKT conditions and implicit function theorem. In MOT task, our method achieves state-of-the-art performance on several MOT datasets. For image matching, our method outperforms state-of-the-art methods on a popular indoor dataset, ScanNet. For point cloud registration, we also achieve competitive results.

Dynamic center point learning for multiple object tracking under Severe occlusions

Multiple Object Tracking (MOT) methods based on per-pixel prediction and association have achieved remarkable progress recently. These approaches prefer to select points in central region of the bounding box as positive samples and other points as negative samples. Under severe occlusions, these sample allocation methods might lead to the central region being contaminated by occluding samples, resulting in significant degradation of both detection and association performance. To address this issue, we propose a novel Dynamic-Center-Point-based Multiple Object Tracking method (DCP-MOT), which aims to self-identify the visible center region for occluded objects. Compared with the previous bounding box center point, our Dynamic-Center-Point (DCP) can better represent the visible region of the occluded object. Specifically, we design an Iterative Refinement Branch to generate dynamic center points for each occluded object. It includes two parts: Center Probability Predictor and Center Generator. Initially, we utilize the Center Probability Predictor to derive an accurate probability map for the occluded object. Subsequently, we use Center Generator to quantify the probability map by introducing a Mutual Exclusive Potential Function, yielding a dynamic center point for the occluded object that is distinct from its occluding counterpart. Finally, by joining the bounding box center points for unoccluded objects and our dynamic center points for occluded objects, our JDE branch can achieve better tracking performance. Extensive experiments demonstrate our DCP-based MOT method surpasses the bounding box center based SOTA in metrics MOTA (+0.7,+2.1,+1.3) and IDF1 (+0.7,+0.9,+1.5) on the challenging MOT16, MOT17, MOT20 datasets.