Multiple Object Tracking Method Research Articles

Real-Time Detection and Localization of Weeds in Dictamnus dasycarpus Fields for Laser-Based Weeding Control

Traditional Chinese medicinal herbs have strict environmental requirements and are highly susceptible to weed damage, while conventional herbicides can adversely affect their quality. Laser weeding has emerged as an effective method for managing weeds in precious medicinal herbs. This technique allows for precise weed removal without chemical residue and protects the surrounding ecosystem. To maximize the effectiveness of this technology, accurate detection and localization of weeds in the medicinal herb fields are crucial. This paper studied seven species of weeds in the field of Dictamnus dasycarpus, a traditional Chinese medicinal herb. We propose a lightweight YOLO-Riny weed-detection algorithm and develop a YOLO-Riny-ByteTrack Multiple Object Tracking method by combining it with the ByteTrack algorithm. This approach enables accurate detection and localization of weeds in medicinal fields. The YOLO-Riny weed-detection algorithm is based on the YOLOv7-tiny network, which utilizes the FasterNet lightweight structure as the backbone, incorporates a lightweight upsampling operator, and adds structure reparameterization to the detection network for precise and rapid weed detection. The YOLO-Riny-ByteTrack Multiple Object Tracking method provides quick and accurate feedback on weed identification and location, reducing redundant weeding and saving on laser weeding costs. The experimental results indicate that (1) YOLO-Riny improves detection accuracy for Digitaria sanguinalis and Acalypha australis, ultimately amounting to 5.4% and 10%, respectively, compared to the original network. It also diminishes the model size by 2 MB and inference time by 10 ms, making it more suitable for resource-constrained edge devices. (2) YOLO-Riny-ByteTrack enhances Multiple Object Tracking accuracy by 3%, reduces ID switching by 14 times, and improves overall tracking accuracy by 3.4%. The proposed weed-detection and localization method for Dictamnus dasycarpus offers fast detection speed, high localization accuracy, and stable tracking, supporting the implementation of laser weeding during the seedling stage of Dictamnus dasycarpus.

Dynamic center point learning for multiple object tracking under Severe occlusions

Multiple Object Tracking (MOT) methods based on per-pixel prediction and association have achieved remarkable progress recently. These approaches prefer to select points in central region of the bounding box as positive samples and other points as negative samples. Under severe occlusions, these sample allocation methods might lead to the central region being contaminated by occluding samples, resulting in significant degradation of both detection and association performance. To address this issue, we propose a novel Dynamic-Center-Point-based Multiple Object Tracking method (DCP-MOT), which aims to self-identify the visible center region for occluded objects. Compared with the previous bounding box center point, our Dynamic-Center-Point (DCP) can better represent the visible region of the occluded object. Specifically, we design an Iterative Refinement Branch to generate dynamic center points for each occluded object. It includes two parts: Center Probability Predictor and Center Generator. Initially, we utilize the Center Probability Predictor to derive an accurate probability map for the occluded object. Subsequently, we use Center Generator to quantify the probability map by introducing a Mutual Exclusive Potential Function, yielding a dynamic center point for the occluded object that is distinct from its occluding counterpart. Finally, by joining the bounding box center points for unoccluded objects and our dynamic center points for occluded objects, our JDE branch can achieve better tracking performance. Extensive experiments demonstrate our DCP-based MOT method surpasses the bounding box center based SOTA in metrics MOTA (+0.7,+2.1,+1.3) and IDF1 (+0.7,+0.9,+1.5) on the challenging MOT16, MOT17, MOT20 datasets.

BoostTrack: boosting the similarity measure and detection confidence for improved multiple object tracking

Handling unreliable detections and avoiding identity switches are crucial for the success of multiple object tracking (MOT). Ideally, MOT algorithm should use true positive detections only, work in real-time and produce no identity switches. To approach the described ideal solution, we present the BoostTrack, a simple yet effective tracing-by-detection MOT method that utilizes several lightweight plug and play additions to improve MOT performance. We design a detection-tracklet confidence score and use it to scale the similarity measure and implicitly favour high detection confidence and high tracklet confidence pairs in one-stage association. To reduce the ambiguity arising from using intersection over union (IoU), we propose a novel Mahalanobis distance and shape similarity additions to boost the overall similarity measure. To utilize low-detection score bounding boxes in one-stage association, we propose to boost the confidence scores of two groups of detections: the detections we assume to correspond to the existing tracked object, and the detections we assume to correspond to a previously undetected object. The proposed additions are orthogonal to the existing approaches, and we combine them with interpolation and camera motion compensation to achieve results comparable to the standard benchmark solutions while retaining real-time execution speed. When combined with appearance similarity, our method outperforms all standard benchmark solutions on MOT17 and MOT20 datasets. It ranks first among online methods in HOTA metric in the MOT Challenge on MOT17 and MOT20 test sets. We make our code available at https://github.com/vukasin-stanojevic/BoostTrack.

Detection-Free Object Tracking for Multiple Occluded Targets in Plenoptic Video

Multiple object tracking (MOT) is a fundamental task in vision, but MOT techniques for plenoptic video are scarce. Almost all 2D MOT algorithms that show high performance mostly use the detection-based method which has the disadvantage of operating only for a specific object. To enable tracking of arbitrary desired objects, this paper introduces a groundbreaking detection-free tracking method for MOT in plenoptic videos. The proposed method deviates from traditional detection-based tracking methods, emphasizing the challenges of tracking targets with occlusions. The paper presents specialized algorithms that exploit the multifocal information of plenoptic video, including the focal range restriction and dynamic focal range adjustment schemes to secure robustness for occluded object tracking. To the improvement of the spatial searching capability, the anchor ensemble and the dynamic change of spatial search region algorithms are also proposed. Additionally, in terms of MOT, to reduce the computation time involved, the motion-adaptive time scheduling technique is proposed, which improves computation speed while guaranteeing a certain level of accuracy. Experimental results show a significant improvement in tracking performance, with a 77% success rate based on intersection over union for occluded targets in plenoptic videos, marking a substantial advancement in the field of plenoptic object tracking.

SCGTracker: Spatio-temporal correlation and graph neural networks for multiple object tracking

Multiple Object Tracking is an important vitally important fundamental task in computer vision. Visual tracking becomes challenging when objects move in groups and are obscured from each other. There are two mainstream solution strategies for these group models. One is to transform the data association problem into a graph matching problem for solving, while the other is to apply the social power model as an advanced constraint for group tracking. In the former case, the solving difficulty geometric growth as the number of tracked objects increases, and the computing efficiency for real-time tracking demand cannot be met. The latter strategy tends to set up fixed-size groups or offline training rules, resulting in a lack of flexibility that limits their scenario generalization. According to the shortcomings of existing methods, this paper proposes a novel multiple object tracking method with spatio-temporal correlation and graph neural networks. Firstly, the relational features of the historical trajectories are extracted through the spatio-temporal relationship learning module, which models the spatio-temporal correlations of the objects and dynamically constructs the group structure online. Then, the graph neural network is combined with appearance and motion information, and the similarity between each detection and tracklet is used as a weight in node feature aggregation to make powerful distinctions between node features. Meanwhile, the spatio-temporal correlation method is also used to solve target loss issues caused by occlusion. Even collocated with linearly assigned data association method, good tracking results are still achieved, with a low computational complexity. Experiments on three challenging public datasets, namely MOT16, MOT17, and MOT20, validated the accuracy and efficiency of the proposed tracking method.

BEVEFNet: A Multiple Object Tracking Model Based on LiDAR-Camera Fusion

As a crucial task in the field of computer vision, object tracking models are widely used in various application domains, such as autonomous driving. However, existing multiple object tracking methods still face challenges in accurately and efficiently tracking moving multi-targets in real time. This paper presents BEVEFNet, a camera-LiDAR multi-target tracking model based on multistage fusion, which effectively utilizes the semantic information from optical images and the spatial and geometric information from LiDAR data to unify multi-modal features in a shared Bird’s Eye View(BEV) representation space. By leveraging LiDAR data to complement optical images, multi-level fusion is achieved at both the feature and decision levels. The proposed efficient sparse 3D feature extraction network significantly enhances the speed of multiple object tracking by incorporating sparse convolution. Experiments conducted on the nuSences dataset demonstrate that BEVEFNet achieves an AMOTA of 69.7, improving the accuracy of multiple object tracking.

Transformer-Based Multiple-Object Tracking via Anchor-Based-Query and Template Matching.

Multiple object tracking (MOT) plays an important role in intelligent video-processing tasks, which aims to detect and track all moving objects in a scene. Joint-detection-and-tracking (JDT) methods are thriving in MOT tasks, because they accomplish the detection and data association in a single stage. However, the slow training convergence and insufficient data association limit the performance of JDT methods. In this paper, the anchor-based query (ABQ) is proposed to improve the design of the JDT methods for faster training convergence. By augmenting the coordinates of the anchor boxes into the learnable queries of the decoder, the ABQ introduces explicit prior spatial knowledge into the queries to focus the query-to-feature learning of the JDT methods on the local region, which leads to faster training speed and better performance. Moreover, a new template matching (TM) module is designed for the JDT methods, which enables the JDT methods to associate the detection results and trajectories with historical features. Finally, a new transformer-based MOT method, ABQ-Track, is proposed. Extensive experiments verify the effectiveness of the two modules, and the ABQ-Track surpasses the performance of the baseline JDT methods, TransTrack. Specifically, the ABQ-Track only needs to train for 50 epochs to achieve convergence, while that for TransTrack is 150 epochs.

Segmentation and tracking of vegetable plants by exploiting vegetable shape feature for precision spray of agricultural robots

AbstractFor robotic precision spray application in vegetable farms, simultaneous accurate instance segmentation and robust tracking of plants are of great importance and a prerequisite for the following spray action. With onboard cameras, agricultural robots can apply Multiple Object Tracking and Segmentation (MOTS) methods, for instance, segmentation and tracking of plants. By assigning a unique identification for each vegetable, it ensures the robot to spray each vegetable exactly once, while traversing along the farm rows. Conventional MOTS methods, which are mostly designed for tracking pedestrians or vehicles, usually extract their color and texture features for associating different targets in consecutive images. However, vegetable plants of the same species normally show similar color and texture, which leads to degraded performance when conventional MOTS methods are used. To solve the challenging problem of associating vegetables with similar color and texture in consecutive images, in this paper, a novel MOTS method that exploits contour and blob features is proposed, for instance, segmentation and tracking of multiple vegetable plants. The method takes advantage of the fact that different plants normally possess different shape contours and blob properties. With images captured on top of them, these features of the same plant show little difference in consecutively captured images. Comprehensive experiments including ablation studies are conducted, which prove its superior performance over two state‐of‐the‐art MOTS methods. Compared with the conventional MOTS methods, the proposed method is able to re‐identify objects which have gone out of the camera field of view and re‐appear again using the proposed data association strategy, which is important to ensure each vegetable be sprayed only once when the robot travels back and forth. Although the method is tested on lettuce farm, it can be applied to other similar vegetables, such as broccoli and canola. Both the code and the dataset of this paper are publicly released for the benefit of the community: https://github.com/NanH5837/LettuceMOTS.

Evaluating and modeling the effects of brightness on visual attention using multiple object tracking method

This article systematically evaluates and models how brightness affects performance in Multiple Object Tracking (MOT) within screen-based environments. MOT proficiency is essential across various fields, and comprehending the elements that impact MOT performance holds significance for occupational health and safety. While previous studies have scrutinized the influence of brightness on object recognition, its repercussions on MOT performance in screen-based environments remain comparatively less comprehended. This research aims to bridge this gap by delving into the distinct and combined impacts of brightness-related factors on MOT performance. Additionally, it seeks to construct a computational model that can forecast MOT performance across diverse brightness conditions. The outcomes of this study will offer valuable insights into core psychological processes, thereby steering the development of more efficient visual displays to enhance occupational health and safety. Our findings revealed a significant correlation between brightness levels and MOT performance, with optimal tracking observed at medium brightness levels. Additionally, complex object motion patterns were found to exacerbate the challenges of tracking in low brightness settings. These insights have direct implications for screen-based interfaces, suggesting the need for adaptive brightness settings based on the content's complexity and the user's task.

QDTrack: Quasi-Dense Similarity Learning for Appearance-Only Multiple Object Tracking.

Similarity learning has been recognized as a crucial step for object tracking. However, existing multiple object tracking methods only use sparse ground truth matching as the training objective, while ignoring the majority of the informative regions in images. In this paper, we present Quasi-Dense Similarity Learning, which densely samples hundreds of object regions on a pair of images for contrastive learning. We combine this similarity learning with multiple existing object detectors to build Quasi-Dense Tracking (QDTrack), which does not require displacement regression or motion priors. We find that the resulting distinctive feature space admits a simple nearest neighbor search at inference time for object association. In addition, we show that our similarity learning scheme is not limited to video data, but can learn effective instance similarity even from static input, enabling a competitive tracking performance without training on videos or using tracking supervision. We conduct extensive experiments on a wide variety of popular MOT benchmarks. We find that, despite its simplicity, QDTrack rivals the performance of state-of-the-art tracking methods on all benchmarks and sets a new state-of-the-art on the large-scale BDD100K MOT benchmark, while introducing negligible computational overhead to the detector.

Comparing Multiple Extended Object Tracking with Point Based Multi Object Tracking for LiDAR in a Maritime Context

In this paper, we compare a recently developed multiple extended object tracking method with a point-based tracker for maritime applications by evaluating the methods on both real and simulated LiDAR data. Being able to track other vessels is a key part of maritime situational awareness, and multi-object tracking is well suited to this task. Traditionally, multi-object tracking uses the point approximation to simplify the target tracking problem, meaning that an object is assumed to only generate a single measurement. With recent advances in sensor technology, this approximation is no longer valid. Multiple extended object tracking instead makes use of methods that estimates the extent of a target using all available measurements. However, the target models used in extended object tracking are by necessity more complex. We compare these two approaches on real and simulated LiDAR data, which was gathered by tracking smaller vessels. We find that the extended object method has greater performance on simulated data and experimental data with comparatively little wake clutter. However, on experimental data with more wake clutter, the point-based method outperforms the extended object method since it is less sensitive to the disturbance caused by wake clutter. We can conclude that multiple extended object tracking methods have the potential to be more accurate for LiDAR data, but since the models are more complex, measurement sources need to be modeled correctly.

Novel multiple object tracking method for yellow feather broilers in a flat breeding chamber based on improved YOLOv3 and deep SORT

Novel multiple object tracking method for yellow feather broilers in a flat breeding chamber based on improved YOLOv3 and deep SORT

RelationTrack: Relation-Aware Multiple Object Tracking With Decoupled Representation

Existing online multiple object tracking (MOT) algorithms often consist of two subtasks, detection and re-identification (ReID). In order to enhance the inference speed and reduce the complexity, current methods commonly integrate these double subtasks into a unified framework. Nevertheless, detection and ReID demand diverse features. This issue results in an optimization contradiction during the training procedure. With the target of alleviating this contradiction, we devise a module named Global Context Disentangling (GCD) that decouples the learned representation into detection-specific and ReID-specific embeddings. As such, this module provides an implicit manner to balance the different requirements of these two subtasks. Moreover, we observe that preceding MOT methods typically leverage local information to associate the detected targets and neglect to consider the global semantic relation. To resolve this limitation, we develop a module, referred to as Guided Transformer Encoder (GTE), by combining the powerful reasoning ability of Transformer encoder and deformable attention. Unlike previous works, GTE avoids analyzing all the pixels and only attends to capture the relation between query nodes and a few self-adaptively selected key samples. Therefore, it is computationally efficient. Extensive experiments have been conducted on the MOT16, MOT17 and MOT20 benchmarks to demonstrate the superiority of the proposed MOT framework, namely RelationTrack. The experimental results indicate that RelationTrack has surpassed preceding methods significantly and established a new state-of-the-art performance, e.g., IDF1 of 70.5% and MOTA of 67.2% on MOT20.

Bidirectional Multiple Object Tracking Based on Trajectory Criteria in Satellite Videos

Multiple object tracking (MOT) in satellite videos requires to detect all objects belonging to specified categories and identify each object, which plays a basic and necessary role in automatic driving, traffic surveillance and smart city. The traditional MOT methods in satellite videos mostly follow the detection-association framework. However, detection-association framework works under a strict assumption that all objects are correctly localized by the detector. In practice, MOT in satellite videos faces challenges such as low resolution, tiny objects, and the wide field of view, which leads to the degradation of detector performance. In order to reduce the impact of detector degradation, we propose a bi-directional multiple object tracking framework based on trajectory criteria (BMTC) in satellite videos. In BMTC, the SOT tracker carries out locating the objects between consecutive frames and the detector is just used for finding new objects. Therefore, it is less dependent on the detector performance. According to the characteristics of satellite videos, the trajectory criteria are designed to control the state of the tracker, which includes trajectory density, the limit of consecutive virtual motion predictions, and trajectory similarity measurement. Invalid fragment trajectory backtracking is implemented to alleviate the misalignment caused by the above subsection trajectory criteria. The method is validated on the VISO benchmark and SkySat-1 dataset. The experimental results show the improvement of completeness and accuracy, and the proposed tracker achieves state-of-art performance.

LettuceTrack: Detection and tracking of lettuce for robotic precision spray in agriculture.

The precision spray of liquid fertilizer and pesticide to plants is an important task for agricultural robots in precision agriculture. By reducing the amount of chemicals being sprayed, it brings in a more economic and eco-friendly solution compared to conventional non-discriminated spray. The prerequisite of precision spray is to detect and track each plant. Conventional detection or segmentation methods detect all plants in the image captured under the robotic platform, without knowing the ID of the plant. To spray pesticides to each plant exactly once, tracking of every plant is needed in addition to detection. In this paper, we present LettuceTrack, a novel Multiple Object Tracking (MOT) method to simultaneously detect and track lettuces. When the ID of each plant is obtained from the tracking method, the robot knows whether a plant has been sprayed before therefore it will only spray the plant that has not been sprayed. The proposed method adopts YOLO-V5 for detection of the lettuces, and a novel plant feature extraction and data association algorithms are introduced to effectively track all plants. The proposed method can recover the ID of a plant even if the plant moves out of the field of view of camera before, for which existing Multiple Object Tracking (MOT) methods usually fail and assign a new plant ID. Experiments are conducted to show the effectiveness of the proposed method, and a comparison with four state-of-the-art Multiple Object Tracking (MOT) methods is shown to prove the superior performance of the proposed method in the lettuce tracking application and its limitations. Though the proposed method is tested with lettuce, it can be potentially applied to other vegetables such as broccoli or sugar beat.

Multiple Object Tracking in Robotic Applications: Trends and Challenges

The recent advancement in autonomous robotics is directed toward designing a reliable system that can detect and track multiple objects in the surrounding environment for navigation and guidance purposes. This paper aims to survey the recent development in this area and present the latest trends that tackle the challenges of multiple object tracking, such as heavy occlusion, dynamic background, and illumination changes. Our research includes Multiple Object Tracking (MOT) methods incorporating the multiple inputs that can be perceived from sensors such as cameras and Light Detection and Ranging (LIDAR). In addition, a summary of the tracking techniques, such as data association and occlusion handling, is detailed to define the general framework that the literature employs. We also provide an overview of the metrics and the most common benchmark datasets, including Karlsruhe Institute of Technology and Toyota Technological Institute (KITTI), MOTChallenges, and University at Albany DEtection and TRACking (UA-DETRAC), that are used to train and evaluate the performance of MOT. At the end of this paper, we discuss the results gathered from the articles that introduced the methods. Based on our analysis, deep learning has introduced significant value to the MOT techniques in recent research, resulting in high accuracy while maintaining real-time processing.

A Robust Underwater Multiclass Fish-School Tracking Algorithm

State-of-the-art multiple-object tracking methods are frequently applied to people or vehicle tracking, but rarely involve underwater-object tracking. Compared with the processing in non-underwater photos or videos, underwater fish tracking is challenging due to variations in light conditions, water turbidity levels, shape deformations, and the similar appearances of fish. This article proposes a robust underwater fish-school tracking algorithm (FSTA). The FSTA is based on the tracking-by-detection paradigm. To solve the problem of low recognition accuracy in an underwater environment, we add an amendment detection module that uses prior knowledge to modify the detection result. Second, we introduce an underwater data association algorithm for aquatic non-rigid organisms that recombines representation and location information to refine the data matching process and improve the tracking results. The Resnet50-IBN network is used as a re-identification network to track fish. We introduce a triplet loss function based on a centroid to train the feature extraction network. The multiple-object tracking accuracy (MOTA) of the FSTA is 79.1% on the underwater dataset, which shows that it can achieve state-of-the-art performance in a complex real-world marine environment.

Multiple Object Tracking of Drone Videos by a Temporal-Association Network with Separated-Tasks Structure

The task of multi-object tracking via deep learning methods for UAV videos has become an important research direction. However, with some current multiple object tracking methods, the relationship between object detection and tracking is not well handled, and decisions on how to make good use of temporal information can affect tracking performance as well. To improve the performance of multi-object tracking, this paper proposes an improved multiple object tracking model based on FairMOT. The proposed model contains a structure to separate the detection and ReID heads to decrease the influence between every function head. Additionally, we develop a temporal embedding structure to strengthen the representational ability of the model. By combing the temporal-association structure and separating different function heads, the model’s performance in object detection and tracking tasks is improved, which has been verified on the VisDrone2019 dataset. Compared with the original method, the proposed model improves MOTA by 4.9% and MOTP by 1.2% and has better tracking performance than the models such as SORT and HDHNet on the UAV video dataset.

Online Multiple Object Tracking Using Spatial Pyramid Pooling Hashing and Image Retrieval for Autonomous Driving

Multiple object tracking (MOT) is a fundamental issue and has attracted considerable attention in the autonomous driving community. This paper presents a novel MOT framework for autonomous driving. The framework consists of two stages of object representation and data association. In the stage of object representation, we employ appearance, motion, and position features to characterize objects. We design a spatial pyramidal pooling hash network (SPPHNet) to generate the appearance features. Multiple-level representative features in the SPPHNet are mapped into a similarity-preserving binary space, called hash features. The hash features retain the visual discriminability of high-dimensional features and are beneficial for computational efficiency. For data association, a two-tier data association scheme is designed to address the occlusion issue, consisting of an affinity cost model and a hash-based image retrieval model. The affinity cost model accommodates the hash features, disparity, and optical flow as the first tier of data association. The hash-based image retrieval model exploits the hash features and adopts image retrieval technology to handle reappearing objects as the second tier of data association. Experiments on the KITTI public benchmark dataset and our campus scenario sequences show that our method has superior tracking performance to the state-of-the-art vision-based MOT methods.

Multiple object tracking in farmland based on fusion point cloud data

Object detection and tracking in a farmland environment is greatly important to unmanned agricultural machinery navigation technology. Based on the existing methods of fusion point cloud acquisition and object detection, the multiple object tracking method was further researched. Constant velocity model was selected as the motion model, and state estimation was achieved using Kalman filter. Data association between the detectors and trackers was realized through an improved GNN method based on the cost of category conversion. A tracking state machine and two threshold mechanisms were proposed to avoid “combined explosion” disaster. Fusion point cloud data were used for motion judgment to achieve the real motion attributes and speed of the objects based on ground reference. As the results, the average static drift of the tracking method was less than 6.5 cm, the discrimination accuracy of static objects exceeded 75%. Besides, the multiple object tracking precision was less than 35 cm, and the multiple object tracking accuracy exceeded 85%. Therefore, the proposed multiple object tracking method could satisfy the requirements in the farmland environment.