Multi-object Tracking Approach Research Articles

DiffusionTrack: Diffusion Model for Multi-Object Tracking

Multi-object tracking (MOT) is a challenging vision task that aims to detect individual objects within a single frame and associate them across multiple frames. Recent MOT approaches can be categorized into two-stage tracking-by-detection (TBD) methods and one-stage joint detection and tracking (JDT) methods. Despite the success of these approaches, they also suffer from common problems, such as harmful global or local inconsistency, poor trade-off between robustness and model complexity, and lack of flexibility in different scenes within the same video. In this paper we propose a simple but robust framework that formulates object detection and association jointly as a consistent denoising diffusion process from paired noise boxes to paired ground-truth boxes. This novel progressive denoising diffusion strategy substantially augments the tracker's effectiveness, enabling it to discriminate between various objects. During the training stage, paired object boxes diffuse from paired ground-truth boxes to random distribution, and the model learns detection and tracking simultaneously by reversing this noising process. In inference, the model refines a set of paired randomly generated boxes to the detection and tracking results in a flexible one-step or multi-step denoising diffusion process. Extensive experiments on three widely used MOT benchmarks, including MOT17, MOT20, and DanceTrack, demonstrate that our approach achieves competitive performance compared to the current state-of-the-art methods. Code is available at https://github.com/RainBowLuoCS/DiffusionTrack.

Rlm-tracking: online multi-pedestrian tracking supported by relative location mapping

AbstractThe challenge of multi-object tracking stands as a fundamental focus in computer vision research, finding widespread applications in areas such as public safety, transportation, autonomous vehicles, robotics, and other domains involving artificial intelligence. Given the intricate nature of natural scenes, the occurrence of object occlusion and semi-occlusion is commonplace in basic tracking tasks. These factors often result in challenges such as ID switching, object loss, detection errors, and misaligned bounding boxes, thereby significantly impacting the precision of multi-object tracking.This paper aims to address the aforementioned issues and proposes a novel multi-object tracker, incorporating Relative location mapping (RLM) and Target region density (TRD) modeling. The new tracker is more sensitive to differences in the spatial relationships between targets, allowing it to dynamically introduce low-scoring detection boxes into different regions based on the density of target regions in the image. This improves the accuracy of target tracking while avoiding the consumption of a significant amount of computational resources.Our research results indicate that when applying this method to state-of-the-art multi-object tracking approaches, the proposed model achieves improvements of 0.4 to 0.8 points in the HOTA and IDF1 metrics on the MOT17 and MOT20 datasets. This demonstrates the effectiveness of the proposed method in enhancing multi-object tracking performance.

3D LiDAR Multi-Object Tracking with Short-Term and Long-Term Multi-Level Associations

LiDAR-based Multi-Object Tracking (MOT) is a critical technology employed in various autonomous systems, including self-driving vehicles and autonomous delivery robots. In this paper, a novel LiDAR-based 3D MOT approach is introduced. The proposed method was built upon the Tracking-by-Detection (TbD) paradigm and incorporated multi-level associations that exploit an object’s short-term and long-term relationships with the existing tracks. Specifically, the short-term association leverages the fact that objects do not move much between consecutive frames. In contrast, the long-term association assesses the degree to which a long-term trajectory aligns with current detections. The evaluation of the matching between the current detection and the maintained trajectory was performed using a Graph Convolutional Network (GCN). Furthermore, an inactive track was maintained to address the issue of incorrect ID switching for objects that have been occluded for an extended period. The proposed method was evaluated on the KITTI benchmark MOT tracking dataset and achieved a Higher-Order Tracking Accuracy (HOTA) of 75.65%, marking a 5.66% improvement over the benchmark method AB3DMOT, while also accomplishing the number of ID switches of 39, 74 less than AB3DMOT. These results confirmed the effectiveness of the proposed approach in diverse road environments.

A Multi-Object Tracking Approach Combining Contextual Features and Trajectory Prediction

Aiming to solve the problem of the identity switching of objects with similar appearances in real scenarios, a multi-object tracking approach combining contextual features and trajectory prediction is proposed. This approach integrates the motion and appearance features of objects. The motion features are mainly used for trajectory prediction, and the appearance features are divided into contextual features and individual features, which are mainly used for trajectory matching. In order to accurately distinguish the identities of objects with similar appearances, a context graph is constructed by taking the specified object as the master node and its neighboring objects as the branch nodes. A preprocessing module is applied to exclude unnecessary connections in the graph model based on the speed of the historical trajectory of the object, and to distinguish the features of objects with similar appearances. Feature matching is performed using the Hungarian algorithm, based on the similarity matrix obtained from the features. Post-processing is performed for the temporarily unmatched frames to obtain the final object matching results. The experimental results show that the approach proposed in this paper can achieve the highest MOTA.

Pedestrian Tracking Based on Receptive Field Improvement: A One-Shot Multiobject Tracking Approach Based on Vision Sensors

Pedestrian Tracking Based on Receptive Field Improvement: A One-Shot Multiobject Tracking Approach Based on Vision Sensors

SimpleTrack: Rethinking and Improving the JDE Approach for Multi-Object Tracking.

Joint detection and embedding (JDE) methods usually fuse the target motion information and appearance information as the data association matrix, which could fail when the target is briefly lost or blocked in multi-object tracking (MOT). In this paper, we aim to solve this problem by proposing a novel association matrix, the Embedding and GioU (EG) matrix, which combines the embedding cosine distance and GioU distance of objects. To improve the performance of data association, we develop a simple, effective, bottom-up fusion tracker for re-identity features, named SimpleTrack, and propose a new tracking strategy which can mitigate the loss of detection targets. To show the effectiveness of the proposed method, experiments are carried out using five different state-of-the-art JDE-based methods. The results show that by simply replacing the original association matrix with our EG matrix, we can achieve significant improvements in IDF1, HOTA and IDsw metrics, and increase the tracking speed of these methods by around 20%. In addition, our SimpleTrack has the best data association capability among the JDE-based methods, e.g., 61.6 HOTA and 76.3 IDF1, on the test set of MOT17 with 23 FPS running speed on a single GTX2080Ti GPU.

Multiobject Tracking of Wildlife in Videos Using Few-Shot Learning.

Simple SummaryVideo recordings enable scientists to estimate species’ presence, richness, abundance, demography, and activity. The increasing popularity of camera traps has led to a growing interest in developing approaches to more efficiently process images. Advanced artificial intelligence systems can automatically find and identify the species captured in the wild, but they are hampered by dependence on large samples. However, many species rarely occur, such as endangered species, and only a few shot samples are available. Building on recent advances in deep learning and few-shot learning technologies, we developed a multiobject-tracking approach based on a tracking-by-detection paradigm for wildlife to improve multiobject-tracking performance. We hope that it will be beneficial to ecology and wildlife biology by speeding up the process of multiobject tracking in the wild.Camera trapping and video recording are now ubiquitous in the study of animal ecology. These technologies hold great potential for wildlife tracking, but are limited by current learning approaches, and are hampered by dependence on large samples. Most species of wildlife are rarely captured by camera traps, and thus only a few shot samples are available for processing and subsequent identification. These drawbacks can be overcome in multiobject tracking by combining wildlife detection and tracking with few-shot learning. This work proposes a multiobject-tracking approach based on a tracking-by-detection paradigm for wildlife to improve detection and tracking performance. We used few-shot object detection to localize objects using a camera trap and direct video recordings that could augment the synthetically generated parts of separate images with spatial constraints. In addition, we introduced a trajectory reconstruction module for better association. It could alleviate a few-shot object detector’s missed and false detections; in addition, it could optimize the target identification between consecutive frames. Our approach produced a fully automated pipeline for detecting and tracking wildlife from video records. The experimental results aligned with theoretical anticipation according to various evaluation metrics, and revealed the future potential of camera traps to address wildlife detection and tracking in behavior and conservation.

Multi-object tracking based on attention networks for Smart City system

The Smart City is a hot topic at present. Pedestrian tracking and behavior analysis play an important role in Smart City system. Therefore, it is urgent to design an universal and robust multi-object tracking(MOT) algorithm. MOT is a challenging problem with numerous significant applications, including robot navigation, autonomous driving and pedestrian behavior analysis. Among the existing MOT approaches, MOT models which joint learning detection and Re-ID are attracting increasing attention. Subsequently, there is competition between these two tasks jointly in a network inevitably leading to performance degradation. To address these difficulties, we propose an adaptive joint learning approach, called UnionTrack, for MOT in this paper. It handles with the problem of uniform learning between tasks in online MOT system. Firstly, UnionTrack introduces a scale-aware attention network that aggregates features from different scales for object localization and learning of the corresponding discriminative embeddings. Secondly, a cross-correlation module is designed to construct task-relevant feature maps and interact adaptively during training detection and Re-ID. Then, we perform inter-frame correlation at different stages by using high and low confidence detection to avoid occlusions and backgrounds. UnionTrack is trained in an end-to-end manner to better accommodate the training between detection and Re-ID tasks. Finally, our proposed UnionTrack is compared with state-of-the-art methods in extensively experiments and detailed performance analysis is performed in MOT15, MOT16, MOT17 and MOT20 which are all urban scene to demonstrate the effectiveness of our proposed method. Looking forward to playing a role in the Smart City systems.

Adaptive δ-Generalized Labeled Multi-Bernoulli Filter for Multi-Object Detection and Tracking

The δ-generalized labeled multi-Bernoulli (δ-GLMB) filter is an efficient approach for multiobject tracking in case of high clutter density and low detection probability. However, the formulation of the original δ-GLMB filter requires that the birth δ-GLMB filtering density is known a priori. It is inapplicable for the birth object appearing from unknown positions. To address this problem, an adaptive δ-GLMB filter is proposed to detect and track the birth objects with unknown position information. This adaptive filter establishes the birth δ-GLMB filtering density by using measurements at previous three successive times. Simulation results indicate that the proposed adaptive δ-GLMB filter may efficiently detect and track the multiple objects with unknown positions. Simulation results also demonstrate that the proposed adaptive δ-GLMB filter performs better than the other existing adaptive filters.

Hierarchical-Matching-Based Online and Real-Time Multi-Object Tracking with Deep Appearance Features

Based on tracking-by-detection, we propose a hierarchical-matching-based online and real-time multi-object tracking approach with deep appearance features, which can effectively reduce the false positives (FP) in tracking. For the purpose of increasing the accuracy rate of data association, we define the trajectory confidence using its position information, appearance information, and the information of historical relevant detections, after which we can classify the trajectories into different levels. In order to obtain discriminative appearance features, we developed a deep convolutional neural network to extract the appearance features of objects and trained it on a large-scale pedestrian re-identification dataset. Last but not least, we used the proposed diverse and hierarchical matching strategy to associate detection and trajectory sets. Experimental results on the MOT benchmark dataset show that our proposed approach performs well against other online methods, especially for the metrics of FP and frames per second (FPS).

Spatial-Semantic and Temporal Attention Mechanism-Based Online Multi-Object Tracking.

Multi-object tracking (MOT) plays a crucial role in various platforms. Occlusion and insertion among targets, complex backgrounds and higher real-time requirements increase the difficulty of MOT problems. Most state-of-the-art MOT approaches adopt the tracking-by-detection strategy, which relies on compute-intensive sliding windows or anchoring schemes to detect matching targets or candidates in each frame. In this work, we introduce a more efficient and effective spatial–temporal attention scheme to track multiple objects in various scenarios. Using a semantic-feature-based spatial attention mechanism and a novel Motion Model, we address the insertion and location of candidates. Some online-learned target-specific convolutional neural networks (CNNs) were used to estimate target occlusion and classify by adapting the appearance model. A temporal attention mechanism was adopted to update the online module by balancing current and history frames. Extensive experiments were performed on Karlsruhe Institute of Technologyand Toyota Technological Institute (KITTI) benchmarks and an Armored Target Tracking Dataset (ATTD) built for ground-armored targets. Experimental results show that the proposed method achieved outstanding tracking performance and met the actual application requirements.

Long-term multiobject tracking using alternative correlation filters

We propose a real-time multiobject-tracking approach that is minimally affected by environmental conditions and target appearance change. The aim of the proposed approach is to track any object in a scene, regardless of object type, since tracking all of the objects in a scene is critical and widely used in surveillance applications. Thus, motion detection results are used to initialize the trackers. The proposed object-tracking approach is realized with two types of independent correlation filters estimating location and scale. Alternative correlation filters representing different appearances of the target are also proposed in order to increase the robustness of the approach to scene and target changes. Tracking sustainability is provided by putting alternative correlation filters into use when the quality of the tracking output decreases to a critical level. Motion blobs are also used to minimize object boundary drift, which is a challenging problem, especially for long-term tracking. The proposed approach was tested on an object-tracking benchmark dataset and outperformed most state-of-the-art methods.

Multi-Object Tracking with Correlation Filter for Autonomous Vehicle.

Multi-object tracking is a crucial problem for autonomous vehicle. Most state-of-the-art approaches adopt the tracking-by-detection strategy, which is a two-step procedure consisting of the detection module and the tracking module. In this paper, we improve both steps. We improve the detection module by incorporating the temporal information, which is beneficial for detecting small objects. For the tracking module, we propose a novel compressed deep Convolutional Neural Network (CNN) feature based Correlation Filter tracker. By carefully integrating these two modules, the proposed multi-object tracking approach has the ability of re-identification (ReID) once the tracked object gets lost. Extensive experiments were performed on the KITTI and MOT2015 tracking benchmarks. Results indicate that our approach outperforms most state-of-the-art tracking approaches.

Multi-object tracking through learning relational appearance features and motion patterns

Absract Multi-object tracking (MOT) is to simultaneously track multiple targets, e.g., pedestrians in this work, through locating them and maintaining their identities to make their individual trajectories. Despite of recent advances in object detection, MOT based on the tracking-by-detection principle is a still yet challenging and difficult task in complex and crowded conditions. For example, due to occlusion, missed object detection, and frequent entering and leaving of object in a scene, tracking failures such as identity switches and trajectory fragmentation can often occur. To tackle the issues, a new data association approach, namely, the relational appearance features and motion patterns learning (RAFMPL)-based data association, is proposed for facilitating MOT. In RAFMPL-MOT, the proposed relational features-based appearance model is different from conventional approaches in that it generates tracklets based on relational information by selecting one reference object and utilizing the feature differences between the reference object and the other objects. In addition, the motion patterns learning-based motion model enables linear and nonlinear confident motions patterns to be considered in data association. The proposed approach can effectively cover the key difficulties of MOT. In particular, using RAFMPL-MOT, it is possible to assign the same ID for the object that has been disappeared (even for moderately long period) and then is reappeared in the scene more robustly. Further, it also improves its robustness for occlusion problems frequently occurring in real situations. The experimental results show that the RAFMPL-MOT could generally achieve outperformance compared to the existing competitive MOT approaches.

Online multi-object tracking with efficient track drift and fragmentation handling.

This paper addresses the problem of multi-object tracking in complex scenes by a single, static, uncalibrated camera. Tracking-by-detection is a widely used approach for multi-object tracking. Challenges still remain in complex scenes, however, when this approach has to deal with occlusions, unreliable detections (e.g., inaccurate position/size, false positives, or false negatives), and sudden object motion/appearance changes, among other issues. To handle these problems, this paper presents a novel online multi-object tracking method, which can be fully applied to real-time applications. First, an object tracking process based on frame-by-frame association with a novel affinity model and an appearance update that does not rely on online learning is proposed to effectively and rapidly assign detections to tracks. Second, a two-stage drift handling method with novel track confidence is proposed to correct drifting tracks caused by the abrupt motion change of objects under occlusion and prolonged inaccurate detections. In addition, a fragmentation handling method based on a track-to-track association is proposed to solve the problem in which an object trajectory is broken into several tracks due to long-term occlusions. Based on experimental results derived from challenging public data sets, the proposed method delivers an impressive performance compared with other state-of-the-art methods. Furthermore, additional performance analysis demonstrates the effect and usefulness of each component of the proposed method.

Multi-person detecting and tracking based on RGB-D sensor for a robot vision system

In this paper, we address the problem of automatically detecting and tracking a variable number of objects in complex scenes using a RGB-D sensor on the robot system. We propose a novel approach for multi-object detecting by fusing RGB information and depth information. Meanwhile, this paper presents a robust multi-cue approach for multi-object tracking. A spatiotemporal object representation is proposed, which combines a generative colour model and a discriminative texture classifier. We employ a Bayesian framework based on particle filtering to achieve integrated object detection and tracking from a robot vision system. The experimental results show that the proposed method yields good tracking performance in real world environment.

A Novel Multiobject Tracking Approach in the Presence of Collision and Division.

This paper aims to develop a general framework for accurately tracking and quantitatively characterizing multiple cells (objects) when collision and division between cells arise. Through introducing three types of interaction events among cells, namely, independence, collision, and division, the corresponding dynamic models are defined and an augmented interacting multiple model particle filter tracking algorithm is first proposed for spatially adjacent cells with varying size. In addition, to reduce the ambiguity of correspondence between frames, both the estimated cell dynamic parameters and cell size are further utilized to identify cells of interest. The experiments have been conducted on two real cell image sequences characterized with cells collision, division, or number variation, and the resulting dynamic parameters such as instant velocity, turn rate were obtained and analyzed.

Coupled Object Detection and Tracking from Static Cameras and Moving Vehicles

We present a novel approach for multi-object tracking which considers object detection and spacetime trajectory estimation as a coupled optimization problem. Our approach is formulated in a Minimum Description Length hypothesis selection framework, which allows it to recover from mismatches and temporarily lost tracks. Building upon a state-of-the-art object detector, it performs multiview/multicategory object recognition to detect cars and pedestrians in the input images. The 2D object detections are checked for their consistency with (automatically estimated) scene geometry and are converted to 3D observations which are accumulated in a world coordinate frame. A subsequent trajectory estimation module analyzes the resulting 3D observations to find phyically plausible spacetime trajectories. Tracking is achieved by performing model selection after every frame. At each time instant, our approach searches for the globally optimal set of spacetime trajectories which provides the best explanation for the current image and for all evidence collected so far, while satisfying the constraints that no two objects may occupy the same physical space, nor explain the same image pixels at any point in time. Successful trajectory hypotheses are then fed back to guide object detection in future frames. The optimization procedure is kept efficient throught incremental computation and conservative hypothesis pruning. We evaluate our approach on several challenging video sequences and demonstrate its performance on both a surveillance-type scenario and a scenario where the input videos are taken from inside a moving vehicle passing through crowded city areas.