Data Association Method Research Articles

A data association method based on multi-radar detection of GEO targets

A data association method based on multi-radar detection of GEO targets

Deep Efficient Data Association for Multi-Object Tracking: Augmented with SSIM-Based Ambiguity Elimination.

Recently, to address the multiple object tracking (MOT) problem, we harnessed the power of deep learning-based methods. The tracking-by-detection approach to multiple object tracking (MOT) involves two primary steps: object detection and data association. In the first step, objects of interest are detected in each frame of a video. The second step establishes the correspondence between these detected objects across different frames to track their trajectories. This paper proposes an efficient and unified data association method that utilizes a deep feature association network (deepFAN) to learn the associations. Additionally, the Structural Similarity Index Metric (SSIM) is employed to address uncertainties in the data association, complementing the deep feature association network. These combined association computations effectively link the current detections with the previous tracks, enhancing the overall tracking performance. To evaluate the efficiency of the proposed MOT framework, we conducted a comprehensive analysis of the popular MOT datasets, such as the MOT challenge and UA-DETRAC. The results showed that our technique performed substantially better than the current state-of-the-art methods in terms of standard MOT metrics.

Accurate 3D LiDAR SLAM System Based on Hash Multi-Scale Map and Bidirectional Matching Algorithm.

Simultaneous localization and mapping (SLAM) is a hot research area that is widely required in many robotics applications. In SLAM technology, it is essential to explore an accurate and efficient map model to represent the environment and develop the corresponding data association methods needed to achieve reliable matching from measurements to maps. These two key elements impact the working stability of the SLAM system, especially in complex scenarios. However, previous literature has not fully addressed the problems of efficient mapping and accurate data association. In this article, we propose a novel hash multi-scale (H-MS) map to ensure query efficiency with accurate modeling. In the proposed map, the inserted map point will simultaneously participate in modeling voxels of different scales in a voxel group, enabling the map to represent objects of different scales in the environment effectively. Meanwhile, the root node of the voxel group is saved to a hash table for efficient access. Secondly, considering the one-to-many (1 ×103 order of magnitude) high computational data association problem caused by maintaining multi-scale voxel landmarks simultaneously in the H-MS map, we further propose a bidirectional matching algorithm (MSBM). This algorithm utilizes forward-reverse-forward projection to balance the efficiency and accuracy problem. The proposed H-MS map and MSBM algorithm are integrated into a completed LiDAR SLAM (HMS-SLAM) system. Finally, we validated the proposed map model, matching algorithm, and integrated system on the public KITTI dataset. The experimental results show that, compared with the ikd tree map, the H-MS map model has higher insertion and deletion efficiency, both having O(1) time complexity. The computational efficiency and accuracy of the MSBM algorithm are better than that of the small-scale priority matching algorithm, and the computing speed of the MSBM achieves 49 ms/time under a single CPU thread. In addition, the HMS-SLAM system built in this article has also reached excellent performance in terms of mapping accuracy and memory usage.

Improvement methods for feature matching in Basalt SLAM

Abstract Visual simultaneous localization and mapping (SLAM) is widely used to provide localization information for mobile robots, but most visual SLAM algorithms lack effective data association methods, resulting in unstable position tracking results. In this paper, relevant improvement schemes for global feature matching in the Basalt SLAM algorithm are proposed to enhance the effectiveness of its data association method. Firstly, the more efficient box average difference (BAD) descriptor is used instead of the ORB descriptor. Secondly, secondary matching is performed by narrowing the matching range with the epipolar geometric constraint to expand matching pairs. Finally, grid-based motion statistics (GMS) and histogram-based rotation consistency constraints are introduced to eliminate false matches. Experiments on the EuRoC dataset demonstrate that the proposed methods significantly enhance the effectiveness and stability of visual data association and improve the localization accuracy of the Basalt SLAM system.

Improved Gaussian processes linear JPDA filter for multiple extended targets tracking in dense clutter

To track multiple extended targets in dense clutter, an improved Gaussian processes linear joint probabilistic data association (IGP-LJPDA) method is proposed. This method consists of two stages. In the first stage, measurements are associated with targets, and pseudo-measurements are constructed. By integrating linear JPDA with Gaussian processes, probabilities for each measurement's assignment to each target are determined. Following this, a novel pseudo-measurement model is devised by combining the marginal probability of each target with measurements belonging to their respective basic point neighborhood, enabling more efficient state updates. In the second stage, pseudo-measurements are associated with the contour, and states of extended targets are updated accordingly. By associating pseudo-measurements with Gaussian processes basic points of each target, the algorithm achieves globally optimal association. The proposed algorithm demonstrates a notable improvement in Intersection over Union (IOU) metrics in environments with significantly higher clutter rates compared to existing methods.

Object Tracking Using Computer Vision: A Review

Object tracking is one of the most important problems in computer vision applications such as robotics, autonomous driving, and pedestrian movement. There has been a significant development in camera hardware where researchers are experimenting with the fusion of different sensors and developing image processing algorithms to track objects. Image processing and deep learning methods have significantly progressed in the last few decades. Different data association methods accompanied by image processing and deep learning are becoming crucial in object tracking tasks. The data requirement for deep learning methods has led to different public datasets that allow researchers to benchmark their methods. While there has been an improvement in object tracking methods, technology, and the availability of annotated object tracking datasets, there is still scope for improvement. This review contributes by systemically identifying different sensor equipment, datasets, methods, and applications, providing a taxonomy about the literature and the strengths and limitations of different approaches, thereby providing guidelines for selecting equipment, methods, and applications. Research questions and future scope to address the unresolved issues in the object tracking field are also presented with research direction guidelines.

A dynamic detection and data association method based on probabilistic models for visual SLAM

Visual Simultaneous Localization and Mapping (VSLAM) is a critical foundation in mobile robotics and augmented reality (AR). However, VSLAM faces challenges in dynamic environments since both the camera and the object are in motion, which contradicts the classical static scene assumption. Generally, multi-view geometry is employed for static features to estimate camera pose and reconstruct environment maps. Hence, dynamic feature detection and data association become key issues in dynamic VSLAM. To solve these problems, we propose an innovative probability-based approach that combines instance segmentation and nonparametric Kolmogorov–Smirnov test approaches to detect the distribution of features on an object. Furthermore, we propose a data association algorithm based on the Bayesian model that comprehensively utilizes the descriptors of feature points and their spatial information. Experiments on the KITTI public dataset and the Oxford Multi-motion dataset validate the effectiveness of our method.

Multiple Moving Vehicles Tracking Algorithm with Attention Mechanism and Motion Model

With the acceleration of urbanization and the increasing demand for travel, current road traffic is experiencing rapid growth and more complex spatio-temporal logic. Vehicle tracking on roads presents several challenges, including complex scenes with frequent foreground–background transitions, fast and nonlinear vehicle movements, and the presence of numerous unavoidable low-score detection boxes. In this paper, we propose AM-Vehicle-Track, following the proven-effective paradigm of tracking by detection (TBD). At the detection stage, we introduce the lightweight channel block attention mechanism (LCBAM), facilitating the detector to concentrate more on foreground features with limited computational resources. At the tracking stage, we innovatively propose the noise-adaptive extended Kalman filter (NSA-EKF) module to extract vehicles’ motion information while considering the impact of detection confidence on observation noise when dealing with nonlinear motion. Additionally, we borrow the Byte data association method to address unavoidable low-score detection boxes, enabling secondary association to reduce ID switches. We achieve 42.2 MOTA, 51.2 IDF1, and 364 IDs on the test set of VisDrone-MOT with 72 FPS. The experimental results showcase our approach’s highly competitive performance, attaining SOTA tracking performance with a fast speed.

SCGTracker: Spatio-temporal correlation and graph neural networks for multiple object tracking

Multiple Object Tracking is an important vitally important fundamental task in computer vision. Visual tracking becomes challenging when objects move in groups and are obscured from each other. There are two mainstream solution strategies for these group models. One is to transform the data association problem into a graph matching problem for solving, while the other is to apply the social power model as an advanced constraint for group tracking. In the former case, the solving difficulty geometric growth as the number of tracked objects increases, and the computing efficiency for real-time tracking demand cannot be met. The latter strategy tends to set up fixed-size groups or offline training rules, resulting in a lack of flexibility that limits their scenario generalization. According to the shortcomings of existing methods, this paper proposes a novel multiple object tracking method with spatio-temporal correlation and graph neural networks. Firstly, the relational features of the historical trajectories are extracted through the spatio-temporal relationship learning module, which models the spatio-temporal correlations of the objects and dynamically constructs the group structure online. Then, the graph neural network is combined with appearance and motion information, and the similarity between each detection and tracklet is used as a weight in node feature aggregation to make powerful distinctions between node features. Meanwhile, the spatio-temporal correlation method is also used to solve target loss issues caused by occlusion. Even collocated with linearly assigned data association method, good tracking results are still achieved, with a low computational complexity. Experiments on three challenging public datasets, namely MOT16, MOT17, and MOT20, validated the accuracy and efficiency of the proposed tracking method.

BPMTrack: Multi-Object Tracking with Detection Box Application Pattern Mining.

The key to multi-object tracking is its stability and the retention of identity information. A common problem with most detection-based approaches is trusting and using all the detector outputs for the association. However, some settings of detectors can affect stable long-range tracking. Based on the principle of reducing the association noise in the detection processing step, we propose a new framework, the Box application Pattern Mining Tracker (BPMTrack), to address this issue. Specifically, we worked on three main aspects: output threshold, association strategy, and motion model. Due to the problem of inconsistency between classification scores and localization accuracy, we propose the Box Quality Estimation Network (BQENet) to predict the localization quality scores of all detections in the current frame, reserving high-quality boxes for the tracker. In addition, based on observations of intensive scenarios, we propose a simple and effective data association method, the Non-Maximum Suppression Integration (NMSI) matching strategy. It recovers the Non-Maximum Suppression (NMS) detection, inputs them into BQENet, and then performs hierarchical matching with reasonable control of box priority to alleviate the problem of absent objects caused by occlusion. Finally, we propose an improved Measurement Correct and Noise Scale (MCNS) Kalman algorithm to improve the prediction accuracy of object positions and, thus, the association quality. We performed an extensive ablation evaluation of the proposed framework to prove its effectiveness. Moreover, the three tracking benchmarks show our method's accuracy and long-distance performance.

The Data Fusion and Semantic Segmentation Method Based on Multi-Sensor System

The design and implementation of a prototype environmental perception system are introduced, which consists of the multi-camera, LiDAR, and IMU sensor. The processing steps of the environmental data collected by the multi-sensor platform include joint calibration, data fusion, and semantic segmentation. First, the optimization method based on the epipolar constraint is proposed for the joint calibration of the multi-camera and the LiDAR system. In the data fusion section, an improved data association method for the point cloud is proposed where the foreground segmentation method is used to reduce scale estimation error due to the scale-sudden-change. Finally, the semantic segmentation method based on the deep-learning is proposed. The convolutional neural network based on the Squeeze-and-Excitation Net is designed to identify and classify the point cloud data accurately. The physical simulation result shows that the system collects and identifies the environmental information accurately.

Research on multi-target data association of the infrared fish-eye warning system

The multi-target data association method is studied in order to realize multi-target tracking in infrared fish-eye warning system. The Neural Joint Probabilistic Data Association (NJPDA) algorithm is analyzed. It is found that the NJPDA algorithm only considers the distance information between the measurement and the target in the data association process, and its tracking accuracy needs to be further improved. Therefore, a new method fused with direction information is proposed based on the NJPDA algorithm. The proposed algorithm defines the concept of direction difference, introduces the direction information of target motion, and modifies the likelihood function by Gaussian weighting method, so as to fuse the direction information of target motion into the calculation of data interconnection probability. Experimental results demonstrate that the tracking success rate of the proposed algorithm is nearly 10 % higher than that of JPDA and NJPDA algorithms and its consuming time meets the real-time requirement of the infrared fish-eye warning system.

Impact of moving target on underwater positioning by using state measurement

The localization of moving targets in an underwater acoustic wireless sensor network (UAWSN) is inaccurate due to the various underwater forces (viscous, hydrodynamic forces, perturbation of underwater). The false measurements in the sensor network cause position errors and velocity errors which disrupt the localization of the moving target. A randomly fluctuated spillover effect is introduced in the present paper. The absorption losses generated due to the spillover effect cause false measurements of the moving target. Theorem 1 describes the genesis of these absorption losses and their consequences in UAWSN. The measurements from each moving target in the presence of absorption losses are formulated in the elliptical region. A joint probabilistic data association (JPDA) method is proposed to quantify the false measurements in the elliptical region. A moving target state estimation (MTSE) algorithm is proposed to eliminate the false measurements from the moving targets and to measure the localization of moving targets with the help of the propagation speed of targets. The theoretical measurements of position RMSE and velocity RMSE are verified with standard methods. The proposed MTSE method improves the localization performance of the moving targets by 29.42 % and reduces 32.16 % of position errors and 36.23 % of velocity errors up to 550 ​m. The proposed algorithm will be useful for the sub-aquatic Internet of underwater things (IoUT).

Target‐tracking algorithm based on improved probabilistic data association

AbstractWhen tracking a single manoeuvring target in clutter environment, when the number of effective measurements within the detection threshold is small, it usually has a greater and more obvious impact on target‐tracking results. If the observation data error is large at this time, the tracking position and speed error will be larger. To solve this problem, a target‐tracking algorithm based on improved probabilistic data association is proposed in this paper. By dynamically adjusting the detection threshold, the effective quantity within the detection threshold of each frame is basically stable. Simulation results show that the improved algorithm is more accurate in location and speed than the traditional probabilistic data association method and Kalman filter, and the availability and effectiveness of the algorithm are verified.

Elliptic localization of multiple objects without position and synchronization of the transmitter

Multi-object elliptic localization is a challenging problem, considering that the association between the indirect path measurements and the objects is not known, and the transmitter is un-coordinated without synchronization with the receivers. Realizing that the direct path measurements do not have the data association issue, we propose a two-stage estimation method for this problem. In the first stage, a minimum measurement solution (MMS) method is developed to obtain a coarse estimate of the transmitter position and range offset coming from the unsynchronized transmitter and receivers, and the best linear unbiased estimator (BLUE) follows to improve the coarse estimate. In the second stage, we first propose a data association method based on the MMS for elliptic localization with known transmitter position, and then propose a BLUE for estimating the object positions using the indirect path measurements and the transmitter position and range offset estimate. The proposed method does not require any prior information about the object positions. Moreover, it has closed-form solutions in both stages, and thus is very computationally efficient. Mean square error analysis is conducted to show that the performance achieves the Cramer–Rao lower bound under correct data association and uncorrelated noise between the direct and indirect paths. Simulation results validate the good performance of the proposed method.

Study on Multi-Heterogeneous Sensor Data Fusion Method Based on Millimeter-Wave Radar and Camera.

This study presents a novel multimodal heterogeneous perception cross-fusion framework for intelligent vehicles that combines data from millimeter-wave radar and camera to enhance target tracking accuracy and handle system uncertainties. The framework employs a multimodal interaction strategy to predict target motion more accurately and an improved joint probability data association method to match measurement data with targets. An adaptive root-mean-square cubature Kalman filter is used to estimate the statistical characteristics of noise under complex traffic scenarios with varying process and measurement noise. Experiments conducted on a real vehicle platform demonstrate that the proposed framework improves reliability and robustness in challenging environments. It overcomes the challenges of insufficient data fusion utilization, frequent leakage, and misjudgment of dangerous obstructions around vehicles, and inaccurate prediction of collision risks. The proposed framework has the potential to advance the state of the art in target tracking and perception for intelligent vehicles.

Tracking and Data Association Based on Reinforcement Learning

Currently, most multi-target data association methods require the assumption that the target motion model is known, but this assumption is clearly not valid in a real environment. In the case of an unknown system model, the influence of environmental clutter and sensor detection errors on the association results should be considered, as well as the occurrence of strong target maneuvers and the sudden appearance of new targets during the association process. To address these problems, this paper designs a target tracking and data association algorithm based on reinforcement learning. First, this algorithm combines the dynamic exploration capability of reinforcement learning and the long-time memory function of LSTM network to design a policy network that predicts the probability of associating a point with its various possible source targets. Then, the Bayesian network and the multi-order least squares curve fitting method are combined to predict the location of target, and the results are fed into the Bayesian recursive function to obtain the reward. Simultaneously, some corresponding mechanisms are proposed for possible problems that interfere with the association process. Finally, the simulation experimental results show that this algorithm associates the results with higher accuracy compared to other algorithms when faced with the above problem.

Multiple object tracking based on quadratic graph matching

AbstractRecently, with the development of deep‐learning, the performance of multiple object tracking (MOT) algorithm based on deep neural networks has been greatly improved. However, it is still a difficult problem to successfully solve the tracking misalignment caused by occlusion and complex tracking scenes. Most of the work focusses on designing a sophisticated network, only little work focusses on data association. Actually, data association is very helpful in MOT. In this study, data association in tracking is taken as the main research task, and the authors introduce quadratic graph matching into MOT. Considering the objects in each frame as a graph, we can model the data association between two frames as a quadratic graph matching problem. And then it is transformed into a convex quadratic optimisation problem. Introducing high‐order structure features into the matching function effectively solves some tracking problems. The data association is designed into the tracking model as an overall solution. Experiments show that the proposed data association algorithm performs favourably against several state‐of‐the‐art data association methods and can be used in any tracking‐by‐detection method. Our code is publicly available from https://github.com/gjy0514/G_Model.

Real-Time Moving Ship Detection from Low-Resolution Large-Scale Remote Sensing Image Sequence

Optical remote sensing ship target detection has become an essential means of ocean supervision, coastal defense, and frontier defense. Accurate, effective, fast, and real-time remote sensing data processing is the critical technology in this field. This paper proposes a real-time detection algorithm for moving targets in low-resolution wide-area remote sensing images, which includes four steps: pre-screening, simplified HOG feature identification, sequence correlation identification, and facilitated Yolo identification. It can effectively detect and track targets in low-resolution sequence data. Firstly, iterative morphological processing was used to improve the contrast of low-resolution ship target profile edge features compared with the sea surface background. Next, the target area after adaptive segmentation was used to eliminate false alarms. As a result, the invalid background information of extensive comprehensive data was quickly eliminated. Then, support vector machine classification of S-HOG feature was carried out for suspected targets, and interference such as islands and reefs, broken clouds, and waves were eliminated according to the shape characteristics of ship targets. The method of multi-frame data association and searching for adjacent target information between frames was adopted to eliminate the interference of static targets and broken clouds with similar contours. Finally, the sequential marks were further trained and learned, and further false alarm elimination was completed based on the clipped Yolo network. Compared with the traditional Yolo Tiny V2/V3 series network, this method had higher computational speed and better detection performance. The F1 number of detection results was increased by 3%, and the calculation time was reduced by 66%.

SemanticLoop: Loop Closure With 3D Semantic Graph Matching

Loop closure can effectively correct the accumulated error in robot localization, which plays a critical role in the long-term navigation of the robot. Traditional appearance-based methods rely on local features and are prone to failure in ambiguous environments. On the other hand, object recognition can infer objects' category, pose, and extent. These objects can serve as stable semantic landmarks for viewpoint-independent and non-ambiguous loop closure. However, there is a critical object-level data association problem due to the lack of efficient and robust algorithms. We introduce a novel object-level data association algorithm, which incorporates IoU, instance-level embedding, and detection uncertainty, formulated as a linear assignment problem. Then, we model the objects as TSDF volumes and represent the environment as a 3D graph with semantics and topology. Next, we propose a graph matching-based loop detection based on the reconstructed 3D semantic graphs and correct the accumulated error by aligning the matched objects. Finally, we refine the object poses and camera trajectory in an object-level pose graph optimization. Experimental results show that the proposed object-level data association method significantly outperforms the commonly used nearest neighbor method in accuracy. Our graph matching-based loop closure is more robust to environmental appearance changes than existing appearance-based methods.