Perform Data Association Research Articles

Group-k consistent measurement set maximization via maximum clique over k-uniform hypergraphs for robust multi-robot map merging

This paper unifies the theory of consistent-set maximization for robust outlier detection in a simultaneous localization and mapping framework. We first describe the notion of pairwise consistency before discussing how a consistency graph can be formed by evaluating pairs of measurements for consistency. Finding the largest set of consistent measurements is transformed into an instance of the maximum clique problem and can be solved relatively quickly using existing maximum-clique solvers. We then generalize our algorithm to check consistency on a group- k basis by using a generalized notion of consistency and using generalized graphs. We also present modified maximum clique algorithms that function over generalized graphs to find the set of measurements that is internally group- k consistent. We address the exponential nature of group- k consistency and present methods that can substantially decrease the number of necessary checks performed when evaluating consistency. We extend our prior work to perform data association, and to multi-agent systems in both simulation and hardware, and provide a comparison with other state-of-the-art methods.

A Graph-Based Algorithm for Robust Sequential Localization Exploiting Multipath for Obstructed-LOS-Bias Mitigation

This paper presents a factor graph formulation and particle-based sum-product algorithm (SPA) for robust sequential localization in multipath-prone environments. The proposed algorithm jointly performs data association, sequential estimation of a mobile agent position, and adapts all relevant model parameters. We derive a novel non-uniform false alarm (FA) model that captures the delay and amplitude statistics of the multipath radio channel. This model enables the algorithm to indirectly exploit position-related information contained in the multipath components (MPCs) for the estimation of the agent position without using any prior information such as floorplan information or training data. Using simulated and real measurements in different channel conditions, we demonstrate that the algorithm can provide high-accuracy position estimates even in fully obstructed line-of-sight (OLOS) situations and show that the performance of our algorithm constantly attains the posterior Cramér-Rao lower bound (P-CRLB), facilitating the additional information contained in the presented FA model. The algorithm is shown to provide robust estimates in both, dense multipath channels as well as channels showing specular, resolved MPCs, significantly outperforming state-of-the-art radio-based localization methods.

Efficient and Accurate Object Detection With Simultaneous Classification and Tracking Under Limited Computing Power

Interacting with the environment, such as object detection and tracking, is a crucial ability of mobile robots. Besides high accuracy, efficiency in terms of processing effort and energy consumption are also desirable. To satisfy both requirements, we propose a detection framework based on simultaneous classification and tracking in the point stream. In this framework, a tracker performs data association in sequences of the point cloud, guiding the detector to avoid redundant processing (i.e. classifying already-known objects). For objects whose classification is not sufficiently certain, a fusion model is designed to fuse selected key observations that provide different perspectives across the tracking span. Therefore, performance (accuracy and efficiency of detection) can be enhanced. This method is particularly suitable for detecting and tracking moving objects, a process which would require expensive computations if solved using conventional procedures. Experiments were conducted on the benchmark dataset, and the results showed that the proposed method outperforms original tracking-by-detection approaches in both efficiency and accuracy.

Object Clustering With Dirichlet Process Mixture Model for Data Association in Monocular SLAM

Semantic simultaneous localization and mapping (SLAM) with a monocular camera is particularly attractive because of the deployment simplicity and economic availability. Data association problem which assigns unique identities for objects shown in multiple frames plays a fundamental role in semantic SLAM. Previous prevalent methods which mainly focused on associating geometric KeyPoints are no longer suitable. Some naive methods that rely on object distance or 2-D/3-D Intersection over Union are also vulnerable when occlusions happen. In this article, we propose a novel data association method for cuboid landmarks based on Dirichlet process mixture model. By jointly considering object class, position, and size, our method can perform data association robustly. We evaluated our method in simulated datasets, public benchmark KITTI, and on a real robot in an office environment. Experimental results show that our method not only associates cuboids robustly but also achieves SOTA pose estimation accuracy in monocular SLAMs.

Research on multi-target tracking method based on multi-sensor fusion

Purpose Intelligent and connected vehicle technology is in the ascendant. High-level autonomous driving places more stringent requirements on the accuracy and reliability of environmental perception. Existing research works on multitarget tracking based on multisensor fusion mostly focuses on the vehicle perspective, but limited by the principal defects of the vehicle sensor platform, it is difficult to comprehensively and accurately describe the surrounding environment information. Design/methodology/approach In this paper, a multitarget tracking method based on roadside multisensor fusion is proposed, including a multisensor fusion method based on measurement noise adaptive Kalman filtering, a global nearest neighbor data association method based on adaptive tracking gate, and a Track life cycle management method based on M/N logic rules. Findings Compared with fixed-size tracking gates, the adaptive tracking gates proposed in this paper can comprehensively improve the data association performance in the multitarget tracking process. Compared with single sensor measurement, the proposed method improves the position estimation accuracy by 13.5% and the velocity estimation accuracy by 22.2%. Compared with the control method, the proposed method improves the position estimation accuracy by 23.8% and the velocity estimation accuracy by 8.9%. Originality/value A multisensor fusion method with adaptive Kalman filtering of measurement noise is proposed to realize the adaptive adjustment of measurement noise. A global nearest neighbor data association method based on adaptive tracking gate is proposed to realize the adaptive adjustment of the tracking gate.

SimpleTrack: Rethinking and Improving the JDE Approach for Multi-Object Tracking.

Joint detection and embedding (JDE) methods usually fuse the target motion information and appearance information as the data association matrix, which could fail when the target is briefly lost or blocked in multi-object tracking (MOT). In this paper, we aim to solve this problem by proposing a novel association matrix, the Embedding and GioU (EG) matrix, which combines the embedding cosine distance and GioU distance of objects. To improve the performance of data association, we develop a simple, effective, bottom-up fusion tracker for re-identity features, named SimpleTrack, and propose a new tracking strategy which can mitigate the loss of detection targets. To show the effectiveness of the proposed method, experiments are carried out using five different state-of-the-art JDE-based methods. The results show that by simply replacing the original association matrix with our EG matrix, we can achieve significant improvements in IDF1, HOTA and IDsw metrics, and increase the tracking speed of these methods by around 20%. In addition, our SimpleTrack has the best data association capability among the JDE-based methods, e.g., 61.6 HOTA and 76.3 IDF1, on the test set of MOT17 with 23 FPS running speed on a single GTX2080Ti GPU.

Multi-Spacecraft Tracking and Data Association Based on Uncertainty Propagation

This paper proposed a novel multi-spacecraft tracking and data association method based on the orbit uncertainty propagation. The proposed method makes full use of the dynamic information and thus the data association performance is enhanced. The proposed method is divided into three portions, i.e., the uncertainty propagation, the data association, and the orbit estimation. The second-order solutions derived for state and measurement prediction, on which to base the optimal association, are set up. The optimal association is solved by the contract network algorithm to reduce the computing cost. Finally, a second-order extended Kalman filter is designed to estimate the orbit of each spacecraft. The proposed method is successfully applied for solving a four-spacecraft tracking problem. Simulations show that all the four targets are well tracked. The method demonstrates close to 100% data association precision. The proposed method is proved to be efficient and effective to solve the multi-spacecraft tracking problem.

Multiple Traffic Target Tracking with Spatial-Temporal Affinity Network.

Traffic target tracking is a core task in intelligent transportation system because it is useful for scene understanding and vehicle autonomous driving. Most state-of-the-art (SOTA) multiple object tracking (MOT) methods adopt a two-step procedure: object detection followed by data association. The object detection has made great progress with the development of deep learning. However, the data association still heavily depends on hand crafted constraints, such as appearance, shape, and motion, which need to be elaborately trained for a special object. In this study, a spatial-temporal encoder-decoder affinity network is proposed for multiple traffic targets tracking, aiming to utilize the power of deep learning to learn a robust spatial-temporal affinity feature of the detections and tracklets for data association. The proposed spatial-temporal affinity network contains a two-stage transformer encoder module to encode the features of the detections and the tracked targets at the image level and the tracklet level, aiming to capture the spatial correlation and temporal history information. Then, a spatial transformer decoder module is designed to compute the association affinity, where the results from the two-stage transformer encoder module are fed back to fully capture and encode the spatial and temporal information from the detections and the tracklets of the tracked targets. Thus, efficient affinity computation can be applied to perform data association in online tracking. To validate the effectiveness of the proposed method, three popular multiple traffic target tracking datasets, KITTI, UA-DETRAC, and VisDrone, are used for evaluation. On the KITTI dataset, the proposed method is compared with 15 SOTA methods and achieves 86.9% multiple object tracking accuracy (MOTA) and 85.71% multiple object tracking precision (MOTP). On the UA-DETRAC dataset, 12 SOTA methods are used to compare with the proposed method, and the proposed method achieves 20.82% MOTA and 35.65% MOTP, respectively. On the VisDrone dataset, the proposed method is compared with 10 SOTA trackers and achieves 40.5% MOTA and 74.1% MOTP, respectively. All those experimental results show that the proposed method is competitive to the state-of-the-art methods by obtaining superior tracking performance.

An end-to-end identity association network based on geometry refinement for multi-object tracking

In multi-target tracking, object interactions and occlusions are two significant factors that affect tracking performance. To settle this, we propose an identity association network (IANet) that integrates the geometry refinement network (GRNet) and the identity verification (IV) module to perform data association and reason the mapping between the detections and tracklets. In our data association process, the object drifts caused by object interactions are suppressed effectively by encoding the direction and velocity of objects to refine the geometric position of tracklets. The tracklets with refined geometric information are further utilized in the IV module to achieve a sufficient encoding of multivariate spatial cues including both appearance and geometry information, which defeats the misleading impacts of interactions and occlusions dramatically in multi-object tracking. The extensive experiments and comparative evaluations have demonstrated that our proposed method can significantly outperform many state-of-the-art methods on benchmarks of 2D MOT2015, MOT16, MOT17, MOT20, and KITTI by using public detection and online settings.

환경 변화에 강인한 영상 항법 기술 동향

Visual navigation technology enables the pose of a robot to be estimated and the surrounding environment to be perceived using a vision sensor mounted on the robot. This technology is essential to autonomous driving systems in unmanned mobile vehicles and has been actively researched in visual odometry (VO) and visual simultaneous localization and mapping (vSLAM). Generally, the vision-based navigation algorithms perform data association and pose estimation under the assumption that the brightness of surrounding environments does not change over time and that the scene obtained from vision sensors is static. However, in realistic industrial sites or urban environments, the brightness of the environment varies, and dynamic objects such as workers and cars are present. These conditions may lead to a decline in the reliability and performance of visual navigation. Research on robust visual navigation under environmental variations, such as illumination changes and dynamic circumstances, has sought to solve this problem. This study proposes a state-of-the-art robust visual navigation system that is robust to illumination changes and dynamic environments. Moreover, our analysis and classification is based on the methodology used in each robust visual navigation.

Sequence-tracker: Multiple object tracking with sequence features in severe occlusion scene

Multiple object tracking is one of the most fundamental tasks in computer vision, and it is still very challenging for real-world applications due to its severe occlusion and motion blur. Most of the existing methods solve these multiple object tracking issues by performing data association based on the deep features of the detections in consecutive frames, which only contain the spatial information of the detected objects. Therefore, the inaccuracy of data association would easily occur, especially in the severe occlusion scenes. In this paper, a novel multiple object tracking model named sequence-tracker (STracker) has been proposed, which combines both the temporal and spatial features to perform data association. We trained a sequence feature extraction network based on video pedestrian re-identification offline, fused the obtained sequence features with the depth features of the previous frame, and then implemented the Hungarian algorithm for data association. Experiments have been carried out to validate the effectiveness of the proposed algorithm and the corresponding results indicates that it can significantly improve the trajectory quality of our dataset in this paper. Remarkably, for the public detector results from MOT official website, the proposed algorithm can achieve up to 57.2% MOTA and 50.9% IDF1 on the MOT17 dataset.

Performance Study of Distance-Weighting Approach with Loopy Sum-Product Algorithm for Multi-Object Tracking in Clutter.

In this paper, we explore the performance of the distance-weighting probabilistic data association (DWPDA) approach in conjunction with the loopy sum-product algorithm (LSPA) for tracking multiple objects in clutter. First, we discuss the problem of data association (DA), which is to infer the correspondence between targets and measurements. DA plays an important role when tracking multiple targets using measurements of uncertain origin. Second, we describe three methods of data association: probabilistic data association (PDA), joint probabilistic data association (JPDA), and LSPA. We then apply these three DA methods for tracking multiple crossing targets in cluttered environments, e.g., radar detection with false alarms and missed detections. We are interested in two performance metrics: tracking accuracy and computation time. LSPA is known to be superior to PDA in terms of the former and to dominate JPDA in terms of the latter. Last, we consider an additional DA method that is a modification of PDA by incorporating a weighting scheme based on distances between position estimates and measurements. This distance-weighting approach, when combined with PDA, has been shown to enhance the tracking accuracy of PDA without significant change in the computation burden. Since PDA constitutes a crucial building block of LSPA, we hypothesize that DWPDA, when integrated with LSPA, would perform better under the two performance metrics above. Contrary to expectations, the distance-weighting approach does not enhance the performance of LSPA, whether in terms of tracking accuracy or computation time.

Appearance Guidance Attention for Multi-Object Tracking

Appearance information is one of the most important matching indicators for multi-object data association. In tracking by detection model, appearance information and detection information are usually integrated in the same sub-network for learning and output. This phenomenon will result in the appearance embedding vectors to be coupled to the network inference method during learning stage. As a result, the appearance embedding vectors contains too much background information and affects the accuracy of data association. Based on the non-end-to-end tracking model, we design an appearance guidance attention module for the appearance extraction branch. This module can effectively strengthen the network's learning of the object visual appearance features and reduce the attention in the learning of background features. Finally, we utilize the appearance embedding vectors that decoupled from the inference method as the input of the back-end tracker and perform data association. The proposed method is tested on the MOT16 and MOT17 datasets. Experiments show that the proposed method provides more high-quality appearance representation information for the back-end tracker and the tracking performance on two datasets is better than other comparison models. At the same time, our model can reach 24.9FPS on a single 1080Ti GPU. The code is available at: https://github.com/JoJoliking/AGM

P2BA: A Privacy-Preserving Protocol With Batch Authentication Against Semi-Trusted RSUs in Vehicular Ad Hoc Networks

Vehicular Ad-hoc Networks (VANETs) supporting the seamless operation of autonomous vehicles introduce various network-connected devices. The widespread devices are engaged in VANETs so that users can enjoy advantageous computing and reliable services. The combination brings in massive real-time message propagation and dissemination, which would be leveraged by the adversaries to perform data association, integration analysis and privacy mining. To address such challenges, existing authentication schemes use n pseudonym certificates for pre-defined k times and try to keep the vehicles anonymous. These schemes require fresh certificates for each authentication process, which cost more communication and storage resources. In this paper, we propose a novel privacy-preserving authentication protocol (P2BA) in bilinear groups, where a registered vehicle signs a traffic-related message and sends it to the nearby Road-side Unit (RSU) together with its blinded certificate. The RSU is able to independently check the message for validity based on a non-interactive zero-knowledge proof protocol. In this way, the computation time has been reduced from <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">O</i> (n) to <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">O</i> (1) while the storage overhead from <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">O</i> (nk) to <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">O</i> (n) compared to anonymous authentication protocols. Moreover, our scheme provides privacy properties such as anonymity and unlinkability. The simulations show that the message authentication can be processed by individual RSUs within 1 ms under the batch-enabled scheme, which outperforms the existing schemes in terms of computation overhead and latency.

Effective Passive Multitarget Localization Using Maximum Likelihood

Localization of multiple targets is a challenging task due to immense complexity regarding data fusion received at the sensors. In this context, we propose an algorithm to solve the problem for an unknown number of emitters without prior knowledge to address the data fusion problem. The proposed technique combines the time difference of arrival (TDOA) and frequency difference of arrival (FDOA) measurement data fusion which further uses the maximum likelihood of the measurements received at each sensor of the surveillance region. The measurement grids of the sensors are used to perform data association. The simulation results show that the proposed algorithm outperforms the multipass grid search and further effectively eliminated the ghost targets created due to the fusion of measurements received at each sensor. Moreover, the proposed algorithm reduces the computational complexity compared to other existing algorithms as it does not use repeated steps for convergence or any biological evolutions. Furthermore, the experimental testing of the proposed technique was executed successfully for tracking multiple targets in different scenarios passively.

An Improved AGV Real-Time Location Model Based on Joint Compatibility Branch and Bound

Automated Guided Vehicle (AGV) indoor autonomous cargo handling and commodity transportation are inseparable from AGV autonomous navigation, and positioning and navigation in an unknown environment are the keys of AGV technology. In this paper, the extended Kalman filter algorithm is used to match the sensor observations with the existing features in the map to determine the accurate positioning of the AGV. This paper proposes an improved joint compatibility branch and bound (JCBB) method to divide the data and then randomly extract part of the data in the divided data set, thereby reducing the data association space; then, the JCBB algorithm is used to perform data association and finally merge the associated data. This method can solve the problem of the increased computational complexity of JCBB when the amount of data to be matched is large to achieve the effect of increasing the correlation speed and not reducing the accuracy rate, thereby ensuring the real-time positioning of the AGV.

Ground and Aerial Collaborative Mapping in Urban Environments

A heterogeneous multi-robot system consisting of Unmanned Ground Vehicles (UGVs) and Unmanned Aerial Vehicles (UAVs) have advantages over a single-robot system in efficiency and flexibility, enabling them to perform a larger range of tasks. To allow heterogeneous platforms to work together in GPS-denied scenarios, it is crucial to build a complete 3D map of the environment. In this letter, a novel method is presented to perform ground and aerial collaborative mapping leveraging visual and range data collected by cameras and 3D LiDAR sensors. In the proposed system, a visual-LiDAR ego-motion estimation module that considers point, line and planar constraints can provide robust odometry information. Thumbnail images representing obstacle outlines are generated and descriptors are extracted using a neural network to help perform data association between separate runs. Map segments and the robot poses are organized together and are updated during a pose graph optimization procedure. The proposed ground-aerial collaborative mapping approach is evaluated on both synthetic and real-world datasets comparing with other methods. Experiment results demonstrate that our method can achieve outstanding mapping results.

Online target tracking via deep convolutional network approach

There is a useful approach for multiple objects tracking easy and efficient that is called simple online and real time tracking(SORT). SORT algorithm performance can be improved by adding visual information. This can reduce the number of identity switches. Because the main framework of the algorithm has a lot of computational complexity, a deep network has been used that is offline on a large data set of trained pedestrians. the focus of this article is on the architecture of this deep network in order to extract more and higher quality visual information that can help the object recognition algorithm. The paper also used a particle filter instead of a Kalman filter to improve data association performance. We tested our proposed method on two standard datasets, MOT16 and MOT17, and compared its performance with other available methods. The results show that the tracking accuracy(52.2) on the MOT17 dataset is improved compared to the existing methods in this field. Experimental evaluation shows that our proposed architecture improves the number of identity switches and ideally tracks goals in complex environments.

Joint Probabilistic Hypergraph Matching Labeled Multi-Bernoulli Filter for Rigid Target Tracking

The likelihood determined by the distance between measurements and predicted states of targets is widely used in many filters for data association. However, if the actual motion model of targets is not coincided with the preset dynamic motion model, this criterion will lead to poor performance when close-space targets are tracked. For rigid target tracking task, the structure of rigid targets can be exploited to improve the data association performance. In this paper, the structure of the rigid target is represented as a hypergraph, and the problem of data association is formulated as a hypergraph matching problem. However, the performance of hypergraph matching degrades if there are missed detections and clutter. To overcome this limitation, we propose a joint probabilistic hypergraph matching labeled multi-Bernoulli (JPHGM-LMB) filter with all undetected cases being considered. In JPHGM-LMB, the likelihood is built based on group structure rather than the distance between predicted states and measurements. Consequently, the probability of each target associated with each measurement (joint association probabilities) can be obtained. Then, the structure information is integrated into LMB filter by revising each single target likelihood with joint association probabilities. However, because all undetected cases is considered, proposed approach is usable in real time only for a limited number of targets. Extensive simulations have demonstrated the significant performance improvement of our proposed method.

A Multidimensional TDOA Association Algorithm for Joint Multitarget Localization and Multisensor Synchronization

This article considers the problem of multitarget localization using time difference of arrival (TDOA) measurements at multiple sensors with misaligned clocks in the presence of data association uncertainty. Sensor synchronization and data association are two essential steps in target localization and tracking to align sensor clocks and to associate measurements from different sensors. In practice, sensor synchronization errors can adversely affect data association performance and vice versa. Although these two processes affect each other, they are usually addressed separately. We propose a novel joint multidimensional association algorithm for multisensor synchronization (JMDA4MS), which performs data association and yields sensor clock offset and target position estimates simultaneously using TDOA measurements. Considering the observability of the unknown parameters, the joint multiframe multidimensional association algorithm is developed as a multiframe extension of the JMDA4MS algorithm. A gating method and a multidimensional plus sequential two-dimensional association approach are utilized to improve the efficiency of the proposed algorithms. The Cramer–Rao lower bound for the proposed maximum likelihood estimator is derived as a performance benchmark. Computer simulations are carried out to evaluate the performance of the proposed algorithms on different scenarios.