Data Association Method Research Articles

A Multiple Target Data Association Method for TDOA Passive Localization

In order to solve the multiple target data association problem in underwater acoustic passive localization system, a new algorithm focus on data association in Time Difference of Arriva l(TDOA) is studied. Firstly, the zero-sum condition is used to select TDOA measurements according to the continuous measurement values within a certain period of time. Then, the selected TDOA measurements are associated with the multiple targets and the correct correlation probability is measured by the correlation degree. The algorithm is used in sea trial. Processing results show that the method proposed in this paper can effectively solve the TDOA measurement association problem, and can locate multiple targets at the same time.

Joint Data Association, Spatiotemporal Bias Compensation and Fusion for Multisensor Multitarget Tracking

Spatiotemporal bias compensation and data association are two essential prerequisites for correct data fusion. Neglect of either in multisensor systems prevents the algorithms from successful application. In this paper, a joint data association, bias compensation and fusion method is presented for multitarget tracking in asynchronous multisensor systems with spatiotemporal biases. The time stamp interval is used to model the state transition, while the difference between the measurement and time stamp interval is regarded as temporal bias and augmented in the state vector as well as spatial bias. The measurements are then formulated as a function of both target states and spatiotemporal biases, where the unknown temporal bias is used to align measurements and target states. Based on the formulation, the joint data association and bias compensation problem is converted into the classical data association and filtering problem in a unified Bayesian framework, without requirement of iterative optimization procedure between data association and bias estimation. The data association is formulated as a 2-D assignment problem and is solved using the generalized Auction algorithm. For each track-measurement pair in association results, the unscented Kalman filter is used to handle the measurement to produce estimates of target states and spatiotemporal bias simultaneously. Furthermore, a bias fusion approach with feedback is presented to fuse the spatiotemporal bias estimates from multiple targets to improve the bias compensation performance and solve the correlation problems therein. Simulation results demonstrate the effectiveness of the proposed method.

Object Clustering With Dirichlet Process Mixture Model for Data Association in Monocular SLAM

Semantic simultaneous localization and mapping (SLAM) with a monocular camera is particularly attractive because of the deployment simplicity and economic availability. Data association problem which assigns unique identities for objects shown in multiple frames plays a fundamental role in semantic SLAM. Previous prevalent methods which mainly focused on associating geometric KeyPoints are no longer suitable. Some naive methods that rely on object distance or 2-D/3-D Intersection over Union are also vulnerable when occlusions happen. In this article, we propose a novel data association method for cuboid landmarks based on Dirichlet process mixture model. By jointly considering object class, position, and size, our method can perform data association robustly. We evaluated our method in simulated datasets, public benchmark KITTI, and on a real robot in an office environment. Experimental results show that our method not only associates cuboids robustly but also achieves SOTA pose estimation accuracy in monocular SLAMs.

Deep Attention Aware Feature Learning for Data Association in Multiple Source Localization

The letter addresses the problem of data association for multiple source localization in a distributed sensor network. To increase the efficiency of localization, we present a learning methodology to generate highly distinctive measurement features for data association. We propose an attention-based graph neural network that exploits the relationship among the phases from the Short Time Fourier transform and the estimated measurements to generate highly descriptive features for each of the detected source signals. The methodology is free from any selective threshold and processes each of the measurements to detect and enrich the similar and distinguishable aspects of the measurements. These aspects are then aggregated in feature representations for each source measurement. The generated features can then be employed with any data association method to segregate the measurements belonging to the same source. It is observed in experimental evaluation that the proposed features employed with state-of-the-art association frameworks yield lower association and localization errors.

Plot Quality Aided Plot-to-Track Association in Dense Clutter for Compact High-Frequency Surface Wave Radar

Due to high false alarm rate and low positioning accuracy of compact high-frequency surface wave radar in moving vessel detection, false plot-to-track association often occurs during moving vessel tracking, thus leading to track fragmentation and false tracking. In order to address this problem, a plot quality evaluation method is proposed and applied to plot-to-track association. Firstly, the differences in spatial correlation of echo spectrum amplitudes and position among moving vessels, clutters, and noise on a range-Doppler map are analyzed, and a plot quality index integrating multi-directional gradient, local variance, and plot position probability is developed. Then, the plots labeled as low quality are removed to reduce both the negative impact of false alarms on plot-to-track association and the computational burden. Eventually, both plot quality index and kinematic parameters are used to calculate the association cost and determine the plot-track pairs during the plot-to-track association procedure. Experimental results with field data demonstrate that the proposed plot quality index can effectively distinguish moving vessel and other plots. Compared with both the nearest neighbor data association method and the joint probability data association method, the association accuracy of the proposed method is greatly improved and, thus, the tracking continuity is enhanced in dense clutter scenarios.

Multi-object tracking by multi-feature fusion to associate all detected boxes

Multi-object tracking (MOT) aims to estimate object trajectory in videos using either a public or private detection approach. Current trackers trained in private mode on diverse datasets make the comparison with cutting-edge methods unfair. Instead of relying exclusively on a single feature, integrating several features is more effective as it helps in data association in various conditions such as occlusion and fast Motion. Driven by this, we propose an improvised data association method by employing public detection tracker built on a CenterNet detector. In contrast to prior arts that merely used high score boxes, we improvise the data association in two stages by considering all the detections. The first stage is enforced by fusing feature embedding, IoU, and Motion features extracted from high-score boxes, followed by unmatched trajectories associated with low-score boxes using IoU similarity in the second stage. Our approach greatly surpasses cutting-edge methods in terms of excellent track quality, fewer ID switches, and high accuracy on MOT challenge datasets. The tracklet interpolation is used as a post-processing approach to fill the gap left by missing detections that improve performance even more.

A Joint Data Association Method for Laser-SLAM of Unmanned Delivery Vehicle Based on Heuristic Search Algorithm

In Laser-SLAM system of unmanned delivery vehicle, there are two kinds of association methods applied to solve the data association problem. Compared with the method of independent association for a single measurement and a single feature, the methods of batch association of measurements and features can provide more accurate association results in the state estimation stage of SLAM. In order to obtain a better association solution, a joint data association method based on heuristic search algorithm (HSA-JDA) is proposed to improve the robustness and accuracy of data association. In HSA-JDA, according to the joint maximum likelihood criterion, the data association problem is evolved into a combinatorial optimization problem of how to determine the optimal association set. A heuristic search algorithm that is an optimized artificial fish swarm algorithm by using adaptive step size and adding fish swarm jumping behavior is applied to search the optimal association solution. Experimental results show that HSA-JDA method ensures high association accuracy and then improves the robustness and accuracy of the whole state estimation results of SLAM. It can be used in the Laser-SLAM system based on Kalman filter to provide reliable association results for improving the accuracy of SLAM estimation results for unmanned delivery vehicle.

Object-Aware SLAM Based on Efficient Quadric Initialization and Joint Data Association

Semantic simultaneous localization and mapping (SLAM) is a popular technology enabling indoor mobile robots to sufficiently perceive and interact with the environment. In this paper, we propose an object-aware semantic SLAM system, which consists of a quadric initialization method, an object-level data association method, and a multi-constraint optimization factor graph. To overcome the limitation of multi-view observations and the requirement of dense point clouds for objects, an efficient quadric initialization method based on object detection and surfel construction is proposed, which can efficiently initialize quadrics within fewer frames and with small viewing angles. The robust object-level joint data association method and the tightly coupled multi-constraint factor graph for quadrics optimization and joint bundle adjustment enable the accurate estimation of constructed quadrics and camera poses. Extensive experiments using public datasets show that the proposed system achieves competitive performance with respect to accuracy and robustness of object quadric estimation and camera localization compared with state-of-the-art methods.

Research on multi-target tracking method based on multi-sensor fusion

Purpose Intelligent and connected vehicle technology is in the ascendant. High-level autonomous driving places more stringent requirements on the accuracy and reliability of environmental perception. Existing research works on multitarget tracking based on multisensor fusion mostly focuses on the vehicle perspective, but limited by the principal defects of the vehicle sensor platform, it is difficult to comprehensively and accurately describe the surrounding environment information. Design/methodology/approach In this paper, a multitarget tracking method based on roadside multisensor fusion is proposed, including a multisensor fusion method based on measurement noise adaptive Kalman filtering, a global nearest neighbor data association method based on adaptive tracking gate, and a Track life cycle management method based on M/N logic rules. Findings Compared with fixed-size tracking gates, the adaptive tracking gates proposed in this paper can comprehensively improve the data association performance in the multitarget tracking process. Compared with single sensor measurement, the proposed method improves the position estimation accuracy by 13.5% and the velocity estimation accuracy by 22.2%. Compared with the control method, the proposed method improves the position estimation accuracy by 23.8% and the velocity estimation accuracy by 8.9%. Originality/value A multisensor fusion method with adaptive Kalman filtering of measurement noise is proposed to realize the adaptive adjustment of measurement noise. A global nearest neighbor data association method based on adaptive tracking gate is proposed to realize the adaptive adjustment of the tracking gate.

Tackling multiple object tracking with complicated motions — Re-designing the integration of motion and appearance

Although numerous data association methods have been proposed for Multiple Object Tracking (MOT), how to integrate different features in the data association remains an open problem. For instance, over-relying on the motion feature may fail to do the necessary data association when object movements are complicated, while only depending on the appearance feature may lead to incorrect association results when intra-frame objects have similar appearances. To make an improved trade-off between the appearance feature and motion feature, we re-designed the integration of motion and appearance. In our online approach, the location and motion of each object are cast to adaptive searching windows, and within searching windows, matching is only related to the similarity of appearance features. In our offline approach, tracklets generated from our online approach are refined by forming the motion feature as spatiotemporal constraints and utilizing the appearance for clustering. We conduct experiments on multiple MOT datasets from diverse perspectives, including variant motion speed, illumination condition, object categories, etc. , and verify that our method can reach robust performance. Moreover, this method also demonstrates its effectiveness by further improving our previous 1 st place solutions in two CVPR 2020 MOT challenges. • Re-designing a novel integration of motion and appearance to tackle MOT with complicated motions • Reaching robust performance under variant motion speeds, illumination conditions, and object categories • Improving two 1st-place solutions of CVPR’20 WAD MOT Challenge and CVPR’20 MOTS challenge

Multi-Spacecraft Tracking and Data Association Based on Uncertainty Propagation

This paper proposed a novel multi-spacecraft tracking and data association method based on the orbit uncertainty propagation. The proposed method makes full use of the dynamic information and thus the data association performance is enhanced. The proposed method is divided into three portions, i.e., the uncertainty propagation, the data association, and the orbit estimation. The second-order solutions derived for state and measurement prediction, on which to base the optimal association, are set up. The optimal association is solved by the contract network algorithm to reduce the computing cost. Finally, a second-order extended Kalman filter is designed to estimate the orbit of each spacecraft. The proposed method is successfully applied for solving a four-spacecraft tracking problem. Simulations show that all the four targets are well tracked. The method demonstrates close to 100% data association precision. The proposed method is proved to be efficient and effective to solve the multi-spacecraft tracking problem.

Roadside HD Map Object Reconstruction Using Monocular Camera

The traditional HD map production based on mapping vehicles equipped with expensive sensors (e.g. LiDAR) struggles to keep the frequently changing HD maps up-to-date, thus calling for great research efforts in the HD map reconstruction using low-cost vision sensors. However, most existing works in this area limit their focus on reconstructing the landmarks on the road surface. This letter widens this limited scope by focusing on the roadside object reconstruction with a monocular camera. We propose a novel object-level reconstruction framework that boasts two advantageous features: 1) a novel 3D model estimation method to directly reconstruct the specific roadside objects in a vectorized format using their prior geometric knowledge, and 2) a novel data association method to solve the complex tracking problem of roadside objects that often yield thin and coherent observations. The proposed framework is evaluated on both typical highway and urban scenarios in public KAIST Urban dataset. The results demonstrate that our algorithm can reconstruct the roadside objects with high accuracy and recall, and outperforms classic Structure from Motion (SfM) and deep-learning-based methods by a large margin.

Hierarchical generator of tracking global hypotheses

The presence of crowds, crossing people, occlusions, and individuals entering and leaving the monitored scenario turns the automatization of Multi-Object Tracking into a demanding task. Due to the difficulties in dealing with those situations, the data association between the incoming observations and their corresponding identities could produce split, merged, and even missed tracks. This article proposes a Hierarchical Generator of Tracking Global Hypotheses (HGTGH) to prevent those errors. In this method, the data association process is divided into hierarchical levels according to multiple factors, such as the duration of tracking on the individuals or the number of frames in a row where they have been missed. A dedicated formulation of the association cost at each level properly combines various affinity metrics. Instead of generating hypotheses for each individual and analyzing them through a batch of future frames, the proposed method immediately generates a global hypothesis that describes the assignment of a whole set of identities on every incoming frame. The generated hypothesis is also able to render new people entering the scene. Thanks to this advantage, the proposed method simultaneously addresses the reduction of identity switches and the problem of starting new tracks. This novel data association method constitutes the core of an online tracking algorithm, which has been evaluated over the MOT17 dataset to demonstrate its effectiveness.

Online multitarget tracking system for autonomous vehicles using discriminative dictionary learning with embedded auto‐encoder algorithm

AbstractWith the advancements in 5G network and mobile edge computing technology, autonomous vehicle technology has gained new development opportunities. Multitarget tracking becomes the research hotspots in autonomous vehicles. Since many factors such as motion blur, partial occlusion, and illumination changes affect the performance of target tracking, the problem of target tracking is still an open topic. In this article, inspired by the strong discriminative ability of dictionary learning, the discriminative dictionary learning with embedded auto‐encoder (DDLEA) algorithm is developed for the multitarget tracking system. The DDLEA algorithm integrates the auto‐encoder into the dictionary learning framework and learns sparse representations while preserving the local structure and discriminative information of data. The learned dictionary model has the strong recognition ability. Further, a multitarget tracking system is developed based on the proposed DDLEA algorithm and the hierarchical data association scheme. Based on the target confidence in the STKSVD model, the hierarchical data association method first uses the Hungarian algorithm to complete the preliminary matching of high confidence targets, and then further tracks the low confidence targets to improve the tracking ability. Experiments are carried out the public MOT 2015 dataset. Compared with several popular multitarget tracking algorithms, the tracking performance of our system is satisfactory.

Multiple acoustic source localization using deep data association

Acoustic source localization is a key component of the various acoustic monitoring systems. In this paper, we address the data association problem occurring in the localization of multiple acoustic sources for a distributed acoustic sensor network. The direction of arrival (DOA) of multiple signals is computed at each node and forwarded to a central system for localization. A challenging problem known as data association occurs because DOAs received from different nodes belonging to the same acoustic source are not known. In a realistic environment, the complexity of the data association method increases with the increase in the number of array nodes and sources. With the increase in complexity, the time required for estimating the location of all the sources also increases. Time-critical scenarios of acoustic monitoring require the location to be estimated as close to real-time as possible. To overcome this problem a data-driven deep learning-based solution is presented to control the complexity of the system. A learning approach that formulates the data association among DOAs as combinatorial optimization is proposed. Firstly, features are extracted along with the DOAs of every detected source at each node in the network. Subsequently, a dynamic multiplex Long Short-term memory(LSTM) network is formulated to estimate the association probabilities of the DOAs from different nodes. The methodology is tested using simulations and real data experiments. Comparative analysis is done to validate that the methodology achieves higher association and localization accuracy with a reduced computational time as compared to present state-of-the-art methods.

Efficient Global MOT Under Minimum-Cost Circulation Framework.

We developed a minimum-cost circulation framework for solving the global data association problem, which plays a key role in the tracking-by-detection paradigm of multi-object tracking (MOT). The global data association problem was extensively studied under the minimum-cost flow framework, which is theoretically attractive as being flexible and globally solvable. However, the high computational burden has been a long-standing obstacle to its wide adoption in practice. While enjoying the same theoretical advantages and maintaining the same optimal solution as the minimum-cost flow framework, our new framework has a better theoretical complexity bound and leads to orders of practical efficiency improvement. This new framework is motivated by the observation that minimum-cost flow only partially models the data association problem and it must be accompanied by an additional and time-consuming searching scheme to determine the optimal object number. By employing a minimum-cost circulation framework, we eliminate the searching step and naturally integrate the number of objects into the optimization problem. By exploring the special property of the associated graph, that is, an overwhelming majority of the vertices are with unit capacity, we designed an implementation of the framework and proved it has the best theoretical computational complexity so far for the global data association problem. We evaluated our method with 40 experiments on five MOT benchmark datasets. Our method was always the most efficient in every single experiment and averagely 53 to 1,192 times faster than the three state-of-the-art methods. When our method served as a sub-module for global data association methods utilizing higher-order constraints, similar running time improvement was attained. We further illustrated through several case studies how the improved computational efficiency enables more sophisticated tracking models and yields better tracking accuracy. We made the source code publicly available on GitHub with both Python and MATLAB interfaces.

Enabling Plug-and-Play and Crowdsourcing SLAM in Wireless Communication Systems

Simultaneous localization and mapping (SLAM) during communication is emerging. This technology promises to provide information on propagation environments and transceivers’ location, thus creating several new services and applications for the Internet of Things and environment-aware communication. Using crowdsourcing data collected by multiple agents appears to be much potential for enhancing SLAM performance. However, the measurement uncertainties in practice and biased estimations from multiple agents may result in serious errors. This study develops a robust SLAM method with measurement plug-and-play and crowdsourcing mechanisms to address the above problems. First, we divide measurements into different categories according to their unknown biases and realize a measurement plug-and-play mechanism by extending the classic belief propagation (BP)-based SLAM method. The proposed mechanism can obtain the time-varying agent location, radio features, and corresponding measurement biases (such as clock bias, orientation bias, and received signal strength model parameters), with high accuracy and robustness in challenging scenarios without any prior information on anchors and agents. Next, we establish a probabilistic crowdsourcing-based SLAM mechanism, in which multiple agents cooperate to construct and refine the radio map in a decentralized manner. Our study presents the first BP-based crowdsourcing that resolves the “double count” and “data reliability” problems through the flexible application of probabilistic data association methods. Numerical results reveal that the crowdsourcing mechanism can further improve the accuracy of the mapping result, which, in turn, ensures the decimeter-level localization accuracy of each agent in a challenging propagation environment.

STN-Track: Multiobject Tracking of Unmanned Aerial Vehicles by Swin Transformer Neck and New Data Association Method

The gradual development of remote sensing object tracking technology based on unmanned aerial vehicles (UAV) videos has become one of the main research directions in the field of visual tracking. However, due to characteristics of the UAV platform, typical visual tracking algorithms currently applied to natural scenes cannot be used directly. Small-scale objects in UAV remote sensing videos are difficult to detect and have the problem of tracking identity switching. In order to solve these problems, we designed the Swin transformer neck YOLOX (STN-YOLOX) object detection algorithm as the detection module, and the G-Byte data association method as the tracking module. We then combined the two into a new multi-object tracking algorithm named STN-Track. We used STN-Track to conduct experiments on the UAVDT and VisDrone MOT datasets. The experimental results show that compared with the current state-of-the-art (SOTA) methods, our STN-Track has improved detection and tracking accuracy of small-scale objects and greatly improved identification capabilities for object tracking. Compared with the SOTA ByteTrack algorithm, MOTA of STN-Track can be improved by up to 3.2%, AP <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">S</sub> can be improved by up to 4.4%, MT can be improved by up to 6.8%, and IDSW can be reduced by up to 28.0%.

A Detachable and Expansible Multisensor Data Fusion Model for Perception in Level 3 Autonomous Driving System

Various sensors are adopted by autonomous driving systems to perceive objects and surroundings. Thus, the multi-sensor data fusion techniques become essential to combine different sensors’ advantages for better perception performance. However, the current multi-sensor data fusion techniques suffer from the high cost of computation resources, low expansibility for more diverse sensors, and insufficient systematic consideration for modeling. This paper first constructs a detachable and expansible multi-sensor data fusion model based on three main modules: front fusion, global fusion, and synthesizer, where the methods for flexible association gating and virtual targets have been designed. The model can be disassembled and configured for different trim levels of vehicles and is easily expansible for adding more heterogeneous sensors. Next, the presented multi-sensor data fusion model is compared with the cheap Joint Probabilistic Data Association (C-JPDA) method. The comparison shows the superior accuracy of the designed model on false association and effective narrowing of the variance of object detection. Finally, the presented multi-sensor data fusion model is integrated into an embedded system and experimented on urban roads and highways with the engaged Level 3 autonomous driving function. The experiment results indicate that the proposed model has excellent sensor data fusion performance and provides accurate and timely object information in the Level 3 autonomous driving system.

A Moving Ship Detection and Tracking Method Based on Optical Remote Sensing Images from the Geostationary Satellite.

Geostationary optical remote sensing satellites, such as the GF-4, have a high temporal resolution and wide coverage, which enables the continuous tracking and observation of ship targets over a large range. However, the ship targets in the images are usually small and dim and the images are easily affected by clouds, islands and other factors, which make it difficult to detect the ship targets. This paper proposes a new method for detecting ships moving on the sea surface using GF-4 satellite images. First, the adaptive nonlinear gray stretch (ANGS) method was used to enhance the image and highlight small and dim ship targets. Second, a multi-scale dual-neighbor difference contrast measure (MDDCM) method was designed to enable detection of the position of the candidate ship target. The shape characteristics of each candidate area were analyzed to remove false ship targets. Finally, the joint probability data association (JPDA) method was used for multi-frame data association and tracking. Our results suggest that the proposed method can effectively detect and track moving ship targets in GF-4 satellite optical remote sensing images, with better detection performance than other classical methods.