Explicit Data Association Research Articles

Radar nonlinear multi-target tracking method with parallel PHD filter

Since probability hypothesis density (PHD) filters do not need explicit data association, they have recently been widely used in radar multi-target tracking (MTT). However, in existing PHD filters, sampling times are generally considered the same for all targets. Due to the limitation of antenna beam width in radar applications, the same sampling time for all targets will lead to a mismatch between the predicted data and measurement data, reducing the accuracy of radar MTT. In order to eliminate the estimation error with less computational cost, a radar nonlinear multi-target tracking method with a parallel PHD filter is proposed in this article. The measurement area is divided into several subspaces according to the beam width of the radar antenna, and the PHD of all subspaces is calculated in parallel. Then, multi-feature information in radar echo assists tracking and improves real-time performance. Experimental results in various scenarios illustrate that the proposed method can eliminate the estimation errors introduced by sampling time diversity at the cost of less computation cost, especially in cluttered environments.

Looking Beyond Two Frames: End-to-End Multi-Object Tracking Using Spatial and Temporal Transformers.

Tracking a time-varying indefinite number of objects in a video sequence over time remains a challenge despite recent advances in the field. Most existing approaches are not able to properly handle multi-object tracking challenges such as occlusion, in part because they ignore long-term temporal information. To address these shortcomings, we present MO3TR: a truly end-to-end Transformer-based online multi-object tracking (MOT) framework that learns to handle occlusions, track initiation and termination without the need for an explicit data association module or any heuristics. MO3TR encodes object interactions into long-term temporal embeddings using a combination of spatial and temporal Transformers, and recursively uses the information jointly with the input data to estimate the states of all tracked objects over time. The spatial attention mechanism enables our framework to learn implicit representations between all the objects and the objects to the measurements, while the temporal attention mechanism focuses on specific parts of past information, allowing our approach to resolve occlusions over multiple frames. Our experiments demonstrate the potential of this new approach, achieving results on par with or better than the current state-of-the-art on multiple MOT metrics for several popular multi-object tracking benchmarks.

Multi‐Event‐Camera Depth Estimation and Outlier Rejection by Refocused Events Fusion

Event cameras are bio‐inspired sensors that offer advantages over traditional cameras. They operate asynchronously, sampling the scene at microsecond resolution and producing a stream of brightness changes. This unconventional output has sparked novel computer vision methods to unlock the camera's potential. Here, the problem of event‐based stereo 3D reconstruction for SLAM is considered. Most event‐based stereo methods attempt to exploit the high temporal resolution of the camera and the simultaneity of events across cameras to establish matches and estimate depth. By contrast, this work investigates how to estimate depth without explicit data association by fusing disparity space images (DSIs) originated in efficient monocular methods. Fusion theory is developed and applied to design multi‐camera 3D reconstruction algorithms that produce state‐of‐the‐art results, as confirmed by comparisons with four baseline methods and tests on a variety of available datasets.

Photometric Visual-Inertial Navigation With Uncertainty-Aware Ensembles

In this article, we propose a visual-inertial navigation system that directly minimizes a photometric error without an explicit data-association. We focus on the photometric error parametrized by pose and structure parameters that is highly nonconvex due to the nonlinearity of image intensity. The key idea is to introduce an <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">optimal intensity gradient</i> that accounts for a projective uncertainty of a pixel. Ensembles sampled from the state uncertainty contribute to the proposed gradient and yield a correct update direction even in a bad initialization point. We present two sets of experiments to demonstrate the strengths of our framework. First, a thorough Monte Carlo simulation in a virtual trajectory is designed to reveal robustness to large initial uncertainty. Second, we show that the proposed framework can achieve superior estimation accuracy with efficient computation time over state-of-the-art visual-inertial fusion methods in a real-world UAV flight test, where most scenes are composed of a featureless floor.

Multi-Feature Matching GM-PHD Filter for Radar Multi-Target Tracking.

Multi-target tracking (MTT) is one of the most important functions of radar systems. Traditional multi-target tracking methods based on data association convert multi-target tracking problems into single-target tracking problems. When the number of targets is large, the amount of computation increases exponentially. The Gaussian mixture probability hypothesis density (GM-PHD) filtering based on a random finite set (RFS) provides an effective method to solve multi-target tracking problems without the requirement of explicit data association. However, it is difficult to track targets accurately in real-time with dense clutter and low detection probability. To solve this problem, this paper proposes a multi-feature matching GM-PHD (MFGM-PHD) filter for radar multi-target tracking. Using Doppler and amplitude information contained in radar echo to modify the weights of Gaussian components, the weight of the clutter can be greatly reduced and the target can be distinguished from clutter. Simulations show that the proposed MFGM-PHD filter can improve the accuracy of multi-target tracking as well as the real-time performance with high clutter density and low detection probability.

Good Feature Matching: Toward Accurate, Robust VO/VSLAM With Low Latency

Analysis of state-of-the-art VO/VSLAM system exposes a gap in balancing performance (accuracy & robustness) and efficiency (latency). Feature-based systems exhibit good performance, yet have higher latency due to explicit data association; direct & semidirect systems have lower latency, but are inapplicable in some target scenarios or exhibit lower accuracy than feature-based ones. This paper aims to fill the performance-efficiency gap with an enhancement applied to feature-based VSLAM. We present good feature matching, an active map-to-frame feature matching method. Feature matching effort is tied to submatrix selection, which has combinatorial time complexity and requires choosing a scoring metric. Via simulation, the Max-logDet matrix revealing metric is shown to perform best. For real-time applicability, the combination of deterministic selection and randomized acceleration is studied. The proposed algorithm is integrated into monocular & stereo feature-based VSLAM systems. Extensive evaluations on multiple benchmarks and compute hardware quantify the latency reduction and the accuracy & robustness preservation.

Generic Tracking and Probabilistic Prediction Framework and Its Application in Autonomous Driving

Accurately tracking and predicting behaviors of surrounding objects are key prerequisites for intelligent systems such as autonomous vehicles to achieve safe and high-quality decision making and motion planning. However, there still remain challenges for multi-target tracking due to object number fluctuation and occlusion. To overcome these challenges, we propose a constrained mixture sequential Monte Carlo (CMSMC) method in which a mixture representation is incorporated in the estimated posterior distribution to maintain multi-modality. Multiple targets can be tracked simultaneously within a unified framework without explicit data association between observations and tracking targets. The framework can incorporate an arbitrary prediction model as the implicit proposal distribution of the CMSMC method. An example in this paper is a learning-based model for hierarchical time-series prediction, which consists of a behavior recognition module and a state evolution module. Both modules in the proposed model are generic and flexible so as to be applied to a class of time-series prediction problems where behaviors can be separated into different levels. Finally, the proposed framework is applied to a numerical case study as well as a task of on-road vehicle tracking, behavior recognition, and prediction in highway scenarios. Instead of only focusing on forecasting trajectory of a single entity, we jointly predict continuous motions for interactive entities simultaneously. The proposed approaches are evaluated from multiple aspects, which demonstrate great potential for intelligent vehicular systems and traffic surveillance systems.

Time-Matching Random Finite Set-Based Filter for Radar Multi-Target Tracking.

The random finite set (RFS) approach provides an elegant Bayesian formulation of the multi-target tracking (MTT) problem without the requirement of explicit data association. In order to improve the performance of the RFS-based filter in radar MTT applications, this paper proposes a time-matching Bayesian filtering framework to deal with the problem caused by the diversity of target sampling times. Based on this framework, we develop a time-matching joint generalized labeled multi-Bernoulli filter and a time-matching probability hypothesis density filter. Simulations are performed by their Gaussian mixture implementations. The results show that the proposed approach can improve the accuracy of target state estimation, as well as the robustness.

EMVS: Event-Based Multi-View Stereo—3D Reconstruction with an Event Camera in Real-Time

Event cameras are bio-inspired vision sensors that output pixel-level brightness changes instead of standard intensity frames. They offer significant advantages over standard cameras, namely a very high dynamic range, no motion blur, and a latency in the order of microseconds. However, because the output is composed of a sequence of asynchronous events rather than actual intensity images, traditional vision algorithms cannot be applied, so that a paradigm shift is needed. We introduce the problem of event-based multi-view stereo (EMVS) for event cameras and propose a solution to it. Unlike traditional MVS methods, which address the problem of estimating dense 3D structure from a set of known viewpoints, EMVS estimates semi-dense 3D structure from an event camera with known trajectory. Our EMVS solution elegantly exploits two inherent properties of an event camera: (1) its ability to respond to scene edges—which naturally provide semi-dense geometric information without any pre-processing operation—and (2) the fact that it provides continuous measurements as the sensor moves. Despite its simplicity (it can be implemented in a few lines of code), our algorithm is able to produce accurate, semi-dense depth maps, without requiring any explicit data association or intensity estimation. We successfully validate our method on both synthetic and real data. Our method is computationally very efficient and runs in real-time on a CPU.

Robust Tracking of Soccer Robots Using Random Finite Sets

Maintaining a good estimation of the other robots’ positions is crucial in soccer robotics, as in most multirobot systems applications. Classical approaches use a vector representation of the robots’ positions and Bayesian filters to propagate them over time. However, these approaches suffer from the data association problem. To tackle this issue, this article presents a new methodology for the robust tracking of robots based on the Random Finite Sets framework, which doesn’t require any explicit data association. Moreover, the proposed methodology is able to integrate information shared by teammate robots, their positions, and their estimations of the other robots’ positions. The robots’ tracking is based on the use of a GM-PHD filter, where the estimations of the robots’ positions and observations are represented using mixture of Gaussians, but instead of associating a robot’s hypothesis or an observation to a given Gaussian, the weight of each Gaussian maintains an estimation of the number of robots that it represents. The methodology is validated in several soccer matches and compared with a classical multihypothesis EKF tracking methodology. The proposed method is able to reduce the errors of the estimated robots’ positions in about 35 percent.

Iterative update correction and multi-frame state extraction based probability hypothesis density filter

Probability hypothesis density (PHD) filter is an effective means to track multiple targets in that it avoids explicit data association between measurements and tracks. The Gaussian mixture (GM) implementation of the PHD filter is a closed-form solution to the PHD filter for linear Gaussian model. However, the Gaussian mixture PHD filter suffers from filtering performance degradation problem in multi-target tracking scenarios with low probability of detection, especially when it comes to tracking nearby targets in the imperfect probability of detection conditions. Aiming at the problem, a robust Gaussian mixture PHD algorithm for tracking multiple targets is proposed. First, a novel nearby target tracking method is introduced to reallocate the possible incorrect weights of the nearby targets. Then, a novel target state estimation scheme, making full use of the multiple previous weights of the targets, is adopted to extract the estimates of the target states. Simulation experiments have demonstrated that the proposed approach can achieve better performance in terms of target states and their number than the other related algorithms when tracking multiple nearby targets in the low probability of detection scenarios.

Probability hypothesis density filter with imperfect detection probability for multi-target tracking

Probability hypothesis density (PHD) filter is an effective means to track multiple targets in that it avoids explicit data association between measurements and targets. However, the PHD filter cannot be directly applied to track targets in imperfect detection probability conditions. Otherwise, the performance of almost all the PHD-based filters significantly decreases. Aiming at improving the estimate accuracy as for target states and their number, a multi-target tracking algorithm using the probability hypothesis density filter is proposed, where a novel multi-frame scheme is introduced to cope with estimates of undetected targets caused by the imperfect detection probability. According to the weights of targets at different time steps, both the previous weight array and state extraction identifier of individual targets are constructed. When the targets are undetected at some times, the states of the undetected targets are extracted based on previous weight arrays and state extraction identifiers of correlative targets. Simulation results show that the proposed algorithm effectively improves the performance of the existing relevant PHD-based filters in imperfect detection of probability scenarios.

RGB-D SLAM Combining Visual Odometry and Extended Information Filter.

In this paper, we present a novel RGB-D SLAM system based on visual odometry and an extended information filter, which does not require any other sensors or odometry. In contrast to the graph optimization approaches, this is more suitable for online applications. A visual dead reckoning algorithm based on visual residuals is devised, which is used to estimate motion control input. In addition, we use a novel descriptor called binary robust appearance and normals descriptor (BRAND) to extract features from the RGB-D frame and use them as landmarks. Furthermore, considering both the 3D positions and the BRAND descriptors of the landmarks, our observation model avoids explicit data association between the observations and the map by marginalizing the observation likelihood over all possible associations. Experimental validation is provided, which compares the proposed RGB-D SLAM algorithm with just RGB-D visual odometry and a graph-based RGB-D SLAM algorithm using the publicly-available RGB-D dataset. The results of the experiments demonstrate that our system is quicker than the graph-based RGB-D SLAM algorithm.

Bayesian approach to joint super-resolution and trajectory estimation for midcourse closely spaced objects via space-based infrared sensor

The problem of super-resolution and tracking for midcourse closely spaced objects (CSO) is examined using a space-based infrared sensor. Within a short time window, the midcourse CSO trajectories on the focal plane can be modeled as following a straight line with a constant velocity. Thus, the object's initial state (location and velocity on the focal plane) exclusively corresponds to its trajectory on the focal plane. Thereupon, the objects number, intensities and initial states, as well as the sensor noise variances, are considered random variables, and a Bayesian model is proposed which is utilized to define a posterior distribution on the joint parameter space. To maximize this distribution, reversible jump Markov chain Monte Carlo algorithm is adopted to perform the Bayesian computation. The proposed approach simultaneously used the multiple time-consecutive frame data to estimate model parameters. Compared with the single-frame method, it not only gains the super-resolution capability but also can directly estimate focal plane trajectories without using explicit data association techniques. Results show that the performance (estimation precision of objects number, focal plane locations, intensities and ballistic trajectories for the CSO, together with the computation load) of the proposed approach outperforms the conventional single-frame and multiframe approaches.

Real-time Visual Tracking of Multiple Targets Using Bootstrap Importance Sampling

Ambiguity is the major difficulty in multi-object tracking problem due to the interactions of multiple targets (partial or complete occlusion). This ambiguity can be resolved by Markov random field (MRF) without explicit data association. However, the computational cost of general probabilistic inference algorithms of MRF is expensive. This paper presents a novel approach to this problem. Firstly, a new recursive Bayesian estimation framework, bootstrap importance sampling particle filter (BIS-PF), is devised, which has a “distributed-central-distributed” structure. The core of this framework is a suboptimal importance density which uses the observation at present time. So, it does not suffer from the curse of dimensionality. Secondly, a new Monte Carlo strategy is proposed, which uses bootstrap sampling to generate lowcost and high-quality samples, maintains multi-modality and decreases the number of likelihood computations. Thirdly, a new marginalization technology is presented, which uses an auxiliary variable sampler to obtain marginal samples and bootstrap based histogram for density estimation. The experiments show that the proposed method can track multiple targets in real-time, handle the complex interaction and maintain multi-modalities even the objects disappear.

Stereo vision specific models for particle filter-based SLAM

This work addresses the SLAM problem for stereo vision systems under the unified formulation of particle filter methods. In contrast to most existing approaches to visual SLAM, the present method does not rely on restrictive smooth camera motion models, but on computing incremental 6-DoF pose differences from the image flow through a probabilistic visual odometry method. Moreover, our observation model, which considers both the 3D positions and the SIFT descriptors of the landmarks, avoids explicit data association between the observations and the map by marginalizing the observation likelihood over all the possible associations. We have experimentally validated our research with two experiments in indoor scenarios.

Low computation and low latency algorithms for distributed sensor network initialization

In this paper, we show how an underlying system’s state vector distribution can be determined in a distributed heterogeneous sensor network with reduced subspace observability at the individual nodes. The presented algorithm can generate the initial state vector distribution for networks with a variety of sensor types as long as the collective set of measurements from all the sensors provides full state observability. Hence the network, as a whole, can be capable of observing the target state vector even if the individual nodes are not capable of observing it locally. Initialization is accomplished through a novel distributed implementation of the particle filter that involves serial particle proposal and weighting strategies that can be accomplished without sharing raw data between individual nodes. If multiple events of interest occur, their individual states can be initialized simultaneously without requiring explicit data association across nodes. The resulting distributions can be used to initialize a variety of distributed joint tracking algorithms. We present two variants of our initialization algorithm: a low complexity implementation and a low latency implementation. To demonstrate the effectiveness of our algorithms we provide simulation results for initializing the states of multiple maneuvering targets in smart sensor networks consisting of acoustic and radar sensors.

Convergence results for the particle PHD filter

Bayesian single-target tracking techniques can be extended to a multiple-target environment by viewing the multiple-target state as a random finite set, but evaluating the multiple-target posterior distribution is currently computationally intractable for real-time applications. A practical alternative to the optimal Bayes multitarget filter is the probability hypothesis density (PHD) filter, which propagates the first-order moment of the multitarget posterior instead of the posterior distribution itself. It has been shown that the PHD is the best-fit approximation of the multitarget posterior in an information-theoretic sense. The method avoids the need for explicit data association, as the target states are viewed as a single global target state, and the identities of the targets are not part of the tracking framework. Sequential Monte Carlo approximations of the PHD using particle filter techniques have been implemented, showing the potential of this technique for real-time tracking applications. This paper presents mathematical proofs of convergence for the particle filtering algorithm and gives bounds for the mean-square error