Multi-target Tracking Methods Research Articles

Multi-target tracking in underwater multistatic AUV networks with a robust Poisson Multi-Bernoulli filter

The study investigates underwater multi-target tracking (MTT) in a multistatic system using autonomous underwater vehicles (AUVs) as receivers separately from transmitters. The most critical aspect of MTT is the accurate extraction of target tracks from the collected measurements, which can be affected by clutter, missed detections, and port-starboard ambiguities. Moreover, the detection parameters—the target-detection probability and clutter rate—used in MTT methods are typically unknown and change with varying environmental conditions and geometric distributions of the AUV, target, and sound transmitter. In view of these challenges, we propose a robust multi-sensor Poisson Multi-Bernoulli Mixture (R-MS-PMBM) filter immune to unknown detection parameters that treats clutter rates and detection probabilities as continuous random variables modeled by beta and gamma distributions, respectively. The R-MS-PMBM filter is a conjugate prior, and its closed form is derived. We introduce a measurement-driven adaptive birth model to initiate target tracks quickly and efficiently. Furthermore, a suboptimal Gibbs sampler is used to reduce the computational complexity of its practical implementation. The simulation results show that the proposed filter is resilient to unknown detection parameters and can effectively perform in challenging underwater scenarios.

Group target tracking via jointly optimizing group partition and association

This paper considers the problem of indistinguishable group target tracking, where the individual targets within a group are closely spaced and resolvable, move in a coordinated manner, and the group splitting and merging may occur. In such a case, the performance of the commonly used multi-target tracking methods may be degraded. We focus on three aspects of the problem: deal with the group association, group splitting and merging, and derive the state estimates of the individual targets. First, in order to deal with the group partition and the group association globally, we propose a novel group target tracking method within the multiple hypothesis tracking (MHT) framework by extending the target partition and the measurement partition as parts of global hypotheses. An efficient algorithm is given to find the M-best group partition and association simultaneously. Second, we use the undirected graph to represent the interaction between groups and then obtain an identification method of the group splitting and merging. Third, in response to the state estimate of the group target, a group-individual Bayesian filter is designed to estimate the states of the virtual leader and the individual targets. Finally, numerical experiments are presented to illustrate the effectiveness of the proposed method.

Multi-Feature Matching GM-PHD Filter for Radar Multi-Target Tracking.

Multi-target tracking (MTT) is one of the most important functions of radar systems. Traditional multi-target tracking methods based on data association convert multi-target tracking problems into single-target tracking problems. When the number of targets is large, the amount of computation increases exponentially. The Gaussian mixture probability hypothesis density (GM-PHD) filtering based on a random finite set (RFS) provides an effective method to solve multi-target tracking problems without the requirement of explicit data association. However, it is difficult to track targets accurately in real-time with dense clutter and low detection probability. To solve this problem, this paper proposes a multi-feature matching GM-PHD (MFGM-PHD) filter for radar multi-target tracking. Using Doppler and amplitude information contained in radar echo to modify the weights of Gaussian components, the weight of the clutter can be greatly reduced and the target can be distinguished from clutter. Simulations show that the proposed MFGM-PHD filter can improve the accuracy of multi-target tracking as well as the real-time performance with high clutter density and low detection probability.

Multitarget Vehicle Tracking and Motion State Estimation Using a Novel Driving Environment Perception System of Intelligent Vehicles

The multitarget vehicle tracking and motion state estimation are crucial for controlling the host vehicle accurately and preventing collisions. However, current multitarget tracking methods are inconvenient to deal with multivehicle issues due to the dynamically complex driving environment. Driving environment perception systems, as an indispensable component of intelligent vehicles, have the potential to solve this problem from the perspective of image processing. Thus, this study proposes a novel driving environment perception system of intelligent vehicles by using deep learning methods to track multitarget vehicles and estimate their motion states. Firstly, a panoramic segmentation neural network that supports end-to-end training is designed and implemented, which is composed of semantic segmentation and instance segmentation. A depth calculation model of the driving environment is established by adding a depth estimation branch to the feature extraction and fusion module of the panoramic segmentation network. These deep neural networks are trained and tested in the Mapillary Vistas Dataset and the Cityscapes Dataset, and the results showed that these methods performed well with high recognition accuracy. Then, Kalman filtering and Hungarian algorithm are used for the multitarget vehicle tracking and motion state estimation. The effectiveness of this method is tested by a simulation experiment, and results showed that the relative relation (i.e., relative speed and distance) between multiple vehicles can be estimated accurately. The findings of this study can contribute to the development of intelligent vehicles to alert drivers to possible danger, assist drivers’ decision-making, and improve traffic safety.

Multi-target tracking algorithm based on PHD filter against multi-range-false-target jamming

Multi-range-false-target (MRFT) jamming is particularly challenging for tracking radar due to the dense clutter and the repeated multiple false targets. The conventional association-based multi-target tracking (MTT) methods suffer from high computational complexity and limited usage in the presence of MRFT jamming. In order to solve the above problems, an efficient and adaptable probability hypothesis density (PHD) filter is proposed. Based on the gating strategy, the obtained measurements are firstly classified into the generalized newborn target and the existing target measurements. The two categories of measurements are independently used in the decomposed form of the PHD filter. Meanwhile, an amplitude feature is used to suppress the dense clutter. In addition, an MRFT jamming suppression algorithm is introduced to the filter. Target amplitude information and phase quantization information are jointly used to deal with MRFT jamming and the clutter by modifying the particle weights of the generalized newborn targets. Simulations demonstrate the proposed algorithm can obtain superior correct discrimination rate of MRFT, and high-accuracy tracking performance with high computational efficiency in the presence of MRFT jamming in the dense clutter.

Multi-agent informed path planning using the probability hypothesis density

An Informed Path Planning algorithm for multiple agents is presented. It can be used to efficiently utilize available agents when surveying large areas, when total coverage is unattainable. Internally the algorithm has a Probability Hypothesis Density (PHD) representation, inspired by modern multi-target tracking methods, to represent unseen objects. Using the PHD, the expected number of observed objects is optimized. In a sequential manner, each agent maximizes the number of observed new targets, taking into account the probability of undetected objects due to previous agents’ actions and the probability of detection, which yields a scalable algorithm. Algorithm properties are evaluated in simulations, and shown to outperform a greedy base line method. The algorithm is also evaluated by applying it to a sea ice tracking problem, using two datasets collected in the Arctic, with reasonable results. An implementation is provided under an Open Source license.

Exclusive Association Sampling to Improve Bayesian Multi-Target Tracking

Multi-target tracking has been studied for many years, yet it remains a challenging problem, particularly in terms of implementing data association when tracking targets over several time steps. To achieve robustness, probabilistic approaches have been proposed, including Bayesian multi-target tracking methods. However, these approaches involve high calculation costs, which are incompatible with real-time applications. We propose exclusive association sampling (EAS) to improve the efficiency of Bayesian multi-target tracking methods. Although EAS is a simple procedure composed of random sorting and sampling based on the observation probability, it can be employed to increase the efficiency of association candidate generation and the calculation of statistical values. In this study, we proposed two Bayesian multi-target tracking methods based on EAS: stochastic joint probabilistic data association (SJPDA) and Rao-Blackwellized particle filter with EAS (RBPF with EAS). Evaluation of these methods with simulated data shows that integrating EAS into these methods can enhance their speed and accuracy. Moreover, evaluation on open datasets used for pedestrian tracking on camera sequences shows that the proposed methods achieve significantly better performance on some important metrics compared with representative methods.

Research and Application of Multi-Target Tracking Based on GM-PHD Filter

In recent years, multi-target tracking technology based on Gaussian Mixture- Probability Hypothesis Density (GM-PHD) filtering has become a hot field of information fusion research. This article outlines the generation and development of multi-target tracking methods based on GM-PHD filtering, and the principle and implementation method of GM-PHD filtering are explained, and the application status based on GM-PHD filtering is summarized, and the key issues of the development of GM-PHD filtering technology are analyzed.

Tracking Persons-of-Interest via Unsupervised Representation Adaptation

Multi-face tracking in unconstrained videos is a challenging problem as faces of one person often can appear drastically different in multiple shots due to significant variations in scale, pose, expression, illumination, and make-up. Existing multi-target tracking methods often use low-level features which are not sufficiently discriminative for identifying faces with such large appearance variations. In this paper, we tackle this problem by learning discriminative, video-specific face representations using convolutional neural networks (CNNs). Unlike existing CNN-based approaches which are only trained on large-scale face image datasets offline, we automatically generate a large number of training samples using the contextual constraints for a given video, and further adapt the pre-trained face CNN to the characters in the specific videos using discovered training samples. The embedding feature space is fine-tuned so that the Euclidean distance in the space corresponds to the semantic face similarity. To this end, we devise a symmetric triplet loss function which optimizes the network more effectively than the conventional triplet loss. With the learned discriminative features, we apply an EM clustering algorithm to link tracklets across multiple shots to generate the final trajectories. We extensively evaluate the proposed algorithm on two sets of TV sitcoms and YouTube music videos, analyze the contribution of each component, and demonstrate significant performance improvement over existing techniques.

Noise estimation and probability of detection in non-resolved images: Application to space object observation

Charged Couple Device (CCD) technology is widely used in various scientific measurement contexts. CCD equipped cameras have revolutionized astronomy and space-related optical telescope measurements in recent years. They are also used in electroscopic measurements, e.g., in fields such as geology, biology, and medicine. The signal-to-noise ratio and the probability of detection are crucial to design experiments observation setups properly and to employ further mathematical methods for data exploitation such as, e.g. multi-target tracking methods. Previous attempts to correctly characterize the signal-to-noise ratio for star observations are revisited in this work and adapted for the application of near-Earth object observations and high precision measurements, leading to a modified CCD equation. Our formulation proposes a novel distribution of the signal noise that accurately accounts for the truncation noise and the presence of ambiguous pixels. These improvements are employed to derive the probability of detection and the SNR with significant improvements compared to existing formulations when ambiguous pixels are present.

Survey of amplitude‐aided multi‐target tracking methods

An amplitude-aided multi-target tracking (MTT) exploits amplitude as well as spatial features for MTT. Compared to MTT using a spatial feature only, it can maintain its performance even in densely cluttered environment because more accurate association is achieved by using both features for likelihood evaluation between tracks and measurements. Therefore, the goal of this study is to review the state-of-the-art amplitude-aided MTT methods and compare each other extensively. For a fair comparison, a unified MTT framework is developed, and various methods are implemented and compared based on the same framework. On the challenging visual MTT datasets, the implemented methods are evaluated for several aspects such as accuracy, speed, sensitivity, and stability. Moreover, this study presents a summary of the extensive evaluation and guideline to select an appropriate method for amplitude-aided tracking.

Message Passing Algorithms for Scalable Multitarget Tracking

Situation-aware technologies enabled by multitarget tracking will lead to new services and applications in fields such as autonomous driving, indoor localization, robotic networks, and crowd counting. In this tutorial paper, we advocate a recently proposed paradigm for scalable multitarget tracking that is based on message passing or, more concretely, the loopy sum–product algorithm. This approach has advantages regarding estimation accuracy, computational complexity, and implementation flexibility. Most importantly, it provides a highly effective, efficient, and scalable solution to the probabilistic data association problem, a major challenge in multitarget tracking. This fact makes it attractive for emerging applications requiring real-time operation on resource-limited devices. In addition, the message passing approach is intuitively appealing and suited to nonlinear and non-Gaussian models. We present message-passing-based multitarget tracking methods for single-sensor and multiple-sensor scenarios, and for a known and unknown number of targets. The presented methods can cope with clutter, missed detections, and an unknown association between targets and measurements. We also discuss the integration of message-passing-based probabilistic data association into existing multitarget tracking methods. The superior performance, low complexity, and attractive scaling properties of the presented methods are verified numerically. In addition to simulated data, we use measured data captured by two radar stations with overlapping fields-of-view observing a large number of targets simultaneously.

Binary Quadratic Programing for Online Tracking of Hundreds of People in Extremely Crowded Scenes

Multi-object tracking has been studied for decades. However, when it comes to tracking pedestrians in extremely crowded scenes, we are limited to only few works. This is an important problem which gives rise to several challenges. Pre-trained object detectors fail to localize targets in crowded sequences. This consequently limits the use of data-association based multi-target tracking methods which rely on the outcome of an object detector. Additionally, the small apparent target size makes it challenging to extract features to discriminate targets from their surroundings. Finally, the large number of targets greatly increases computational complexity which in turn makes it hard to extend existing multi-target tracking approaches to high-density crowd scenarios. In this paper, we propose a tracker that addresses the aforementioned problems and is capable of tracking hundreds of people efficiently. We formulate online crowd tracking as Binary Quadratic Programing. Our formulation employs target's individual information in the form of appearance and motion as well as contextual cues in the form of neighborhood motion, spatial proximity and grouping, and solves detection and data association simultaneously. In order to solve the proposed quadratic optimization efficiently, where state-of art commercial quadratic programing solvers fail to find the solution in a reasonable amount of time, we propose to use the most recent version of the Modified Frank Wolfe algorithm, which takes advantage of SWAP-steps to speed up the optimization. We show that the proposed formulation can track hundreds of targets efficiently and improves state-of-art results by significant margins on eleven challenging high density crowd sequences.

Posterior Cramer–Rao lower bounds for complicated multi-target tracking with labeled FISST based filters

As science develops, multi-target tracking technique advances towards dealing with complicated scenes, in which target number is time-varying and unknown, detection, measurement source and data association are uncertain. Among the new tracking methods, Mahler׳s Finite Set Statistics (FISST) based multi-target tracking technology naturally suits such complicated scenes. A performance metric with rigid theoretical explanation and clear physical connotation is the cornerstone for further improving multi-target tracking methods on precision and stability. The Posterior Cramer–Rao lower bounds (PCRLB) is widely used for assessing tracking performance. While, for the complicated multi-target tracking mentioned above, existing PCRLBs do not work well. Therefore, we derived multi-target PCRLB (MT-PCRLB) under random finite set frame as well as its iterative expression through analyzing lower bound of the FISST based filters’ performance in complicated multi-target tracking. The derived lower bound is compatible with current labeled FISST based filters. Based on multi-target tracks obtained by labeled FISST based filters and association relations between tracks and measurement set, recursive calculation of MT-PCRLB is realized. Simulation results demonstrate that the proposed methods behave in a manner consistent with our expectations.

Raw Trajectory Rectification via Scene-Free Splitting and Stitching

Trajectories carry rich motion cues and thus have been leveraged to many high-level computer vision tasks. Due to the easy implementation of simple trackers, most previous work on trajectory-based applications utilizes raw tracking outputs without explicitly considering tracking errors. Reliable trajectories are prerequisite for modeling and recognizing high-level behaviors. Therefore, this paper tackles such problems by rectifying raw trajectories, which aims to post-process existing trajectories. Our approach firstly splits them into short tracks, and then infers identity ambiguity to remove unqualified detection responses. At last, short tracks are stitched via maximum bipartite graph matching. This postprocessing is completely scene-free. Results of trajectory rectification and their benefits are both evaluated on two challenging datasets. Results demonstrate that rectified trajectories are conducive to high-level tasks and the proposed approach is also competitive with state-of-the-art multi-target tracking methods.

Minimum Cost Multi-Way Data Association for Optimizing Multitarget Tracking of Interacting Objects.

This paper presents a general formulation for a minimum cost data association problem which associates data features via one-to-one, m-to-one and one-to-n links with minimum total cost of the links. A motivating example is a problem of tracking multiple interacting nanoparticles imaged on video frames, where particles can aggregate into one particle or a particle can be split into multiple particles. Many existing multitarget tracking methods are capable of tracking non-interacting targets or tracking interacting targets of restricted degrees of interactions. The proposed formulation solves a multitarget tracking problem for general degrees of inter-object interactions. The formulation is in the form of a binary integer programming problem. We propose a polynomial time solution approach that can obtain a good relaxation solution of the binary integer programming, so the approach can be applied for multitarget tracking problems of a moderate size (for hundreds of targets over tens of time frames). The resulting solution is always integral and obtains a better duality gap than the simple linear relaxation solution of the corresponding problem. The proposed method was validated through applications to simulated multitarget tracking problems and a real multitarget tracking problem.

A Computationally Efficient Particle Filter for Multitarget Tracking Using an Independence Approximation

Particle filter (PF) based multi-target tracking (MTT) methods suffer from the curse of dimensionality. Existing strategies to combat this assume posterior independence between target states, in order to then sample targets independently, or to perform joint sampling of closely spaced targets only. When many targets are in proximity, these strategies either perform poorly or are too computationally expensive. We make two contributions towards addressing these limitations. Firstly, we advocate an alternative view of the use of posterior independence which emphasizes the statistical effect of assuming posterior independence on the Monte Carlo (MC) approximation of posterior density. Our analysis suggests that assuming posterior independence can provide a better MC approximation of the prior distribution at the next time, and therefore the posterior at the next time, without regard for how sampling is performed. Secondly, we present a computationally efficient, measurement directed, joint sampling method to cope with the target coupling and measurement ambiguity when targets are near each other. Consequently, we develop a PF which employs posterior independence while sampling targets jointly. This PF is applicable to both the traditional thresholded and track-before-detect style pixelized models. Simulation results for a challenging tracking scenario show that the proposed PF substantially outperforms existing approaches.

Joint Multitarget Tracking and Sensor Localization in Collaborative Sensor Networks

Multitarget tracking methods in a sensor network often assume the knowledge of the locations of the sensor nodes. However, in reality sensor nodes are randomly deployed with no prior knowledge about their positions. We propose a method to track an unknown and variable number of targets in the presence of false detections with the positions of sensor nodes estimated jointly to avoid the need of extra localization hardware. Moreover, as low-power consumption is a requirement in sensor networks, a collaborative estimation scheme is presented. For each target in the field under observation there is only a small set of sensor nodes that are active while the others remain in an idle state. The proposed technique is based on a Rao-Blackwellized sequential Monte Carlo (SMC) method that takes advantage of the fact that the state space of the unknown variables is separable. Therefore the problem is divided in two parts. The first one generates samples to estimate the number of targets and solves the association uncertainty between measurements and targets; while the second one is a multiple target tracking problem that can be solved with a modified unscented Kalman filter (MUKF) for each sample. It is shown through simulations that it is possible to track the multiple targets and also get accurate estimates of the unknown locations of the sensor nodes.

Tracking Multiple Visual Targets via Particle-Based Belief Propagation

Multiple-target tracking in video (MTTV) presents a technical challenge in video surveillance applications. In this paper, we formulate the MTTV problem using dynamic Markov network (DMN) techniques. Our model consists of three coupled Markov random fields: 1) a field for the joint state of the multitarget; 2) a binary random process for the existence of each individual target; and 3) a binary random process for the occlusion of each dual adjacent target. To make the inference tractable, we introduce two robust functions that eliminate the two binary processes. We then propose a novel belief propagation (BP) algorithm called particle-based BP and embed it into a Markov chain Monte Carlo approach to obtain the maximum a posteriori estimation in the DMN. With a stratified sampler, we incorporate the information obtained from a learned bottom-up detector (e.g., support-vector-machine-based classifier) and the motion model of the target into the message propagation. Other low-level visual cues such as motion and shape can be easily incorporated into our framework to obtain better tracking results. We have performed extensive experimental verification, and the results suggest that our method is comparable to the state-of-art multitarget tracking methods in all the cases we tested.