Multi-target Tracking Problem Research Articles

Multi-Target Tracking Using Hierarchical Convolutional Features and Motion Cues

In this paper, the problem of multi-target tracking with single camera in complex scenes is addressed. A new approach is proposed for multi-target tracking problem that learns from hierarchy of convolution features. First fast Region-based Convolutional Neutral Networks is trained to detect pedestrian in each frame. Then cooperate it with correlation filter tracker which learns target’s appearance from pretrained convolutional neural networks. Correlation filter learns from middle and last convolutional layers to enhances targets localization. However correlation filters fail in case of targets full occlusion. This lead to separated tracklets (mini-trajectories) problem. So a post processing step is added to link separated tracklets with minimum-cost network flow. A cost function is used, that depends on motion cues in associating short tracklets. Experimental results on MOT2015 benchmark show that the proposed approach produce comparable result against state-of-the-art approaches. It shows an increase 4.5 % in multiple object tracking accuracy. Also mostly tracked targets is 12.9% vs 7.5% against state-ofthe- art minimum-cost network flow tracker.

An Efficient Implementation of Track‐Oriented Multiple Hypothesis Tracker Using Graphical Model Approaches

The multiple hypothesis tracker (MHT) is currently the preferred method for addressing data association problem in multitarget tracking (MTT) application. MHT seeks the most likely global hypothesis by enumerating all possible associations over time, which is equal to calculating maximum a posteriori (MAP) estimate over the report data. Despite being a well‐studied method, MHT remains challenging mostly because of the computational complexity of data association. In this paper, we describe an efficient method for solving the data association problem using graphical model approaches. The proposed method uses the graph representation to model the global hypothesis formation and subsequently applies an efficient message passing algorithm to obtain the MAP solution. Specifically, the graph representation of data association problem is formulated as a maximum weight independent set problem (MWISP), which translates the best global hypothesis formation into finding the maximum weight independent set on the graph. Then, a max‐product belief propagation (MPBP) inference algorithm is applied to seek the most likely global hypotheses with the purpose of avoiding a brute force hypothesis enumeration procedure. The simulation results show that the proposed MPBP‐MHT method can achieve better tracking performance than other algorithms in challenging tracking situations.

Adaptive Meshfree Backward SDE Filter

An adaptive meshfree approach is proposed to solve the nonlinear filtering problem based on forward backward stochastic differential equations. The algorithm relies on the fact that the solution of the forward backward stochastic differential equations is the unnormalized filtering density as required in the nonlinear filtering problem. Adaptive space points are constructed in a stochastic manner to improve the efficiency of the algorithm. We also introduce a Markov chain Monte Carlo resampling method to address the degeneracy problem of the adaptive space points. Numerical experiments are carried out against the auxiliary particle filter method in the rugged double well potential problem as well as the multitarget tracking problem to demonstrate the superior performance of our algorithm.

Tracking multiple targets based on min-cost network flows with detection in RGB-D data

Visual multi-target tracking technology is a challenging problem in computer vision. This study proposes a novel approach for multi-target tracking based on min-cost network flows in RGB-D data with tracking-by-detection scheme. Firstly, the moving objects are detected by fusing RGB information and depth information. Then, we formulate the multi-target tracking problem as a maximum a posteriori (MAP) estimation problem with specific constraints, and the problem is converted into a cost-flow network. Finally, using a min-cost flow algorithm, we can obtain the tracking results. Extensive experimental results show that the proposed algorithm greatly improves the robustness and accuracy and outperforms the state-of-the-art significantly.

Learning Optimal Parameters for Multi-target Tracking with Contextual Interactions

We describe an end-to-end framework for learning parameters of min-cost flow multi-target tracking problem with quadratic trajectory interactions including suppression of overlapping tracks and contextual cues about cooccurrence of different objects. Our approach utilizes structured prediction with a tracking-specific loss function to learn the complete set of model parameters. In this learning framework, we evaluate two different approaches to finding an optimal set of tracks under a quadratic model objective, one based on an LP relaxation and the other based on novel greedy variants of dynamic programming that handle pairwise interactions. We find the greedy algorithms achieve almost equivalent accuracy to the LP relaxation while being up to 10x faster than a commercial LP solver. We evaluate trained models on three challenging benchmarks. Surprisingly, we find that with proper parameter learning, our simple data association model without explicit appearance/motion reasoning is able to achieve comparable or better accuracy than many state-of-the-art methods that use far more complex motion features or appearance affinity metric learning.

Multi‐target trackers using cubature Kalman filter for Doppler radar tracking in clutter

A major feature of the Gaussian mixture probability hypothesis density (GM-PHD) filter is that it does not require any measurement-to-track association to complete its update step. This, according to the authors, should constitute significant advantage over conventional data-association based methods, especially in presence of high false-alarm rate, frequent miss-detections and targets in close proximity. To test this hypothesis, a multi-target tracking (MTT) problem using Doppler radar is considered, where the performance of GM-PHD algorithm is compared against six data-association based MTT filters in aforementioned adverse tracking conditions. To handle the non-linearity due to Doppler, cubature Kalman filter (CKF) is used in the framework of all MTT algorithms. Detailed mathematical framework of a new non-linear variant of GM-PHD using CKF has been derived using fundamental principles of non-linear Bayesian filtering. It is named as CK-GM-PHD. CK-GM-PHD is formulated using approximated Gaussian mixture assumption and follows track-oriented approach. Cubature integration method is used to numerically compute mean and covariance of components in the Gaussian mixture. Simulation results support the hypothesis by revealing substantial performance improvement of CK-GM-PHD algorithm over conventional data-association based approaches while tested in moderate to heavy clutter rate with lower detection probability and closely spaced target scenarios.

Cardinality Balanced Multi-Target Multi-Bernoulli Filter with Error Compensation.

The cardinality balanced multi-target multi-Bernoulli (CBMeMBer) filter developed recently has been proved an effective multi-target tracking (MTT) algorithm based on the random finite set (RFS) theory, and it can jointly estimate the number of targets and their states from a sequence of sensor measurement sets. However, because of the existence of systematic errors in sensor measurements, the CBMeMBer filter can easily produce different levels of performance degradation. In this paper, an extended CBMeMBer filter, in which the joint probability density function of target state and systematic error is recursively estimated, is proposed to address the MTT problem based on the sensor measurements with systematic errors. In addition, an analytic implementation of the extended CBMeMBer filter is also presented for linear Gaussian models. Simulation results confirm that the proposed algorithm can track multiple targets with better performance.

Nonlinear Gaussian mixture phd filter with an H∞ criterion

This paper is devoted to the problem of multitarget tracking with nonlinear models and uncertain noise statistics in the framework of random finite sets. Based on the linear approximation strategy, a robust closed-form solution to the probability hypothesis density recursion is proposed in terms of an H∞ norm minimization criterion. This idea is extended to develop another analytic implementation using the unscented transform technique. Simulations using the proposed approach are also presented.

Group Sparsity Based Multi-Target Tracking in Passive Multi-Static Radar Systems Using Doppler-Only Measurements

In this paper, we consider the problem of tracking multiple targets in a passive multi-static radar system using Doppler-only measurements. The number of targets is assumed unknown and time-varying. The Doppler measurements are subject to additive noise, clutter, and missed detections. Doppler-only measurements from a single sensor provide incomplete information about the target state, commonly referred to as single-sensor unobservability. In a passive multi-static radar system, the availability of multiple bistatic links naturally lends itself to the fusion of measurements from spatially distributed sensors. However, data fusion emerges as a computationally intensive problem in multi-sensor multi-target tracking algorithms. We propose a two-step sequential approach to solve the underlying problem. We first cast the underlying problem as a group sparse problem in a discretized position-velocity space. A group sparsity based algorithm is applied to simultaneously exploit the multi-static Doppler frequency measurements to directly obtain the instantaneous target state estimates in the Cartesian coordinate system. These estimates are then fed as inputs to the linear Gaussian mixture probability hypothesis density (GMPHD) filter, which removes the false measurements, compensates for missed detections and reduces the localization error. The optimal sub-pattern assignment metric, which jointly comprises a weighted contribution of cardinality error and localization error, is used to evaluate the performance of the proposed method. Simulation results show that the proposed method successfully handles the multi-target tracking problem and outperforms the existing random receiver selection based multi-sensor implementation of the GMPHD filter.

Unscented-Kalman-Filter-Based Multitarget Tracking Algorithms for Airborne Surveillance Application

This paper addresses the problem of multitarget tracking for aerial surveillance application. An infrared search and track sensor, mounted on an unmanned platform, is deployed for tracking and provides three-dimensional bearing and elevation angle measurements immersed with clutters. A Gaussian-mixture/probability-hypothesis-density multitarget tracker using an unscented Kalman filter is proposed to estimate target trajectories. A new closed-loop formulation of the unscented-Kalman-filter/Gaussian-mixture/probability hypothesis density is derived using fundamental properties of nonlinear Bayesian filtering. A tag management procedure to maintain individual track identity is also discussed. The performance of the unscented-Kalman-filter/Gaussian-mixture/probability hypothesis density is compared with seven data-association-based multitarget tracking algorithms, such as 1) the suboptimal nearest neighbor, 2) the global nearest neighbor using Munkres optimal assignment method, 3) the global nearest neighbor with the Jonker–Volgenant–Castanon assignment technique, 4) the multihypothesis tracking using -best feasible solutions, 5) probabilistic data association, 6) nearest-neighbor probabilistic data association, and 7) -nearest-neighbor probabilistic data association. These algorithms are computed using the unscented Kalman filter and follow a track-oriented approach. Simulation results reveal that the unscented-Kalman-filter/Gaussian-mixture/probability hypothesis density algorithm has superior performance over conventional data-association methods in terms of cardinality accuracy and robustness.

Particle-gating SMC-PHD filter

The Sequential Monte Carlo (SMC) implementation for the probability hypothesis density (PHD) filter, referred to as the SMC-PHD filter, is a good candidate for multi-target tracking (MTT) problems. It recursively propagates the weighted particle set that approximates the multi-target posterior density. In this paper, we propose an improved SMC-PHD filter for MTT called the particle-gating SMC-PHD filter. First, a robust gating based on particles propagated from a previous time period is proposed to select the observations of survival targets. Second, a sigma-nearest-gating is proposed to accurately select the observations of new targets. By employing only the observations obtained by the above algorithms to update the state estimations, the overall processing speed of the filter is significantly improved. In addition, a softening factor is suggested to lower the average number of clutters in the updater. This provides more accurate estimation compared with the basic SMC-PHD filter. Finally, the respective real-time and tracking performances of the proposed SMC-PHD filter are verified by the simulation results.

Labeling uncertainty in multitarget tracking

In multitarget tracking, the problem of track labeling (assigning labels to tracks) is an ongoing research topic. The existing literature, however, lacks an appropriate measure of uncertainty related to the assigned labels that has a sound mathematical basis as well as clear practical meaning to the user. This is especially important in a situation where well separated targets move in close proximity with each other and thereafter separate again; in such a situation, it is well known that there will be confusion on target identities, also known as labeling. In this paper, we specify comprehensively the necessary assumptions for a Bayesian formulation of the multitarget tracking and labeling (MTTL) problem to be meaningful.We provide a mathematical characterization of the labeling uncertainties with clear physical interpretation.We also propose a novel labeling procedure that can be used in combination with any existing (unlabeled)MTT algorithm to obtain a Bayesian solution to the MTTL problem. One advantage of the resulting solution is that it readily provides the labeling uncertainty measures. Using the mixed labeling phenomenon in the presence of two targets as our test bed, we show with simulation results that an unlabeled multitarget sequential Monte Carlo (M-SMC) algorithm that employs sequential importance resampling (SIR) augmented with our labeling procedure performs much better than its naive extension, the labeled SIR M-SMC filter.

Radar and stereo vision fusion for multitarget tracking on the special Euclidean group

Reliable scene analysis, under varying conditions, is an essential task in nearly any assistance or autonomous system application, and advanced driver assistance systems (ADAS) are no exception. ADAS commonly involve adaptive cruise control, collision avoidance, lane change assistance, traffic sign recognition, and parking assistance—with the ultimate goal of producing a fully autonomous vehicle. The present paper addresses detection and tracking of moving objects within the context of ADAS. We use a multisensor setup consisting of a radar and a stereo camera mounted on top of a vehicle. We propose to model the sensors uncertainty in polar coordinates on Lie Groups and perform the objects state filtering on Lie groups, specifically, on the product of two special Euclidean groups, i.e., SE(2)2. To this end, we derive the designed filter within the framework of the extended Kalman filter on Lie groups. We assert that the proposed approach results with more accurate uncertainty modeling, since used sensors exhibit contrasting measurement uncertainty characteristics and the predicted target motions result with banana-shaped uncertainty contours. We believe that accurate uncertainty modeling is an important ADAS topic, especially when safety applications are concerned. To solve the multitarget tracking problem, we use the joint integrated probabilistic data association filter and present necessary modifications in order to use it on Lie groups. The proposed approach is tested on a real-world dataset collected with the described multisensor setup in urban traffic scenarios.

A Newborn Track Detection and State Estimation Algorithm Using Bernoulli Random Finite Sets

In multi-target tracking (MTT) problems, there are many important issues that affect performance, including statistical filtering, measurement-target association, and estimating the number of targets. While newborn target detection and state estimation should also be considered as important factors in MTT, only a few studies have addressed these topics. In this paper, a novel newborn track detection and state estimation method is proposed using the concept of Bernoulli random finite sets. The posterior finite set statistical probability density function (FISST PDF) of a newborn target is analytically derived, and a tractable implementation scheme is proposed using importance sampling. Finally, the validity of the proposed method is demonstrated via integration with a Gaussian mixture probability hypothesis density (GM-PHD) filter and subsequent application to MTT problems.

Localizing activity groups in videos

Beyond recognizing actions of individuals, activity group localization in videos aims to localize groups of persons in spatiotemporal spaces and recognize what activity the group performs. In this paper, we propose a latent graph model to simultaneously address the problem of multi-target tracking, group discovery and activity recognition. Our key insight is to exploit the contextual relations among people. We present them as a latent relational graph, which hierarchically encodes the association potentials between tracklets, intra-group interactions, correlations, and inter-group compatibilities. Our model is capable of propagating multiple evidences among different layers of the latent graph. Particularly, associated tracklets assist accurate group discovery, activity recognition can benefit from knowing the whole structured groups, and the group and activity information in turn provides strong cues for establishing coherent associations between tracklets. Experiments on five datasets demonstrate that our model achieves both significant improvements in activity group localization and competitive performance on activity recognition.

GM-PHD filter-based multi-target tracking in the presence of Doppler blind zone

For the problem of multi-target tracking, utilization of Doppler information can greatly improve the tracking performance. However, the presence of Doppler blind zone (DBZ) strongly deteriorates the performance. To overcome this problem, we first employ a well-known detection probability model with incorporating the minimum detectable velocity (MDV), which determines the width of the DBZ. Then, by substituting the resulting detection probability into the update equation of the standard Gaussian mixture probability hypothesis density (GM-PHD) filter, we derive the update expression for a novel GM-PHD filter where the MDV and Doppler information are fully exploited. Moreover, we present an approximate filter with detailed implementation steps to reduce the amount of calculation without a significant degradation in performance. It is demonstrated through numerical examples that the proposed approach outperforms the existing GM-PHD filter which doesn't incorporate the MDV or Doppler information. Particularly, it has significantly better performance at small MDV value.

Adaptive Gaussian mixture probability hypothesis density for tracking multiple targets

The probability hypothesis density (PHD) filter is a promising method for multi-target tracking problem. The Gaussian mixture PHD filter is an analytic solution to the PHD filter for linear Gaussian multi-target models. However, the PHD filter is inapplicable to the multi-target tracking scenario with unknown newborn target intensity. To overcome the problem, an adaptive Gaussian mixture probability hypothesis density algorithm for multiple target tracking is proposed, where the Dirichlet distribution with negative exponent parameters and target maximum velocity constraint-based schemes are introduced to recursively estimate the newborn target intensity at each time step. The simulation results illustrate that the proposed algorithm has better performance under the unknown newborn target intensity in the multi-target tracking systems.

Tracking Crossing Targets in Passive Sonars Using NNJPDA

This paper presents an effective solution to the problems of multi-target tracking in passive sonar. Since bearing information alone is used, target crossing problems arises in passive sonar. Nearest Neighbor Joint Probabilistic Data Association (NNJPDA) is employed for the information processing. Further a prediction mechanism is used, where the track bearing depends only on the predicted value not on the measurements. This leads to the correct assignment of measurements with the track in the clutter and thereby avoids the track loss. The state estimation is then refined by Kalman filtering based on the corrected measurements from the NNJPDA technique. Thus accurate and continuous track is maintained.

Improved Distributed Model Predictive Control with Control Planning Set

We focus on distributed model predictive control algorithm. Each distributed model predictive controller communicates with the others in order to compute the control sequence. But there are not enough communication resources to exchange information between the subsystems because of the limited communication network. This paper presents an improved distributed model predictive control scheme with control planning set. Control planning set algorithm approximates the future control sequences by designed planning set, which can reduce the exchange information among the controllers and can also decrease the distributed MPC controller calculation demand without degrading the whole system performance much. The stability and system performance analysis for distributed model predictive control are given. Simulations of the four-tank control problem and multirobot multitarget tracking problem are illustrated to verify the effectiveness of the proposed control algorithm.

PMHT Approach for Multi-Target Multi-Sensor Sonar Tracking in Clutter.

Multi-sensor sonar tracking has many advantages, such as the potential to reduce the overall measurement uncertainty and the possibility to hide the receiver. However, the use of multi-target multi-sensor sonar tracking is challenging because of the complexity of the underwater environment, especially the low target detection probability and extremely large number of false alarms caused by reverberation. In this work, to solve the problem of multi-target multi-sensor sonar tracking in the presence of clutter, a novel probabilistic multi-hypothesis tracker (PMHT) approach based on the extended Kalman filter (EKF) and unscented Kalman filter (UKF) is proposed. The PMHT can efficiently handle the unknown measurements-to-targets and measurements-to-transmitters data association ambiguity. The EKF and UKF are used to deal with the high degree of nonlinearity in the measurement model. The simulation results show that the proposed algorithm can improve the target tracking performance in a cluttered environment greatly, and its computational load is low.