Multi-target Tracking Method Research Articles

Efficient Multi-View Multi-Target Tracking Using a Distributed Camera Network

In this paper, we propose a multi-target tracking method using a distributed camera network, which can effectively handle the occlusion and reidenfication problems by combining advanced deep learning and distributed information fusion. The targets are first detected using a fast object detection method based on deep learning. We then combine the deep visual feature information and spatial trajectory information in the Hungarian algorithm for robust targets association. The deep visual feature information is extracted from a convolutional neural network, which is pre-trained using a large-scale person reidentification dataset. The spatial trajectories of multiple targets in our framework are derived from a multiple view information fusion method, which employs an information weighted consensus filter for fusion and tracking. In addition, we also propose an efficient track processing method for ID assignment using multiple view information. The experiments on public datasets show that the proposed method is robust to solve the occlusion problem and reidentification problem, and can achieve superior performance compared to the state of the art methods.

Multi-Target Tracking by Associating and Fusing the Multi-Bernoulli Parameter Sets

The Cardinality Balanced MeMBer (CBMeMBer) filter is a single sensor multi-target tracking method based on the random finite set. Compared with the single sensor system, the multi-sensor system can achieve more stable and better performance in tracking targets. However, some problems exist in multi-sensor system based on the CBMeMBer filter. Tracks in the CBMeMBer filter are described by parameter sets which may be generated by miss-detection, targets and clutters. It is difficult to associate the parameter sets correctly because of their complex forms and various types. Moreover, the filter reacts slowly to disappeared targets, which leads to a cardinality overestimation. This problem may be more serious in the multi-sensor system. To deal with the above problems, the parameter sets association and fusion methods are presented in this paper. By three association processes with the adaptive thresholds selection approaches, parameter sets corresponding to the same target are grouped into one parameter set partition. Parameter sets have different association thresholds because of their different accuracies. The fusion method considers the types and relationships of parameter sets in the partition simultaneously and uses a joint credibility to accelerate changes in existence probability. The cardinality estimation decreases rapidly when the target disappears. The theoretical analysis and experiment results of different tracking scenarios show that the proposed methods perform well in both state estimation and cardinality estimation.

Multi-Target Localization and Tracking Using TDOA and AOA Measurements Based on Gibbs-GLMB Filtering.

This paper deals with mobile multi-target detection and tracking. In the traditional method, there are uncertainties such as misdetection and false alarm in the measurement data, and it will be inevitable having to deal with the data association. To solve the target trajectory and state estimation problem under a cluttered environment, this paper proposes a non-concurrent multi-target acoustic localization tracking method based on the Gibbs-generalized labelled multi-Bernoulli (Gibbs-GLMB) filter and considers an acoustic array of a fixed arrangement for the tracking of targets by joint time difference of arrival (TDOA) and angle of arrival (AOA) measurements. Firstly, the TDOAs are calculated by using the generalized cross-correlation algorithm (GCC) and the AOAs are derived from the received signal directions. Secondly, we assume the independence of the targets and fuse the measurements which are used to track the multiple targets via the Gibbs-GLMB filter. Finally, the effectiveness of the method is verified by Monte Carlo simulation experiments.

A new improved firefly clustering algorithm for SMC-PHD filter

Aiming at the difficulty of extracting multi-target states in SMC-PHD filter, this paper proposes a SMC-PHD multi-target tracking method based on the improved firefly clustering algorithm. This method uses the improved firefly clustering algorithm to extract the peaks of the updated PHD function. Firstly, select the particles equal to the number of targets to form firefly individuals, introduce attraction and movement mechanism of the firefly algorithm, use the optimal firefly guiding other fireflies’ movement, and undergo combination and mutation operation for the optimal firefly. Search the optimal solution in the whole solution space and obtain multi-target states. The experimental results show that the improved algorithm has more stable peak extraction ability than K-Means clustering algorithm in SMC-PHD filter.

Multi-target tracking method based on improved firefly algorithm optimized particle filter

Joint probabilistic data association-particle filter (JPDA-PF) algorithm is often used to solve the problem of data association in multi-target tracking, it has the advantages of particle filter in dealing with nonlinear and non-Gaussian systems, but at the same time, the resampling process of particle filter will lead to the problem of particle impoverishment, which will affect the tracking accuracy. In this article, a new multi-target tracking method based on particle filter optimized by improved firefly algorithm is proposed. To solve the problem of weight degradation and avoid the problem of particle impoverishment, the improved firefly algorithm replaces the traditional resampling method and optimizes the distribution of particles in particle filter. According to the simulation results, the improved algorithm has better tracking accuracy than JPDA-PF as the number of particles is equal.

Multi-target tracking by non-linear motion patterns based on hierarchical network flows

We propose a multi-target tracking method for pedestrians, through locating them and maintaining their identities to make their individual trajectories. In most existing data association-based tracking approaches, the motion information in the affinity model is always linear, but the motion of target is non-linear in real-world surveillance video, especially when there is occlusion. This paper proposes a global optimization method based on hierarchical network flows with a non-linear motion model to track multiple targets. In our method, each node in network represents a tracklet, and each edge represents the likelihood of neighboring tracklets belonging to the same trajectory measured by affinity score that we proposed, it differs from other network flow formulation whose nodes represent detection responses. A non-linear motion map is used for explaining non-linear motion pattern between neighboring tracklets and getting motion affinity when there are time gaps. We evaluate our approach on 2DMOT2015, MOT16 database, and several real surveillance videos. Experimental results demonstrate that the proposed method can achieve continuous tracking trajectories under the case of motion direction changes and complete occlusions.

Multi-Objective Optimization Based Multi-Bernoulli Sensor Selection for Multi-Target Tracking.

This paper presents a novel multi-objective optimization based sensor selection method for multi-target tracking in sensor networks. The multi-target states are modelled as multi-Bernoulli random finite sets and the multi-Bernoulli filter is used to propagate the multi-target posterior density. The proposed method is designed to select the sensor that provides the most reliable cardinality estimate, since more accurate cardinality estimate indicates more accurate target states. In the multi-Bernoulli filter, the updated multi-target density is a multi-Bernoulli random finite set formed by a union of legacy tracks and measurement-updated tracks. The legacy track and the measurement-updated track have different theoretical and physical meanings, and hence these two kinds of tracks are considered separately in the sensor management problem. Specifically, two objectives are considered: (1) maximizing the mean cardinality of the measurement-updated tracks, (2) minimizing the cardinality variance of the legacy tracks. Considering the conflicting objectives simultaneously is a multi-objective optimization problem. Tradeoff solutions between two conflicting objectives will be derived. Theoretical analysis and examples show that the proposed approach is effective and direct. The performance of the proposed method is demonstrated using two scenarios with different levels of observability of targets in the passive sensor network.

Multi-Target Tracking With Multiple 2D Range Scanners

We propose a novel efficient online method for tracking performers on stage. Most existing tracking methods have focused on using expensive, high-performing sensors, such as multilayer lidar sensors or high-resolution, short-range radars. Image sensor-based methods are not appropriate for tracking performers on stage because of challenging illumination conditions caused by lighting effects. In this paper, we introduce a robust multi-target tracking method based on the sensor fusing of two-dimensional distance sensors such as single-layer lidar sensors that have a relatively lower cost than the aforementioned types of sensors. In our method, measurements from each sensor are transformed into a reference coordinate system and objects are detected with those transformed measurements. Then, the object detections are used in generating or extending the trajectories of targets by detection-to-trajectory matching. In our experiments, we quantitatively evaluated the proposed method with a newly constructed dataset which consists of two scenarios simulating performances on stage. We collected the scan results of two single-layer lidar sensors and image frames captured by a camera sensor for each scenario. The experimental results show that the proposed method robustly tracks performers in challenging scenarios, in which the performers move abruptly and are densely located.

Multi-target tracking with hierarchical data association using main-parts and spatial-temporal feature models

Multi-target tracking in complex scenes is challenging because target appearance features generate partial or significant variations frequently. In order to solve the problem, we propose a multi-target tracking method with hierarchical data association using main-parts and spatial-temporal feature models. In our tracking framework, target feature models and tracklets are initialized when the new targets appear. Main-parts feature model is presented to represent target with partial or no appearance variations. It is established by partitioning a target template into several parts and formulating appearance variation densities of these parts. For the target with significant appearance variations, the tracker learns its global spatial-temporal feature model by integrating appearance with histogram of optical flow features. During tracking, tracklet confidence is exploited to implement hierarchical data association. According to different tracklet confidence values, main-parts and global data association are respectively performed by employing main-parts and spatial-temporal feature models. As a result, our approach uses the Hungarian algorithm to obtain optimal associated pairs between target tracklets and detections. Finally, target feature models and tracklets are updated by the association detections for subsequently tracking. Experiments conducted on CAVIAR, Parking Lot and MOT15 datasets verify the effectiveness and improvement of our multi-target tracking method.

Multitarget tracking using multifeature model with acceleration feature

Multitarget tracking in video surveillance is challenging because the appearance features of the target are often unreliable in complicated scenes. To solve the problem, we propose a multitarget tracking method using multifeature model with acceleration feature. First, an acceleration feature descriptor is derived from the histograms of the optical flow features according to the background difference. Our approach filters and normalizes the descriptors of consecutive frames to establish acceleration feature models. Then, the multifeature models of target templates are initialized by combining acceleration and multiple spatial feature models. Second, we implement data association based on the tracklet confidence by integrating the acceleration and multiple spatial feature affinities. As a result, the optimal associated pairs between target tracklets and detections are solved by the Hungarian algorithm. Finally, our tracking system updates the multifeature models of target templates online depending on the reliability of the tracklets, and the trajectories of multiple targets are output. Experiments conducted on the challenging multiple object tracking benchmark confirm the effectiveness and superiority of the proposed method.

A Robust Multi-Target Visual Tracking Method using TLD Framework

This paper investigates a robust multi-target visual tracking method based on TLD framework. We propose a novel visual tracking method that divides the long-term tracking task into four parts: tracker module, learning module, weak-detector module and strong-detector module. Before the tracking, it is essential to sample some basic posture of targets and build several patterns by training. Then, a separate process is allocated for each target may appear in the video or not. The strong-detector module makes a global scanning on every frame. Once a target of interest is detected, the tracker module starts to follow it from frame to frame. The weak-detector module makes a local scanning around the position predicted by the tracker module. The learning module estimates the errors of weak-detector module and updates the weak-detector module to avoid these errors from coming up again. At the last, when the posture of some target has large change, it may result in the metal failure of tracker module and weak-detector module. At this time, the strong-detector module will relocate the target and initialize the others modules.

Multitarget tracking in sensor networks via efficient information-theoretic sensor selection

In networks composed of moving robots or static sensing nodes, multitarget tracking is critical and fundamental for high-level applications, such as scene analysis or event detection. However, tracking multiple targets in the sensor network is challenging for two reasons: multisensor multitarget fusion itself is difficult and dynamic sensor scheduling is necessary to balance the tracking accuracy and energy consumption of the sensor network. In this article, we present a novel information-theoretic sensor selection method for multitarget tracking via the multi-Bernoulli filter. The sensor selection is based on the multi-Bernoulli filtering and a collection of subselection problems for individual target to avoid the combinatorial optimization. A subselection problem for each target is investigated under the framework of partially observed Markov decision process, and we propose to solve it by maximizing the information gain of the probability hypothesis density. Simulation results validate the effectiveness and efficiency of our method for multitarget tracking in sensor networks.

An Automatic Multi-Target Independent Analysis Framework for Non-Planar Infrared-Visible Registration.

In this paper, we propose a novel automatic multi-target registration framework for non-planar infrared-visible videos. Previous approaches usually analyzed multiple targets together and then estimated a global homography for the whole scene, however, these cannot achieve precise multi-target registration when the scenes are non-planar. Our framework is devoted to solving the problem using feature matching and multi-target tracking. The key idea is to analyze and register each target independently. We present a fast and robust feature matching strategy, where only the features on the corresponding foreground pairs are matched. Besides, new reservoirs based on the Gaussian criterion are created for all targets, and a multi-target tracking method is adopted to determine the relationships between the reservoirs and foreground blobs. With the matches in the corresponding reservoir, the homography of each target is computed according to its moving state. We tested our framework on both public near-planar and non-planar datasets. The results demonstrate that the proposed framework outperforms the state-of-the-art global registration method and the manual global registration matrix in all tested datasets.

Multi-Target Video Tracking Based on Improved Data Association and Mixed Kalman/<inline-formula> <tex-math notation="LaTeX">$H_{\infty }$ </tex-math> </inline-formula> Filtering

This paper proposes a novel multi-target video tracking (MTVT) method based on improved data association and mixed Kalman/ $H_{\infty }$ filtering. First, multiple features of video targets, such as local texture, spatial color distribution, and edge-oriented gradients, are extracted to form the fused-feature matching matrix. Second, an efficient and accurate video targets association method integrating an improved probabilistic data association (IPDA) and a simplified joint probabilistic data association (SJPDA) is developed. The IPDA combines the augmented posterior probability matrix with the fused-feature matching matrix for a multitarget association. On the other hand, the SJPDA ensures the efficiency of data association, and better accuracy in the presence of low peak signal-to-noise ratio and sparse target environment by sifting out big probability events. Finally, a mixed Kalman/ $H_{\infty }$ filtering fusing the covariances of state estimation is proposed for the fast and robust state estimation of video targets. Experimental results demonstrate that the proposed multifeature fusion MTVT is able to describe video targets accurately, guarantee both the efficiency and accuracy in the video targets association, and also it is more reliable and precise under noisy environments.

Multi-Camera Multi-Target Tracking with Space-Time-View Hyper-graph

Incorporating multiple cameras is an effective solution to improve the performance and robustness of multi-target tracking to occlusion and appearance ambiguities. In this paper, we propose a new multi-camera multi-target tracking method based on a space-time-view hyper-graph that encodes higher-order constraints (i.e., beyond pairwise relations) on 3D geometry, appearance, motion continuity, and trajectory smoothness among 2D tracklets within and across different camera views. We solve tracking in each single view and reconstruction of tracked trajectories in 3D environment simultaneously by formulating the problem as an efficient search of dense sub-hypergraphs on the space-time-view hyper-graph using a sampling based approach. Experimental results on the PETS 2009 dataset and MOTChallenge 2015 3D benchmark demonstrate that our method performs favorably against the state-of-the-art methods in both single-camera and multi-camera multi-target tracking, while achieving close to real-time running efficiency. We also provide experimental analysis of the influence of various aspects of our method to the final tracking performance.

Multi-Target State Extraction for the SMC-PHD Filter.

The sequential Monte Carlo probability hypothesis density (SMC-PHD) filter has been demonstrated to be a favorable method for multi-target tracking. However, the time-varying target states need to be extracted from the particle approximation of the posterior PHD, which is difficult to implement due to the unknown relations between the large amount of particles and the PHD peaks representing potential target locations. To address this problem, a novel multi-target state extraction algorithm is proposed in this paper. By exploiting the information of measurements and particle likelihoods in the filtering stage, we propose a validation mechanism which aims at selecting effective measurements and particles corresponding to detected targets. Subsequently, the state estimates of the detected and undetected targets are performed separately: the former are obtained from the particle clusters directed by effective measurements, while the latter are obtained from the particles corresponding to undetected targets via clustering method. Simulation results demonstrate that the proposed method yields better estimation accuracy and reliability compared to existing methods.

People-flow counting in complex environments by combining depth and color information

People-flow counting is one of the key techniques of intelligence video surveillance systems and the information of people-flow obtained from this technique is an very important evidence for many applications, such as business analysis, staff planning, security, etc. Traditionally, the color image information based methods encounter kinds of challenges, such as shadows, illumination changing, cloth color, etc., while the depth information based methods suffer from lack of texture. In this paper, we propose an effective approach of people-flow counting by combining color and depth information. First, we adopt a background subtraction technique to fast obtain the moving regions on depth images. Second, the water filling algorithm is used to effectively detect head candidates on the moving regions. Then we use the SVM to recognize the real heads from the candidates. Finally, we adopt a weighted K Nearest Neighbor based multi-target tracking method to track each confirmed head and count the people through the surveillance region. Four datasets constructed from two surveillance scenes are used to evaluate the proposed method. Experimental results show that our method outperform the state-of-the-art methods. Our method can work stably on condition of kinds of interruptions and can not only obtain high precisions, but also high recalls on four datasets.

IPDA filters in the sense of Gaussian mixture PHD algorithm

The Integrated Probabilistic Data Association (IPDA) type filters provide estimates of the underlying target probability of existence as well as they track state maintenance. For each scan, IPDA recursive calculates the probability of target existence in order to resolve the uncertainty. Likewise, Random Finite Set (RFS) is a method for single target and multi-target tracking. It provides a Bayesian recursion of multi-target distribution through the Finite Set calculus. Practical implementation of multi-target posterior recursion is too difficult. It was analytically proved that IPDA algorithm can be derived from the RFS based filter recursion under the linear Gaussian assumptions. Probability hypothesis density (PHD) filter is an alternative to this problem where only the first order moment of the complete multi-target posterior is propagated in time. In this article, IPDA and Gausian Mixtures PHD (GM PHD) filters in a single target tracking scenario are derived and compared. Simulations have demonstrated the superiority of IPDA filters in heavy clutters.

Multi-Target Tracking With Time-Varying Clutter Rate and Detection Profile: Application to Time-Lapse Cell Microscopy Sequences.

Quantitative analysis of the dynamics of tiny cellular and sub-cellular structures, known as particles, in time-lapse cell microscopy sequences requires the development of a reliable multi-target tracking method capable of tracking numerous similar targets in the presence of high levels of noise, high target density, complex motion patterns and intricate interactions. In this paper, we propose a framework for tracking these structures based on the random finite set Bayesian filtering framework. We focus on challenging biological applications where image characteristics such as noise and background intensity change during the acquisition process. Under these conditions, detection methods usually fail to detect all particles and are often followed by missed detections and many spurious measurements with unknown and time-varying rates. To deal with this, we propose a bootstrap filter composed of an estimator and a tracker. The estimator adaptively estimates the required meta parameters for the tracker such as clutter rate and the detection probability of the targets, while the tracker estimates the state of the targets. Our results show that the proposed approach can outperform state-of-the-art particle trackers on both synthetic and real data in this regime.

SMC-PHD based multi-target track-before-detect with nonstandard point observations model

Detection and tracking of multi-target with unknown and varying number is a challenging issue, especially under the condition of low signal-to-noise ratio (SNR). A modified multi-target track-before-detect (TBD) method was proposed to tackle this issue using a nonstandard point observation model. The method was developed from sequential Monte Carlo (SMC)-based probability hypothesis density (PHD) filter, and it was implemented by modifying the original calculation in update weights of the particles and by adopting an adaptive particle sampling strategy. To efficiently execute the SMC-PHD based TBD method, a fast implementation approach was also presented by partitioning the particles into multiple subsets according to their position coordinates in 2D resolution cells of the sensor. Simulation results show the effectiveness of the proposed method for time-varying multi-target tracking using raw observation data.