Multi-target Tracking Framework Research Articles

Extended target tracking under multitarget tracking framework for convex polytope shapes

This paper discusses the problem of extended target tracking for a single 2D extended target with a convex polytopic shape and known dynamics. Extended targets are those that produce multiple measurements for a single frame. One of the major challenges in extended target tracking is the joint uncertainty in the shape and the kinematics of the target. Another challenge is the lack of visibility due to self-occlusion in targets with a finite extent (as opposed to zero extent for point targets). To address these challenges, we develop a framework for tracking single (or widely separated) extended targets. This framework is based on the existing point multitarget tracking framework by modeling different parts of an extended target as separate targets. An algorithm is developed using the proposed framework for tracking convex polytope-shaped targets. The proposed shape function consists only of the boundary of the target since the center may not be observable. The algorithm is capable of dynamically changing the number of parameters used to describe the shape as more parts of the target become visible over time. The performance of the algorithm is evaluated for various scenarios using root mean square error (RMSE) of velocity, center, and intersection over union (IoU) metrics. It is seen that the algorithm is able to handle the self-occlusion problem and estimate the whole target shape even when different parts of the target are visible at different frames, for various shapes, and for various conditions of measurement noise covariance and number of measurements. New faces are added to the shape estimate as more parts of the target become visible. The algorithm is able to conserve the parts of the target that were visible in the previous frames but are no longer visible.

Possibility Generalized Labeled Multi-Bernoulli Filter for Multi-Target Tracking Under Epistemic Uncertainty

This paper presents a flexible modeling framework for multi-target tracking based on the theory of Outer Probability Measures (OPMs). The notion of labeled uncertain finite set is introduced and utilized as the basis to derive a possibilistic analog of the <inline-formula><tex-math notation="LaTeX">$\delta$</tex-math></inline-formula>-Generalized Labeled Multi-Bernoulli (<inline-formula><tex-math notation="LaTeX">$\delta$</tex-math></inline-formula>-GLMB) filter, in which the uncertainty in the multi-target system is represented by possibility functions instead of probability distributions. The proposed method inherits the capability of the standard probabilistic <inline-formula><tex-math notation="LaTeX">$\delta$</tex-math></inline-formula>-GLMB filter to yield joint state, number, and trajectory estimates of multiple appearing and disappearing targets. Beyond that, it is capable to account for epistemic uncertainty due to ignorance or partial knowledge regarding the multi-target system, e.g., the absence of complete information on dynamical model parameters (e.g., probability of detection, birth) and initial number and state of newborn targets. The features of the developed filter are demonstrated using two simulated scenarios.

Extended Target Tracking Under Multitarget Tracking Framework for Convex Polytope Shapes

Extended Target Tracking Under Multitarget Tracking Framework for Convex Polytope Shapes

Fast Multi-Shadow Tracking for Video-SAR Using Triplet Attention Mechanism

This article extends the shadow tracking for video-synthetic aperture radar (SAR) from a single-target framework to a multitarget framework, which is crucial for SAR ground moving targets’ identification. Inspired by FairMOT, the multitarget tracking framework for SAR shadow tracking is improved by using the triplet attention (TriAtt) mechanism and the lightweight multiscale network. By employing the ability to fuse spatial and feature dimensions of TriAtt and combining the lightweight network optimized by multiscale encoder–decoder and dilated convolution, a fast multiscale feature extraction module (FMsFEM) embedded with TriAtt is proposed for better tracking efficiency and performance. Experiments on the Sandiego video-SAR dataset validate that the TriAtt mechanism can improve the tracking performance of deep layer aggregation (DLA)-34, DLA-18, and FMsFEM significantly. FMsFEM with embedded TriAtt outperforms the state-of-the-art network (FairMOT with backbones of DLA-34 and DLA-18) with much faster frame rates. The average frame rates of FMsFEM and FMsFEM-TriAtt reach 60.32 and 56.13 fps for datasets with an image size of <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$1088\times 608$ </tex-math></inline-formula> , which are about three times higher than the frame rates of others.

Fusion-Based Multidetection Multitarget Tracking With Random Finite Sets

Multidetection (MD) systems are characterized by multiple observation modes (OMs), and hence, simultaneously produce multiple measurements for each target. The key challenge in exploiting MD systems for multitarget tracking (MTT), compared to single-detection (SD) systems, is the significant amount of extra computational burden involved in order to solve the resulting multidimensional assignment problem among measurements, targets, and OMs. This article presents a novel computationally efficient MTT framework for MD systems, wherein the multitarget state is modeled as a random finite set (RFS), and a bank of OM-dependent MTT RFS filters with SD model are employed to recursively provide OM-dependent posteriors. The latter, which contain both real and false targets, are then suitably fused so as to enhance consensus on the true targets while weakening trust on the existence of the false ones. In this way, the computational complexity is significantly reduced compared to existing MTT algorithms with the MD model. Two representative RFS filters, i.e., unlabeled probability hypothesis density (PHD) and labeled multi-Bernoulli (LMB), are considered in the proposed framework and the computational complexity of the resulting MD MTT algorithms is analyzed. Performance of the proposed approach is assessed by simulation experiments in both over-the-horizon-radar (OTHR) and single-frequency-network passive radar (SFN-PR) MTT applications.

Multi-Target Multi-Camera Tracking of Vehicles Using Metadata-Aided Re-ID and Trajectory-Based Camera Link Model.

In this paper, we propose a novel framework for multi-target multi-camera tracking (MTMCT) of vehicles based on metadata-aided re-identification (MA-ReID) and the trajectory-based camera link model (TCLM). Given a video sequence and the corresponding frame-by-frame vehicle detections, we first address the isolated tracklets issue from single camera tracking (SCT) by the proposed traffic-aware single-camera tracking (TSCT). Then, after automatically constructing the TCLM, we solve MTMCT by the MA-ReID. The TCLM is generated from camera topological configuration to obtain the spatial and temporal information to improve the performance of MTMCT by reducing the candidate search of ReID. We also use the temporal attention model to create more discriminative embeddings of trajectories from each camera to achieve robust distance measures for vehicle ReID. Moreover, we train a metadata classifier for MTMCT to obtain the metadata feature, which is concatenated with the temporal attention based embeddings. Finally, the TCLM and hierarchical clustering are jointly applied for global ID assignment. The proposed method is evaluated on the CityFlow dataset, achieving IDF1 76.77%, which outperforms the state-of-the-art MTMCT methods.

A Linear Algorithm for Multi-Target Tracking in the Context of Possibility Theory

We present a modelling framework for multi-target tracking based on possibility theory and illustrate its ability to account for the general lack of knowledge that the target-tracking practitioner must deal with when working with real data. We also introduce and study variants of the notions of point process and intensity function, which lead to the derivation of an analogue of the probability hypothesis density (PHD) filter. The gains provided by the considered modelling framework in terms of flexibility lead to the loss of some of the abilities that the PHD filter possesses; in particular the estimation of the number of targets by integration of the intensity function. Yet, the proposed recursion displays a number of advantages such as facilitating the introduction of observation-driven birth schemes and the modelling the absence of information on the initial number of targets in the scene. The performance of the proposed approach is demonstrated on simulated data.

Automated extraction of dolphin whistles-A sequential Monte Carlo probability hypothesis density approach.

The need for automated methods to detect and extract marine mammal vocalizations from acoustic data has increased in the last few decades due to the increased availability of long-term recording systems. Automated dolphin whistle extraction represents a challenging problem due to the time-varying number of overlapping whistles present in, potentially, noisy recordings. Typical methods utilize image processing techniques or single target tracking, but often result in fragmentation of whistle contours and/or partial whistle detection. This study casts the problem into a more general statistical multi-target tracking framework and uses the probability hypothesis density filter as a practical approximation to the optimal Bayesian multi-target filter. In particular, a particle version, referred to as a sequential Monte Carlo probability hypothesis density (SMC-PHD) filter, is adapted for frequency tracking and specific models are developed for this application. Based on these models, two versions of the SMC-PHD filter are proposed and the performance of these versions is investigated on an extensive real-world dataset of dolphin acoustic recordings. The proposed filters are shown to be efficient tools for automated extraction of whistles, suitable for real-time implementation.

Joint Target Assignment and Resource Optimization Framework for Multitarget Tracking in Phased Array Radar Network

In this article, a joint target assignment and resource optimization (JTARO) strategy is proposed for the application of multitarget tracking in phased array radar network system. The key mechanism of our proposed JTARO strategy is to employ the optimization technique to jointly optimize the target-to-radar assignment, revisit time control, bandwidth, and dwell time allocation subject to several resource constraints, while achieving better tracking accuracies of multiple targets and low probability of intercept (LPI) performance of phased array radar network. The analytical expression for Bayesian Cramér-Rao lower bound with the aforementioned adaptable parameters is calculated and subsequently adopted as the performance metric for multitarget tracking. After problem partition and reformulation, an efficient three-stage solution methodology is developed to resolve the underlying mixed-integer, nonlinear, and nonconvex optimization problem. To be specific, in Step 1, the revisit time for each target is determined. In Step 2, we implement the joint signal bandwidth and dwell time allocation for fixed target-to-radar assignments, which combine the cyclic minimization algorithm and interior point method. In Step 3, the optimal target-to-radar assignments are obtained, which results in the minimization of both the tracking accuracy for multiple targets and the total dwell time consumption of the network system. Simulation results are provided to demonstrate the advantages of the presented JTARO strategy, in terms of the achievable multitarget tracking accuracy and LPI performance of phased array radar network.

Classification-Aided Multitarget Tracking Using the Sum-Product Algorithm

Multitarget tracking (MTT) is a challenging task that aims at estimating the number of targets and their states from measurements of the target states provided by one or multiple sensors. Additional information, such as imperfect estimates of target classes provided by a classifier, can facilitate the target-measurement association and thus improve MTT performance. In this letter, we describe how a recently proposed MTT framework based on the sum-product algorithm can be extended to efficiently exploit class information. The effectiveness of the proposed approach is demonstrated by simulation results.

Convolutional Neural Network for Monocular Vision-based Multi-target Tracking

This paper addresses multi-target tracking using a monocular vision sensor. To overcome the fundamental observability issue of the monocular vision, a convolutional neural network (CNN)-based method is proposed. The method combines a CNN-based multi-target detection into a model-based multi-target tracking framework. While previous CNN applications to image-based object recognition and tracking focused on prediction of region of interest (RoI), the proposed method allows for prediction of the three-dimensional position information of the moving objects of interest. This is achieved by appropriately construct a network tailored to the moving object tracking problems with potentially occluded objects. In addition, the cubature Kalman filter integrated with a data association scheme is adopted for effective tracking of nonlinear motion of the objects with the measurements information from the learned network. A virtual simulator that generates the trajectories of the target motions and a sequence of images of the scene has been developed and used to test and verify the proposed CNN scheme. Simulation case studies demonstrate that the proposed CNN improves the position accuracy in the depth direction substantially.

Multi-target Tracking in Multiple Non-overlapping Cameras Using Fast-Constrained Dominant Sets

In this paper, a unified three-layer hierarchical approach for solving tracking problem in a multiple non-overlapping cameras setting is proposed. Given a video and a set of detections (obtained by any person detector), we first solve within-camera tracking employing the first two layers of our framework and then, in the third layer, we solve across-camera tracking by associating tracks of the same person in all cameras simultaneously. To best serve our purpose, we propose fast-constrained dominant set clustering (FCDSC), a novel method which is several orders of magnitude faster (close to real time) than existing methods. FCDSC is a parameterized family of quadratic programs that generalizes the standard quadratic optimization problem. In our method, we first build a graph where nodes of the graph represent short-tracklets, tracklets and tracks in the first, second and third layer of the framework, respectively. The edge weights reflect the similarity between nodes. FCDSC takes as input a constrained set, a subset of nodes from the graph which need to be included in the extracted cluster. Given a constrained set, FCDSC generates compact clusters by selecting nodes from the graph which are highly similar to each other and with elements in the constrained set. We have tested this approach on a very large and challenging dataset (namely, MOTchallenge DukeMTMC) and show that the proposed framework outperforms the state-of-the-art approaches. Even though the main focus of this paper is on multi-target tracking in non-overlapping cameras, the proposed approach can also be applied to solve video-based person re-identification problem. We show that when the re-identification problem is formulated as a clustering problem, FCDSC can be used in conjunction with state-of-the-art video-based re-identification algorithms, to increase their already good performances. Our experiments demonstrate the general applicability of the proposed framework for multi-target multi-camera tracking and person re-identification tasks.

Survey of amplitude‐aided multi‐target tracking methods

An amplitude-aided multi-target tracking (MTT) exploits amplitude as well as spatial features for MTT. Compared to MTT using a spatial feature only, it can maintain its performance even in densely cluttered environment because more accurate association is achieved by using both features for likelihood evaluation between tracks and measurements. Therefore, the goal of this study is to review the state-of-the-art amplitude-aided MTT methods and compare each other extensively. For a fair comparison, a unified MTT framework is developed, and various methods are implemented and compared based on the same framework. On the challenging visual MTT datasets, the implemented methods are evaluated for several aspects such as accuracy, speed, sensitivity, and stability. Moreover, this study presents a summary of the extensive evaluation and guideline to select an appropriate method for amplitude-aided tracking.

Tracking People in Highly Dynamic Industrial Environments

To date, the majority of positioning systems have been designed to operate within environments that have long-term stable macro-structure with potential small-scale dynamics. These assumptions allow the existing positioning systems to produce and utilize stable maps. However, in highly dynamic industrial settings these assumptions are no longer valid and the task of tracking people is more challenging due to the rapid large-scale changes in structure. In this paper we propose a novel positioning system for tracking people in highly dynamic industrial environments, such as construction sites. The proposed system leverages the existing CCTV camera infrastructure found in many industrial settings along with radio and inertial sensors within each worker's mobile phone to accurately track multiple people. This multi-target multi-sensor tracking framework also allows our system to use cross-modality training in order to deal with the environment dynamics. In particular, we show how our system uses cross-modality training in order to automatically keep track environmental changes (i.e. new walls) by utilizing occlusion maps. In addition, we show how these maps can be used in conjunction with social forces to accurately predict human motion and increase the tracking accuracy. We have conducted extensive real-world experiments in a construction site showing significant accuracy improvement via cross-modality training and the use of social forces.

Joint estimation of multi‐target signal‐to‐noise ratio and dynamic states in cluttered environment

In this study, the authors consider a multi-target tracking (MTT) problem in a cluttered environment. Due to the difficulty of the problem, the methods relying only on spatial information such as range, bearing and Doppler velocity can be unreliable. To overcome this, they additionally exploit the amplitude information, commonly provided by radar and sonar, for MTT. However, the usage of amplitude information is not straightforward because the signal-to-noise ratio (SNR) should be known in advance or estimated at the same time. To this end, they first propose a novel SNR estimation algorithm based on a maximum a posteriori approach, which helps the tracker to exploit the amplitude information effectively. Based on the estimated SNR, they then propose a complete framework for MTT, which is mainly composed of data association and track state update parts. They extensively evaluate the proposed system in a series of challenging scenarios, and the experimental results verify the effectiveness and robustness of the authors’ methods.

Globality–Locality-Based Consistent Discriminant Feature Ensemble for Multicamera Tracking

Spatiotemporal data association and fusion is a well-known NP-hard problem even in a small number of cameras and frames. Although it is difficult to be tractable, solving them is pivotal for tracking in a multicamera network. Most approaches model association maladaptively toward properties and contents of video, and hence they produce suboptimal associations and association errors propagate over time to adversely affect fusion. In this paper, we present an online multicamera multitarget tracking framework that performs adaptive tracklet correspondence by analyzing and understanding contents and properties of video. Unlike other methods that work only on synchronous videos, our approach uses dynamic time warping to establish correspondence even if videos have linear or nonlinear time asynchronous relationship. Association is a two-stage process based on geometric and appearance descriptor space ranked by their inter- and intra-camera consistency and discriminancy. Fusion is reinforced by weighting the associated tracklets with a confidence score calculated using reliability of individual camera tracklets. Our robust ranking and election learning algorithm dynamically selects appropriate features for any given video. Our method establishes that, given the right ensemble of features, even computationally efficient optimization yields better accuracy in tracking over time and provides faster convergence that is suitable for real-time application. For evaluation on RGB, we benchmark on multiple sequences in PETS 2009 and we achieve performance that is on par with the state of the art. For evaluating on RGB-D, we built a new data set.

Tracklet association based multi-target tracking

This paper proposes a novel multi-target tracking framework, where two different association strategies are utilized to obtain local and global tracking trajectories. Specifically, a scene self-adaptive model is first utilized to generate local trajectories by constructing the association between detection responses and tracking tracklets; then, a novel incremental linear discriminative appearance model is utilized to generate global trajectories by constructing the association between local trajectories; finally, a non-linear motion model is utilized to fill the vacancies between global trajectories to obtain continuous and smooth tracking trajectories. Experimental results conducted on PETS2009/2010, TUD-Stadtmitte, and Town Center video libraries demonstrate the proposed framework can achieve continuous and smooth tracking trajectories under the case of significant deformation, appearance change, similar appearance, motion direction change, and long-time occlusion.

A novel multitarget tracking algorithm for Myosin VI protein molecules on actin filaments in TIRFM sequences.

We propose a novel multitarget tracking framework for Myosin VI protein molecules in total internal reflection fluorescence microscopy sequences which integrates an extended Hungarian algorithm with an interacting multiple model filter. The extended Hungarian algorithm, which is a linear assignment problem based method, helps to solve measurement assignment and spot association problems commonly encountered when dealing with multiple targets, although a two-motion model interacting multiple model filter increases the tracking accuracy by modelling the nonlinear dynamics of Myosin VI protein molecules on actin filaments. The evaluation of our tracking framework is conducted on both real and synthetic total internal reflection fluorescence microscopy sequences. The results show that the framework achieves higher tracking accuracies compared to the state-of-the-art tracking methods, especially for sequences with high spot density.

Multi-Bernoulli sensor-selection for multi-target tracking with unknown clutter and detection profiles

A new sensor-selection solution within a multi-Bernoulli-based multi-target tracking framework is presented. The proposed method is especially designed for the general multi-target tracking case with no prior knowledge of the clutter distribution or the probability of detection, and uses a new task-driven objective function for this purpose. Step-by-step sequential Monte Carlo implementation of the method is presented along with a similar sensor-selection solution formulated using an information-driven objective function (Rényi divergence). The two solutions are compared in a challenging scenario and the results show that while both methods perform similarly in terms of accuracy of cardinality and state estimates, the task-driven sensor-selection method is substantially faster.

Tracking Multiple High-Density Homogeneous Targets

We present a framework for multitarget detection and tracking that infers candidate target locations in videos containing a high density of homogeneous targets. We propose a gradient-climbing technique and an isocontor slicing approach for intensity maps to localize targets. The former uses Markov chain Monte Carlo to iteratively fit a shape model onto the target locations, whereas the latter uses the intensity values at different levels to find consistent object shapes. We generate trajectories by recursively associating detections with a hierarchical graph-based tracker on temporal windows. The solution to the graph is obtained with a greedy algorithm that accounts for false-positive associations. The edges of the graph are weighted with a likelihood function based on location information. We evaluate the performance of the proposed framework on challenging datasets containing videos with high density of targets and compare it with six alternative trackers.