Multi-target Tracking Research Articles

Multi-Target Tracking in Multi-Static Networks with Autonomous Underwater Vehicles Using a Robust Multi-Sensor Labeled Multi-Bernoulli Filter

This paper proposes a centralized MTT method based on a state-of-the-art multi-sensor labeled multi-Bernoulli (LMB) filter in underwater multi-static networks with autonomous underwater vehicles (AUVs). The LMB filter can accurately extract the number of targets and trajectories from measurements affected by noise, missed detections, false alarms and port–starboard ambiguity. However, its complexity increases as the number of sensors increases. In addition, due to the time-varying underwater environment, AUV detection probabilities are time-varying, and their mismatches often lead to poor MTT performance. Consequently, we detail a robust multi-sensor LMB filter that estimates detection probabilities and multi-target states simultaneously in real time. Moreover, we derive an effective approximate form of the multi-sensor LMB filter using Kullback–Leibler divergence and develop an efficient belief propagation (BP) implementation of the multi-sensor LMB filter. Our method scales linearly with the number of AUVs, providing good scalability and low computational complexity. The proposed method demonstrates superior performance in underwater multi-AUV network MTT simulations.

A robust attribute-aware and real-time multi-target multi-camera tracking system using multi-scale enriched features and hierarchical clustering

A robust attribute-aware and real-time multi-target multi-camera tracking system using multi-scale enriched features and hierarchical clustering

A Distributed Underwater Multi-Target Tracking Algorithm Based on Two-Layer Particle Filter

Underwater multi-target tracking is one of the key technologies for military missions, including patrol and combat in the crucial area. Since the underwater environment is complex and targets’ trajectories may intersect when they are in a dense area, it is challenging to guarantee the precision of observed information. In order to provide high-precision underwater localization and tracking services over an underwater monitoring network, a dynamic network resource allocation mechanism and an underwater multi-target tracking algorithm based on a two-layer particle filter with distributed probability fusion (TLPF-DPF) are proposed. The position estimation model based on geometric constraints and the dynamic allocation mechanism of network resources based on prior position estimation are designed. Using the improved filtering algorithm with known initial states, the reliable tracking of multiple targets with trajectory intersection in a small area under complex noises is achieved. In the non-Gaussian environment, the average positioning error of TLPF-DPF is less by nearly 30% than alternative algorithms. When switching from a Gaussian environment to a non-Gaussian environment, the performance degradation of TLPF-DPF is less than 12%, which exhibits stability compared with other algorithms when targets are close to each other with crossing trajectories.

Active modulation of terahertz vortex beams by Dirac semimetals-based space–time-coding​ metasurface

Coding metasurfaces have developed rapidly in the last few years because of the digital thought. Recently, spatiotemporal metasurface have attracted considerable attention for the additional degrees of freedom they provide when manipulating electromagnetic waves in space and time domains. This study proposed Dirac semimetal based space–time-coding metasurfaces in the terahertz band. The conversion from a low- to high-bit programmable metasurface was achieved using a spatiotemporal metasurface, that is, 2-bit coding was equated to 3-bit coding. Space–time-coding metasurfaces facilitated the realization of the function of wide-band terahertz vortex beam generator switching according to the vortex phase arrangement, and were combined with the convolution theory for the successful realization of the modulation of terahertz vortex deflection and beam splitting under space–time coded modulation. The modulation of the equivalent 3-bit vortex beam produced under space–time coded modulation considerably reduced the gain of the side lobes and achieved precise modulation of the terahertz vortex beam. Moreover, the principle and method proposed in this study are also applicable to microwave and visible wavelengths. The results of this study are expected to result in promising applications in future terahertz high-data-capacity communication systems and multi-target precision tracking and guidance.

Collaborative route optimization and resource management strategy for multi-target tracking in airborne radar system

In this paper, a collaborative route optimization and resource management (CRO-RM) strategy is developed for multi-target tracking (MTT) in airborne radar system. The mechanism of the proposed CRO-RM strategy is to collaboratively coordinate the kinematic velocity, heading angle, transmit power, waveform pulselength and bandwidth of the airborne radar, in order to achieve accurate target state estimation while adhering to platform limitations and system resources budget constraints. The analytical expression for the Bayesian Cramér-Rao lower bound (BCRLB) on resources optimization and route planning parameters is derived and utilized as the optimization criterion for the CRO-RM strategy. It is shown that the resulting optimization problem is nonlinear and nonconvex since the above adaptable parameters are coupled in the optimization function and constraints. By exploiting the partition technique and cyclic minimization method, embed with the barrier function method for route planning, the semi-definite programming (SDP) for transmit power allocation and the finite grid search for waveform parameters selection, a partition-based four-step solution method is proposed to tackle the optimization problem effectively. Simulation results demonstrate that the proposed CRO-RM strategy achieves the best MTT performance compared with other benchmarks.

Group target tracking via jointly optimizing group partition and association

This paper considers the problem of indistinguishable group target tracking, where the individual targets within a group are closely spaced and resolvable, move in a coordinated manner, and the group splitting and merging may occur. In such a case, the performance of the commonly used multi-target tracking methods may be degraded. We focus on three aspects of the problem: deal with the group association, group splitting and merging, and derive the state estimates of the individual targets. First, in order to deal with the group partition and the group association globally, we propose a novel group target tracking method within the multiple hypothesis tracking (MHT) framework by extending the target partition and the measurement partition as parts of global hypotheses. An efficient algorithm is given to find the M-best group partition and association simultaneously. Second, we use the undirected graph to represent the interaction between groups and then obtain an identification method of the group splitting and merging. Third, in response to the state estimate of the group target, a group-individual Bayesian filter is designed to estimate the states of the virtual leader and the individual targets. Finally, numerical experiments are presented to illustrate the effectiveness of the proposed method.

Detection, Localization, and Tracking of Multiple MAVs With Panoramic Stereo Camera Networks

Malicious use of micro aerial vehicles (MAVs) has become a serious threat to public safety and personal privacy in recent years. Motivated by this problem, we propose a systematic approach to monitor the intrusion of malicious MAVs based on a novel type of panoramic stereo camera networks. Each sensing node of such a network consists of 16 lenses that can form a 360-degree panoramic vision system. The 16 lenses further form 8 pairs of stereo cameras that can directly localize aerial targets. The effective range for a sensing node localizing a MAV like DJI M300 could reach 80 meters, which is much farther than existing commercial stereo cameras. In terms of algorithms, we propose i) a novel visual MAV detection algorithm based primarily on motion features of MAVs, ii) an efficient stereo localization algorithm based on sparse feature points, and iii) robust multi-target tracking and trajectory fusion algorithm to fuse the observations of different sensing nodes. The effectiveness, robustness, and accuracy of the proposed algorithms together with the overall system have been verified by extensive experimental tests. To the best of our knowledge, this is the first systematic approach to detect, localize, and track unknown MAVs in the literature. Our approach provides a scalable solution to securely cover large areas of interest against malicious MAV intrusion. Note to Practitioners—Micro aerial vehicles (MAVs) have been widely used in many domains nowadays. However, they have also brought many safety problems. To monitor the intrusion of malicious MAVs, this paper proposes a novel type of panoramic stereo camera networks that can detect, localize, and track multiple MAVs simultaneously. Such a network consists of a number of sensing nodes and a central node. Each sensing node is able to detect, localize, and track multiple MAV targets. The role of the central node is to fuse the observations from multiple sensing nodes to generate more accurate trajectories of the MAV targets and in the meantime secure a large area in a coordinated way. This paper presents the details of the prototype of the system and the key algorithms therein.

Exact and Approximate Heterogeneous Bayesian Decentralized Data Fusion

In Bayesian peer-to-peer decentralized data fusion, the underlying distributions held locally by autonomous agents are frequently assumed to be over the same set of variables (homogeneous). This requires each agent to process and communicate the full global joint distribution, and thus leads to high computation and communication costs irrespective of relevancy to specific local objectives. This work formulates and studies heterogeneous decentralized fusion problems, defined as the set of problems in which either the communicated or the processed distributions describe different, but overlapping, random states of interest that are subsets of a larger full global joint state. We exploit the conditional independence structure of such problems and provide a rigorous derivation of novel exact and approximate conditionally factorized heterogeneous fusion rules. We further develop a new version of the homogeneous Channel Filter algorithm to enable conservative heterogeneous fusion for smoothing and filtering scenarios in dynamic problems. Numerical examples show more than $99.5\%$ potential communication reduction for heterogeneous channel filter fusion, and a multi-target tracking simulation shows that these methods provide consistent estimates while remaining computationally scalable.

Joint Power Allocation and Route Planning Scheme for Multitarget Tracking in Airborne Radar Network Under Multiuncertainty

In this article, a joint power allocation and route planning (JPARP) scheme is proposed for multitarget tracking in airborne radar network (ARN) in the presence of multiuncertainty. The main mechanism of the proposed scheme is to utilize the optimization technique to collaboratively plan route and allocate power for each airborne radar, with the aim of minimizing the power consumption of the ARN while satisfying the predetermined multiple target tracking (MTT) performance, inter-radar communication performance, and inter-radar safe distance. The Bayesian Cramer–Rao lower bound (BCRLB) and the communication data rate (CDR) are derived and adopted as metrics to evaluate the MTT performance and inter-radar communication performance, respectively. To ensure the robustness of the algorithm under multiuncertainty, we formulate the JPARP scheme as a chance-constraint programming (CCP) problem. To efficiently solve this no-convex and nondeterministic CCP problem, we relax it into an unconstrainted optimization problem. Then, a hybrid intelligent algorithm based on stochastic simulation and improved genetic algorithm (GA) is proposed to obtain a near-optimal solution. Simulation results are provided to demonstrate that the JPARP scheme is more effective in saving ARN power consumption and more robust under uncertainty than the conventional algorithm.

Tracking Multiple Unmanned Aerial Vehicles through Occlusion in Low-Altitude Airspace

In an intelligent multi-target tracking (MTT) system, the tracking filter cannot track multi-targets significantly through occlusion in a low-altitude airspace. The most challenging issues are the target deformation, target occlusion and targets being concealed by the presence of background clutter. Thus, the true tracks that follow the desired targets are often lost due to the occlusion of uncertain measurements detected by a sensor, such as a motion capture (mocap) sensor. In addition, sensor measurement noise, process noise and clutter measurements degrade the system performance. To avoid track loss, we use the Markov-chain-two (MC2) model that allows the propagation of target existence through the occlusion region. We utilized the MC2 model in linear multi-target tracking based on the integrated probabilistic data association (LMIPDA) and proposed a modified integrated algorithm referred to here as LMIPDA-MC2. We consider a three-dimensional surveillance for tracking occluded targets, such as unmanned aerial vehicles (UAVs) and other autonomous vehicles at low altitude in clutters. We compared the results of the proposed method with existing Markov-chain model based algorithms using Monte Carlo simulations and practical experiments. We also provide track retention and false-track discrimination (FTD) statistics to explain the significance of the LMIPDA-MC2 algorithm.

Improved multi-target tracking crossing paths in MIMO FMCW 8 × 16 radar system using a new hybrid AMC-JPDAF algorithm

Improved multi-target tracking crossing paths in MIMO FMCW 8 × 16 radar system using a new hybrid AMC-JPDAF algorithm

Multi-Target Tracking Based on a Combined Attention Mechanism and Occlusion Sensing in a Behavior-Analysis System.

Multi-object tracking (MOT) is a topic of great interest in the field of computer vision, which is essential in smart behavior-analysis systems for healthcare, such as human-flow monitoring, crime analysis, and behavior warnings. Most MOT methods achieve stability by combining object-detection and re-identification networks. However, MOT requires high efficiency and accuracy in complex environments with occlusions and interference. This often increases the algorithm's complexity, affects the speed of tracking calculations, and reduces real-time performance. In this paper, we present an improved MOT method combining an attention mechanism and occlusion sensing as a solution. A convolutional block attention module (CBAM) calculates the weights of space and channel attention from the feature map. The attention weights are used to fuse the feature maps to extract adaptively robust object representations. An occlusion-sensing module detects an object's occlusion, and the appearance characteristics of an occluded object are not updated. This can enhance the model's ability to extract object features and improve appearance feature pollution caused by the short-term occlusion of an object. Experiments on public datasets demonstrate the competitive performance of the proposed method compared with the state-of-the-art MOT methods. The experimental results show that our method has powerful data association capability, e.g., 73.2% MOTA and 73.9% IDF1 on the MOT17 dataset.

Bayesian detection and tracking of odontocetes in 3D from their echolocation clicks

To study behaviors of marine mammals in a nonintrusive manner, their bio-acoustical signals can be recorded by volumetric hydrophone arrays that provide time difference of arrival (TDOA) measurements for localization and tracking. Multi-target tracking (MTT) in 3-D using TDOA measurements from multiple sensors, however, must cope with non-linear measurement models and high-dimensional states. False alarms, missed detections and unknown data associations impose further challenges, often requiring human operators to annotate the data manually. We propose a data processing chain that automatically detects and tracks odontocetes from their echolocation clicks. The echolocation clicks are detected with a generalized cross-correlation that whitens the instrument noise. Two stages of tracking are performed using a tracking framework based on factor graphs and the sum-product algorithm (SPA). The odontocetes are first tracked in the TDOA domain to remove false alarms and then in the 3-D domain, fusing the tracked TDOAs across all sensors. To efficiently handle the considered non-linear and high-dimensional MTT scenario, particle flow is embedded in the SPA. According to simulation results, the proposed tracking method outperforms the existing approach using manual data annotation. Tracking of Cuvier’s beaked whales (Ziphius cavirostris), whose echolocation clicks are recorded by two volumetric hydrophone arrays, is demonstrated.

Multitarget Detection and Tracking by Mitigating Spot Jammer Attack in 77-GHz mm-Wave Radars: An Experimental Evaluation

Small form factor radar sensors at millimeter wavelengths find numerous applications in the industrial and automotive sectors. These radar sensors provide improved range resolution, good angular resolution, and enhanced Doppler resolution for short range and ultrashort ranges. However, it is challenging to detect and track the targets accurately when a radar is interfered by another radar. This article proposes an experimental evaluation of a 77-GHz IWR1642 radar sensor in the presence of a second 77-GHz AWR1642 radar sensor acting as a spot jammer. A real-time experiment is carried out by considering five different targets of various cross sections, such as a car, a larger size motorcycle, a smaller size motorcycle, a cyclist, and a pedestrian. The collected real-time data are processed by four different constant false alarm rate detectors, cell averaging (CA)-CFAR, ordered statistics (OS)-CFAR, greatest of CA (GOCA)-CFAR, and smallest of CA (SOCA)-CFAR. Following that, data from these detectors are fed into two different clustering algorithms (density-based spatial clustering of applications with noise (DBSCAN) and K-means), followed by the extended Kalman filter (EKF)-based tracker with global nearest neighbor (GNN) data association, which provide tracks of various targets with and without the presence of a jammer. Furthermore, four different metrics [tracks reported (TR), track segments (TSs), false tracks (FTs), and track loss (TL)] are used to evaluate the performance of various tracks generated for two clustering algorithms with four detection schemes. The experimental results show that the DBSCAN clustering algorithm outperforms the K-means clustering algorithm for many cases.

Passive acoustic localization and tracking of Rice’s whales (Balaenoptera ricei) in the northeastern Gulf of Mexico

The endangered Rice’s whale (Balaenoptera ricei) is endemic to the Gulf of Mexico and is also the Gulf’s only resident baleen whale. Limited knowledge of this species’ ecology combined with the high industrialization of this region raises serious conservation concerns. As part of an ongoing project to study the Rice’s whale in its core habitat, an array of moored passive acoustic monitoring sites has been near-continuously deployed in the northeastern Gulf of Mexico since May 2021. While detecting Rice’s whale calls in these data provides valuable insight into the species’ spatiotemporal distribution, further analyses to localize and track individual whales would allow better characterization of their acoustic behavior and movement patterns, possibly allowing for density estimation. Here, we present a method for automated passive acoustic localization and tracking of calling Rice’s whales. The two key components of this approach are (1) the use of opportunistic sound sources to time-synchronize recordings across sites given clock-drift between acoustic recorders and (2) the implementation of automatic multi-target tracking techniques based on the Gaussian mixture probability hypothesis density filter. Analyses of the first four months of data show potential for measuring basic behavioral parameters such as calling rate, source level, and average swim speed.

Framework for Network-Constrained Tracking of Cyclists and Pedestrians

The increase in perception capabilities of connected mobile sensor platforms (e.g., self-driving vehicles, drones, and robots) leads to an extensive surge of sensed features at various temporal and spatial scales. Beyond their traditional use for safe operation, available observations could enable to see how and where people move on sidewalks and cycle paths, to eventually obtain a complete microscopic and macroscopic picture of the traffic flows in a larger area. This paper proposes a new method for advanced traffic applications, tracking an unknown and varying number of moving targets (e.g., pedestrians or cyclists) constrained by a road network, using mobile (e.g., vehicles) spatially distributed sensor platforms. The key contribution in this paper is to introduce the concept of network bound targets into the multi-target tracking problem, and hence to derive a network-constrained multi-hypotheses tracker (NC-MHT) to fully utilize the available road information. This is done by introducing a target representation, comprising a traditional target tracking representation and a discrete component placing the target on a given segment in the network. A simulation study shows that the method performs well in comparison to the standard MHT filter in free space. Results particularly highlight network-constraint effects for more efficient target predictions over extended periods of time, and in the simplification of the measurement association process, as compared to not utilizing a network structure. This theoretical work also directs attention to latent privacy concerns for potential applications.

Variational Bayesian cardinalized probability hypothesis density filter for robust underwater multi-target direction-of-arrival tracking with uncertain measurement noise

The direction-of-arrival (DOA) tracking of underwater targets is an important research topic in sonar signal processing. Considering that the underwater DOA tracking is a typical multi-target problem under unknown underwater environment with missing detection, false alarm, and uncertain measurement noise, a robust underwater multi-target DOA tracking method for uncertain measurement noise is proposed. First, a kinematic model of the multiple underwater targets and bearing angle measurement model with missing detection and false alarms are established. Then, the multi-target DOA tracking algorithm is derived by using the cardinalized probability hypothesis density (CPHD) filter, the performance of which largely depends on the accuracy of the parameter of measurement noise variance. In addition, the variational Bayesian approach is used to adaptively estimate the uncertain measurement of noise variance for each measurement of target in the real time of tracking. Thus, the robust underwater multi-target DOA tracking is carried out. Finally, comprehensive experimental validations and discussions are made to prove that the proposed algorithm can provide robust DOA tracking in the multi-target tracking scenario with uncertain measurement noise.

Appearance Awared Detector for MOT: An Enhanced ReID Branch for Tracking Memorize

The traditional ByteTrack approach to multi-target tracking, focusing on simple, effective algorithms. It performs well in short-term multi-target tracking tasks with 80.3 MOTA, 77.3 IDF1 and 63.1 HOTA on MOT17 30 FPS and is currently ranked number one in the MOTChallenge rankings. But for situations where the camera is moving, or for completing target recovery tasks after a brief loss of position and track information, conventional MOT modules often do not perform as well as they should. To make bytetrack perform better when the motion background pattern is more complex, we used the re-identification module and added appearance information to enhance ByteTrack’s MOT process. We verified our improvement on MOT17, which achieved higher results than the original on the metrics Precision and Recall, and was also more suitable for more complex scenarios.

A General Framework for Reconstructing Full-Sample Continuous Vehicle Trajectories Using Roadside Sensing Data

Vehicle trajectory data play an important role in autonomous driving and intelligent traffic control. With the widespread deployment of roadside sensors, such as cameras and millimeter-wave radar, it is possible to obtain full-sample vehicle trajectories for a whole area. This paper proposes a general framework for reconstructing continuous vehicle trajectories using roadside visual sensing data. The framework includes three modules: single-region vehicle trajectory extraction, multi-camera cross-region vehicle trajectory splicing, and missing trajectory completion. Firstly, the vehicle trajectory is extracted from each video by YOLOv5 and DeepSORT multi-target tracking algorithms. The vehicle trajectories in different videos are then spliced by the vehicle re-identification algorithm fused with lane features. Finally, the bidirectional long-short-time memory model (LSTM) based on graph attention is applied to complete the missing trajectory to obtain the continuous vehicle trajectory. Measured data from Donghai Bridge in Shanghai are applied to verify the feasibility and effectiveness of the framework. The results indicate that the vehicle re-identification algorithm with the lane features outperforms the vehicle re-identification algorithm that only considers the visual feature by 1.5% in mAP (mean average precision). Additionally, the bidirectional LSTM based on graph attention performs better than the model that does not consider the interaction between vehicles. The experiment demonstrates that our framework can effectively reconstruct the continuous vehicle trajectories on the expressway.

Factored Multi-Agent Soft Actor-Critic for Cooperative Multi-Target Tracking of UAV Swarms

In recent years, significant progress has been made in the multi-target tracking (MTT) of unmanned aerial vehicle (UAV) swarms. Most existing MTT approaches rely on the ideal assumption of a pre-set target trajectory. However, in practice, the trajectory of a moving target cannot be known by the UAV in advance, which poses a great challenge for realizing real-time tracking. Meanwhile, state-of-the-art multi-agent value-based methods have achieved significant progress for cooperative tasks. In contrast, multi-agent actor-critic (MAAC) methods face high variance and credit assignment issues. To address the aforementioned issues, this paper proposes a learning-based factored multi-agent soft actor-critic (FMASAC) scheme under the maximum entropy framework, where the UAV swarm is able to learn cooperative MTT in an unknown environment. This method introduces the idea of value decomposition into the MAAC setting to reduce the variance in policy updates and learn efficient credit assignment. Moreover, to further increase the detection tracking coverage of a UAV swarm, a spatial entropy reward (SER), inspired by the spatial entropy concept, is proposed in this scheme. Experiments demonstrated that the FMASAC can significantly improve the cooperative MTT performance of a UAV swarm, and it outperforms existing baselines in terms of the mean reward and tracking success rates. Additionally, the proposed scheme scales more successfully as the number of UAVs and targets increases.