Multi-target Tracking Method Research Articles

STCA: High-Altitude Tracking via Single-Drone Tracking and Cross-Drone Association

In this paper, we introduce a high-altitude multi-drone multi-target (HAMDMT) tracking method called STCA, which aims to collaboratively track similar targets that are easily confused. We approach this challenge by categorizing the HAMDMT tracking into two principal tasks: Single-Drone Tracking and Cross-Drone Association. Single-Drone Tracking employs positional and appearance data vectors to overcome the challenges arising from similar target appearances within the field of view of a single drone. The Cross-Drone Association employs image-matching technology (LightGlue) to ascertain the topological relationships between images captured by disparate drones, thereby accurately determining the associations between targets across multiple drones. In Cross-Drone Association, we enhanced LightGlue into a more efficacious method, designated T-LightGlue, for cross-drone target tracking. This approach markedly accelerates the tracking process while reducing indicator dropout. To narrow down the range of targets involved in the cross-drone association, we develop a Common View Area Model based on the four vertices of the image. Considering to mitigate the occlusion encountered by high-altitude drones, we design a Local-Matching Model that assigns the same ID to the mutually nearest pair of targets from different drones after mapping the centroids of the targets across drones. The MDMT dataset is the only one captured by a high-altitude drone and contains a substantial number of similar vehicles. In the MDMT dataset, the STCA achieves the highest MOTA in Single-Drone Tracking, with the IDF1 system achieving the second-highest performance and the MDA system achieving the highest performance in Cross-Drone Association.

Research on the multitarget 3D trajectory tracking method of Thalassodes immissaria in the thermal infrared region based on YOLOX-GMM and SORT-Pest.

Studying the behavioral responses and movement trajectories of insects under different stimuli is crucial for developing more effective biological control measures. Therefore, accurately obtaining the movement trajectories and behavioral parameters of insects in three-dimensional space is essential. This study used the litchi pest Thalassodes immissaria as the research object. A special binocular vision observation system was designed for nighttime movement. A thermal infrared camera was used for video recording of T. immissaria in a lightless environment. Moreover, a multi-object tracking method based on the YOLOX-GMM and SORT-Pest algorithms was proposed for tracking T. immissaria in thermal infrared images. By obtaining the central coordinates of the two T. immissaria in the video, target matching and 3D trajectory reconstruction in the parallel binocular system were achieved. Error analysis of the T. immissaria detection and tracking model, as well as the 3D reconstruction model, showed that the average accuracy of T. immissaria detection reached 89.6%, tracking accuracy was 96.9%, and the average error of the reconstructed 3D spatial coordinates was 15 mm. These results indicate that the method can accurately obtain the 3D trajectory and motion parameters of T. immissaria. Such data can greatly contribute to researchers' comprehensive understanding of insect behavioral patterns and habits, providing important support for more targeted control strategies.

Research on the method of anti-occlusion of surgical instrument tracking based on multi-camera module information fusion

The traditional optical tracking system is easily affected by occlusion and a narrow field of vision, which leads to the failure of surgical instrument tracking. To solve this problem, a multi-target tracking method based on multi-camera module information fusion is proposed. Firstly, a multi-target tracking block is installed at the end of the surgical instrument. A method of solving the end position of surgical instruments based on multi-object tracking is proposed to improve the success rate of surgical instrument tracking. Secondly, motion blur is easy to occur in surgical instrument tracking. A method to reduce the tracking failure caused by motion blur is proposed to improve the sharp edge of the image. Then, a method of information fusion of multi-camera module is proposed to track surgical instruments. Finally, experiments are designed to verify the stability and accuracy of the proposed method. The tracking accuracy is 1.89 ± 0.24 mm.

An Improved Multi-Target Tracking Method for Space-Based Optoelectronic Systems

An Improved Multi-Target Tracking Method for Space-Based Optoelectronic Systems

Pedestrian tracking method based on S-YOFEO framework in complex scene

A real-time stable multi-target tracking method based on the enhanced You Only Look Once-v8 (YOLOv8) and the optimized Simple Online and Realtime Tracking with a Deep association metric (DeepSORT) for multi-target tracking (S-YOFEO) is proposed with the aim of addressing the issue of target ID transformation and loss caused by the increase of practical background complexity. For the purpose of further enhancing the representation of small-scale features, a small target detection head is first introduced to the detection layer of YOLOv8 in this paper with the aim of collecting more detailed information by increasing the detection resolution of YOLOv8. Secondly, the Omni-Scale Network (OSNet) feature extraction network is implemented to enable accurate and efficient fusion of the extracted complex and comparable feature information, taking into account the restricted computational power of DeepSORT’s original feature extraction network. Again, a novel adaptive forgetting Kalman filter algorithm (FSA) is devised to enhance the precision of model prediction and the effectiveness of parameter updates to adjust to the uncertain movement speed and trajectory of pedestrians in real scenarios. Following that, an accurate and stable association matching process is obtained by substituting Efficient-Intersection over Union (EIOU) for Complete-Intersection over Union (CIOU) in DeepSORT to boost the convergence speed and matching effect during association matching. Last but not least, One-Shot Aggregation (OSA) is presented as the trajectory feature extractor to deal with the various noise interferences in the complex scene. OSA is highly sensitive to information of different scales, and its one-time aggregation property substantially decreases the computational overhead of the model. According to the trial results, S-YOFEO has made some developments as its precision can reach 78.2% and its speed can reach 56.0 frames per second (FPS).

Radar nonlinear multi-target tracking method with parallel PHD filter

Since probability hypothesis density (PHD) filters do not need explicit data association, they have recently been widely used in radar multi-target tracking (MTT). However, in existing PHD filters, sampling times are generally considered the same for all targets. Due to the limitation of antenna beam width in radar applications, the same sampling time for all targets will lead to a mismatch between the predicted data and measurement data, reducing the accuracy of radar MTT. In order to eliminate the estimation error with less computational cost, a radar nonlinear multi-target tracking method with a parallel PHD filter is proposed in this article. The measurement area is divided into several subspaces according to the beam width of the radar antenna, and the PHD of all subspaces is calculated in parallel. Then, multi-feature information in radar echo assists tracking and improves real-time performance. Experimental results in various scenarios illustrate that the proposed method can eliminate the estimation errors introduced by sampling time diversity at the cost of less computation cost, especially in cluttered environments.

An Integrated Detection and Multi-Object Tracking Pipeline for Satellite Video Analysis of Maritime and Aerial Objects

Optical remote sensing videos, as a new source of remote sensing data that has emerged in recent years, have significant potential in remote sensing applications, especially national defense. In this paper, a tracking pipeline named TDNet (tracking while detecting based on a neural network) is proposed for optical remote sensing videos based on a correlation filter and deep neural networks. The pipeline is used to simultaneously track ships and planes in videos. There are many target tracking methods for general video data, but they suffer some difficulties in remote sensing videos with low resolution and those influenced by weather conditions. The tracked targets are usually misty. Therefore, in TDNet, we propose a new multi-target tracking method called MT-KCF and a detecting-assisted tracking (i.e., DAT) module to improve tracking accuracy and precision. Meanwhile, we also design a new target recognition (i.e., NTR) module to recognise newly emerged targets. In order to verify the performance of TDNet, we compare our method with several state-of-the-art tracking methods on optical video remote sensing data sets acquired from the Jilin No. 1 satellite. The experimental results demonstrate the effectiveness and the state-of-the-art performance of the proposed method. The proposed method can achieve more than 90% performance in terms of precision for single-target tracking tasks and more than 85% performance in terms of MOTA for multi-object tracking tasks.

Robust cardinalized probability hypothesis density filter based underwater multi-target direction-of-arrival tracking with uncertain measurement noise

Direction-of-arrival (DOA) tracking of underwater targets is an important research topic in sonar signal processing. Considering the underwater DOA tracking is a typical multi-target scenario with uncertain measurement noise caused by unknown underwater environment, a robust underwater multi-target DOA tracking method with uncertain measurement noise is proposed by implementing a spatial matrix filtering (SMF) technique in the framework of the cardinalized probability hypothesis density (CPHD) filter. Firstly, the kinematic model of underwater targets and the measurement model based on the received signal of the hydrophone array (array signal) are established. Thus, the multi-target DOA tracking algorithm is derived by using the CPHD filter based on the established model. Moreover, to avoid the effect of the uncertain measurement noise, the SMF technique is implemented to process the array signal to extract the signal in the bearing angle of the target. Then, the output of the SMF is substituted into the CPHD filter. In this manner, the robust underwater multi-target DOA tracking method for uncertain measurement noise scenario is proposed. Finally, comprehensive simulations and experimental data processing are performed to prove the robustness of the proposed method in the scenario of multi-target DOA tracking with uncertain measurement noise. Compared with the existing DOA estimation and DOA tracking methods, the proposed methods significantly improve the robustness and the precision of the DOA of the underwater target.

Building the method for multi-target tracking based on the combination of PHD filter and JPDA filter using particle filter in 3D mixed coordinate system

Changing the target number, non-linear measurement models and non-Gaussian noise faces a challenge to multi-target tracking problems which are factors affecting the accuracy, execution time and deciding the success of the method as well. In this paper, the authors present a method to solve these problems. Wherein, the motion of targets is represented in the mixed coordinate system 3D base on combining PHD (Probability Hypothesis Density) and JPDA (Joint Probability Data Association). This method can track multiple targets in the most general case, that is to change the target number, system model and measurement model which is non-linear as the noise is non-Gaussian. The result of this work can be applied to the real-time response system when the targets are moving in close distances with rapid maneuvering.

Evaluation and Analysis of the Implementation Effects in Practical-Course Blended Learning Based on Virtual Reality Technology

Practical-course blended learning based on virtual reality (VR) technology combines VR with traditional practical teaching, which provides students with a more diverse and personalized learning experience. The existing evaluation and analysis methods of teaching model implementation effects have shortcomings. Although VR technology plays an important role in practical-course blended learning, excessive reliance on technical means may lead to limitations in evaluation methods. Therefore, this study aimed to explore the evaluation and analysis of implementation effects of VR-based practical-course blended learning. Different types of teaching models were represented. A robust multi-target collaborative tracking method based on variational Bayesian inference was applied to track and evaluate the implementation effects of practical-course blended learning. The experimental results verified the effectiveness of the proposed method and explored the impact of different teaching models on the average scores and score stability of evaluation methods. Analysis results of score data showed that the assisted-teaching model improved the homework performance of students and the blended-learning model improved the performance of students in tests and final exams, while the complete teaching model performed more balanced in all aspects.

Multitarget-Tracking Method Based on the Fusion of Millimeter-Wave Radar and LiDAR Sensor Information for Autonomous Vehicles.

Multitarget tracking based on multisensor fusion perception is one of the key technologies to realize the intelligent driving of automobiles and has become a research hotspot in the field of intelligent driving. However, most current autonomous-vehicle target-tracking methods based on the fusion of millimeter-wave radar and lidar information struggle to guarantee accuracy and reliability in the measured data, and cannot effectively solve the multitarget-tracking problem in complex scenes. In view of this, based on the distributed multisensor multitarget tracking (DMMT) system, this paper proposes a multitarget-tracking method for autonomous vehicles that comprehensively considers key technologies such as target tracking, sensor registration, track association, and data fusion based on millimeter-wave radar and lidar. First, a single-sensor multitarget-tracking method suitable for millimeter-wave radar and lidar is proposed to form the respective target tracks; second, the Kalman filter temporal registration method and the residual bias estimation spatial registration method are used to realize the temporal and spatial registration of millimeter-wave radar and lidar data; third, use the sequential m-best method based on the new target density to find the track the correlation of different sensors; and finally, the IF heterogeneous sensor fusion algorithm is used to optimally combine the track information provided by millimeter-wave radar and lidar, and finally form a stable and high-precision global track. In order to verify the proposed method, a multitarget-tracking simulation verification in a high-speed scene is carried out. The results show that the multitarget-tracking method proposed in this paper can realize the track tracking of multiple target vehicles in high-speed driving scenarios. Compared with a single-radar tracker, the position, velocity, size, and direction estimation errors of the track fusion tracker are reduced by 85.5%, 64.6%, 75.3%, and 9.5% respectively, and the average value of GOSPA indicators is reduced by 19.8%; more accurate target state information can be obtained than a single-radar tracker.

Traffic Flow Video Image Recognition and Analysis Based on Multi-Target Tracking Algorithm and Deep Learning

Traffic flow parameters are an important data support for the research and development of several technologies in the intelligent transportation system. Therefore, accurate and real-time estimation of traffic flow is particularly important for urban traffic. In this study, a real-time traffic flow detection system framework was constructed based on video image collection and analysis. According to the vehicle detection and tracking results, a traffic flow parameter estimation model and an improved LSTM network are proposed for spatiotemporal counting feature recognition. The results conclude that the developed framework can estimate the traffic flow density and count vehicles, as well as estimate the traffic flow velocity and traffic volume to estimate and optimize traffic flow, respectively. Additionally, the simulation results show that the proposed method can not only counts the two-way traffic vehicles quickly and accurately, but also avoids the use of the complex multi-target tracking method to spatiotemporal correlation of a single target, increases the speed and accuracy of the spatiotemporal information processing procedure, and has stronger scene adaptability.

Bayesian detection and tracking of odontocetes in 3-D from their echolocation clicks.

Localization and tracking of marine animals can reveal key insights into their behaviors underwater that would otherwise remain unexplored. A promising nonintrusive approach to obtaining location information of marine animals is to process their bioacoustic signals, which are passively recorded using multiple hydrophones. In this paper, a data processing chain that automatically detects and tracks multiple odontocetes (toothed whales) in three dimensions (3-D) from their echolocation clicks recorded with volumetric hydrophone arrays is proposed. First, the time-difference-of-arrival (TDOA) measurements are extracted with a generalized cross-correlation that whitens the received acoustic signals based on the instrument noise statistics. Subsequently, odontocetes are tracked in the TDOA domain using a graph-based multi-target tracking (MTT) method to reject false TDOA measurements and close gaps of missed detections. The resulting TDOA estimates are then used by another graph-based MTT stage that estimates odontocete tracks in 3-D. The tracking capability of the proposed data processing chain is demonstrated on real acoustic data provided by two volumetric hydrophone arrays that recorded echolocation clicks from Cuvier's beaked whales (Ziphius cavirostris). Simulation results show that the presented MTT method using 3-D can outperform an existing approach that relies on manual annotation.

Cross-Camera Multi-Object Tracking based on Person Re-Identification and Spatial-Temporal Constraints

In order to reduce the influence of occlusion on the overall feature representation of tracks and improve the accuracy of track correlation between cameras, this paper proposes a cross camera multi-target tracking method based on person appearance and spatial-temporal constraints: a new cross-camera multi-object tracking framework is constructed. Then the person spatial-temporal probability model is established. Finally, the spatial-temporal probability model and the person appearance similarity are jointly measured and the person trajectory correlation under cross-camera is completed by using data correlation; Comparative experiments on the dataset proved that the method is effective.

Variational Bayesian cardinalized probability hypothesis density filter for robust underwater multi-target direction-of-arrival tracking with uncertain measurement noise

The direction-of-arrival (DOA) tracking of underwater targets is an important research topic in sonar signal processing. Considering that the underwater DOA tracking is a typical multi-target problem under unknown underwater environment with missing detection, false alarm, and uncertain measurement noise, a robust underwater multi-target DOA tracking method for uncertain measurement noise is proposed. First, a kinematic model of the multiple underwater targets and bearing angle measurement model with missing detection and false alarms are established. Then, the multi-target DOA tracking algorithm is derived by using the cardinalized probability hypothesis density (CPHD) filter, the performance of which largely depends on the accuracy of the parameter of measurement noise variance. In addition, the variational Bayesian approach is used to adaptively estimate the uncertain measurement of noise variance for each measurement of target in the real time of tracking. Thus, the robust underwater multi-target DOA tracking is carried out. Finally, comprehensive experimental validations and discussions are made to prove that the proposed algorithm can provide robust DOA tracking in the multi-target tracking scenario with uncertain measurement noise.

An Improved Multiple-Target Tracking Scheme Based on IGGM–PMBM for Mobile Aquaculture Sensor Networks

The Poisson multi-Bernoulli Mixture (PMBM) filter, as well as its variants, is a popular and practical multitarget tracking algorithm. There are some pending problems for the standard PMBM filter, such as unknown detection probability, random target newborn distribution, and high energy consumption for limited computational and processing capacity in sensor networks. For the sake of accommodating these existing problems, an improved multitarget tracking method based on a PMBM filter with adaptive detection probability and adaptive newborn distribution is proposed, accompanied by an associated distributed fusion strategy to reduce the computational complexities. Firstly, gamma (GAM) distribution is introduced to present the augmented state of unknown and changing target detection probability. Secondly, the intensity of newborn targets is adaptively derived from the inverse gamma (IG) distribution based on this augmented state. Then, the measurement likelihood is presented as a gamma distribution for the augmented state. On these bases, the detailed recursion and closed-form solutions to the proposed filter are derived by means of approximating the intensity of target birth and potential targets to an inverse gamma Gaussian mixture (IGGM) form and the density of existing Bernoulli components to a single IGGM form. Moreover, the associated distributed fusion strategy generalized covariance intersection (GCI), whose target states are measured by multiple sensors according to their respective fusion weights, is applied to a large-scale aquaculture tracking network. Comprehensive experiments are presented to verify the effectiveness of this IGGM–PMBM method, and comparisons with other multitarget tracking filters also demonstrate that tracking behaviors are largely improved; in particular, tracking energy consumption is reduced sharply, and tracking accuracy is relatively enhanced.

Shallow sea matching field continuous tracking method based on trajectory Poisson multi-Bernoulli hybrid filter

In the shallow water waveguide, matched field tracking methods use the continuity of the peak position of the moving source and the disorder of pseudo-peaks on the sequential ambiguity surfaces to track the underwater source trajectory. However, owing to the dual influence of the space-time fluctuating shallow water waveguide environment and the complex sources motion scene, the existing matching field tracking methods are prone to track interruption, switches and false track phenomena, leading to discontinuous tracking results. Using the consistency between the peak position distance likelihood and the peak amplitude likelihood of sequential ambiguity surfaces, a continuous matched field tracking method is proposed based on the trajectory Poisson multi-Bernoulli mixture filter in this paper. The proposed method is applied to SWellEx-96 experimental data, and the tracking performance is measured by the linear programming metric. The results show that compared with the existing matching field tracking method and multi-target tracking method via random finite set, the proposed method achieves continuous tracking and accurate quantity estimation of moving sources trajectory. Among them, the prediction step and updating step in the trajectory space can avoid the phenomenon of trajectory interruption and switches in unvoiced periods.

Short-term memory and CenterTrack based vehicle-related multi-target tracking method

Short-term memory and CenterTrack based vehicle-related multi-target tracking method

An Improved End-to-End Multi-Target Tracking Method Based on Transformer Self-Attention

Current multi-target multi-camera tracking algorithms demand increased requirements for re-identification accuracy and tracking reliability. This study proposed an improved end-to-end multi-target tracking algorithm that adapts to multi-view multi-scale scenes based on the self-attentive mechanism of the transformer’s encoder–decoder structure. A multi-dimensional feature extraction backbone network was combined with a self-built raster semantic map which was stored in the encoder for correlation and generated target position encoding and multi-dimensional feature vectors. The decoder incorporated four methods: spatial clustering and semantic filtering of multi-view targets; dynamic matching of multi-dimensional features; space–time logic-based multi-target tracking, and space–time convergence network (STCN)-based parameter passing. Through the fusion of multiple decoding methods, multi-camera targets were tracked in three dimensions: temporal logic, spatial logic, and feature matching. For the MOT17 dataset, this study’s method significantly outperformed the current state-of-the-art method by 2.2% on the multiple object tracking accuracy (MOTA) metric. Furthermore, this study proposed a retrospective mechanism for the first time and adopted a reverse-order processing method to optimize the historical mislabeled targets for improving the identification F1-score (IDF1). For the self-built dataset OVIT-MOT01, the IDF1 improved from 0.948 to 0.967, and the multi-camera tracking accuracy (MCTA) improved from 0.878 to 0.909, which significantly improved the continuous tracking accuracy and reliability.

A Video-Based Real-Time Tracking Method for Multiple UAVs in Foggy Weather

Aiming at the real-time tracking problem of multiple unmanned aerial vehicles (UAVs) based on video under fog conditions, we propose a multitarget real-time tracking method that combines the Deepsort algorithm with detection based on improved dark channel defogging and improved You Only Look Once version 5 (YOLOv5) algorithm. The contributions of this paper are as follows: 1. For the multitarget tracking problem under fog interference, a multialgorithm combination method is proposed. 2. By optimizing dark channel defogging, the complexity of the original algorithm is reduced from On2 to On, which simplifies the processing time of the defogging algorithm. 3. The YOLOv5 network structure is optimized so that the network can synchronously reduce the detection time while maintaining high-precision detection. 4. The amount of algorithm processing through image size compression is reduced, and the real-time performance under high-precision tracking is improved. In the experiments conducted, the proposed method improved tracking precision by 36.1% and tracking speed by 39%. The average time of tracking per image frame was 0.036s, satisfying the real-time tracking of multiple UAVs in foggy weather.