Joint Probabilistic Data Association Research Articles

Resilient Cooperative Localization Based on Factor Graphs for Multirobot Systems

With the advancement of intelligent perception in multirobot systems, cooperative localization in dynamic environments has become a critical component. However, existing multirobot cooperative localization systems still fall short in meeting high-precision localization requirements in Global Navigation Satellite System (GNSS)-denied environments. In this paper, we propose a factor-graph-based resilient cooperative localization (FG-RCL) algorithm for multirobot systems. This algorithm integrates measurements from visual sensors and Ultra-WideBand (UWB) to achieve accurate cooperative state estimation—overcoming the visibility issues of visual sensors within limited fields of view. We utilize the Joint Probabilistic Data Association (JPDA) algorithm to calculate the corresponding probabilities of multiple visual detection measurements between robots and assign them to their respective edges in the factor graph, thereby addressing the data association challenges in visual detection measurements. Finally, simulation results demonstrate that the proposed algorithm significantly reduces the influence of visual detection measurement interference on the performance of cooperative localization. Experimental results indicate that the proposed algorithm outperforms UWB-based and vision-based methods in terms of localization accuracy. The system is implemented using a factor-graph-based optimization approach, and it exhibits scalability and enables plug-and-play for sensors. Furthermore, it demonstrates resilience in abnormal situations.

Cauchy–Schwarz Divergence-Based Set Joint Probabilistic Data Association Filter for Tracking Multiple Objects in Cluttered Environment

Cauchy–Schwarz Divergence-Based Set Joint Probabilistic Data Association Filter for Tracking Multiple Objects in Cluttered Environment

AFJPDA: A Multiclass Multi-Object Tracking with Appearance Feature-Aided Joint Probabilistic Data Association

This study addresses a multiclass multi-object tracking problem in consideration of clutters in the environment. To alleviate issues with clutters, we propose the appearance feature-aided joint probabilistic data association filter. We also implemented simple adaptive gating logic for the computational efficiency and track maintenance logic, which can save the lost track for re-association after occlusion or missed detection. The performance of the proposed algorithm was evaluated against a state-of-the-art multi-object tracking algorithm using both multiclass multi-object simulation and real-world aerial images. The evaluation results indicate significant performance improvement of the proposed method against the benchmark state-of-the-art algorithm, especially in terms of reduction in identity switches and fragmentation.

Domain-knowledge-aided airborne ground moving targets tracking

This paper investigates the problem of traffic surveillance using an unmanned aerial vehicle (UAV) and proposes a domain-knowledge-aided airborne ground moving target tracking algorithm. To improve the accuracy of multiple target tracking, the proposed algorithm incorporates domain knowledge into the joint probabilistic data association (JPDA) filter as state constraints. The domain knowledge considered in this paper includes both road information extracted from a given map and local traffic regulations. Conventional track update method is modified to enhance the capability of recognition of temporarily track loss. A variable structure multiple model (VS-MM) method is developed to assign the road segment to a given target. The main contribution of this paper is that we establish a comprehensive framework for target tracking with the aid of domain knowledge and analyze the effect of diverse constraints on target tracking performance, while the limitations involve only 2-D target behaviors are considered, and the sophisticated interactions between targets are neglected. The effectiveness of proposed algorithm is demonstrated through extensive numerical simulations.

Research on multi-target data association of the infrared fish-eye warning system

The multi-target data association method is studied in order to realize multi-target tracking in infrared fish-eye warning system. The Neural Joint Probabilistic Data Association (NJPDA) algorithm is analyzed. It is found that the NJPDA algorithm only considers the distance information between the measurement and the target in the data association process, and its tracking accuracy needs to be further improved. Therefore, a new method fused with direction information is proposed based on the NJPDA algorithm. The proposed algorithm defines the concept of direction difference, introduces the direction information of target motion, and modifies the likelihood function by Gaussian weighting method, so as to fuse the direction information of target motion into the calculation of data interconnection probability. Experimental results demonstrate that the tracking success rate of the proposed algorithm is nearly 10 % higher than that of JPDA and NJPDA algorithms and its consuming time meets the real-time requirement of the infrared fish-eye warning system.

Impact of moving target on underwater positioning by using state measurement

The localization of moving targets in an underwater acoustic wireless sensor network (UAWSN) is inaccurate due to the various underwater forces (viscous, hydrodynamic forces, perturbation of underwater). The false measurements in the sensor network cause position errors and velocity errors which disrupt the localization of the moving target. A randomly fluctuated spillover effect is introduced in the present paper. The absorption losses generated due to the spillover effect cause false measurements of the moving target. Theorem 1 describes the genesis of these absorption losses and their consequences in UAWSN. The measurements from each moving target in the presence of absorption losses are formulated in the elliptical region. A joint probabilistic data association (JPDA) method is proposed to quantify the false measurements in the elliptical region. A moving target state estimation (MTSE) algorithm is proposed to eliminate the false measurements from the moving targets and to measure the localization of moving targets with the help of the propagation speed of targets. The theoretical measurements of position RMSE and velocity RMSE are verified with standard methods. The proposed MTSE method improves the localization performance of the moving targets by 29.42 % and reduces 32.16 % of position errors and 36.23 % of velocity errors up to 550 ​m. The proposed algorithm will be useful for the sub-aquatic Internet of underwater things (IoUT).

ADVANCEMENTS IN OBJECT DETECTION AND TRACKING ALGORITHMS: AN OVERVIEW OF RECENT PROGRESS

In the realm of computer vision, object detection and tracking constitute fundamental tasks, and recent years have borne witness to astounding advancements attributed to the integration of deep learning techniques. This paper aims to provide a comprehensive overview of the remarkable progress achieved in the domain of object detection and tracking algorithms, shedding light on their profound implications for accuracy, speed, and practical applications in the real world. Advances in object detection have been primarily driven by the adoption of Convolutional Neural Networks (CNNs). Prominent models such as Faster R-CNN, YOLO (You Only Look Once), and SSD (Single Shot Multi-Box Detector) have emerged, significantly enhancing the precision of object identification. Additionally, efficient detectors like Efficient Det and Mobile Net have emerged, striking a balance between accuracy and computational efficiency, thereby enabling real-time applications, even on resource-constrained devices. For tracking, Multi-Object Tracking (MOT) algorithms have undergone improvements, incorporating graph-based approaches such as the Hungarian algorithm and Joint Probabilistic Data Association Filter (JPDAF). These advancements have enabled robust object tracking across video frames. This paper also delves into the synergy between deep learning and real-world applications, emphasizing the impact of these algorithms in domains like autonomous vehicles, surveillance systems, robotics, and augmented reality. KEYWORDS: Object detection, Tracking algorithms, Computer vision, Deep learning, Advancements, Real-world applications, Convolutional Neural Networks, Multi-Object Tracking, Autonomous vehicles, Surveillance, Robotics, Augmented reality.

Building the method for multi-target tracking based on the combination of PHD filter and JPDA filter using particle filter in 3D mixed coordinate system

Changing the target number, non-linear measurement models and non-Gaussian noise faces a challenge to multi-target tracking problems which are factors affecting the accuracy, execution time and deciding the success of the method as well. In this paper, the authors present a method to solve these problems. Wherein, the motion of targets is represented in the mixed coordinate system 3D base on combining PHD (Probability Hypothesis Density) and JPDA (Joint Probability Data Association). This method can track multiple targets in the most general case, that is to change the target number, system model and measurement model which is non-linear as the noise is non-Gaussian. The result of this work can be applied to the real-time response system when the targets are moving in close distances with rapid maneuvering.

New joint probabilistic data association algorithm based on variational Bayesian adaptive moment estimation

New joint probabilistic data association algorithm based on variational Bayesian adaptive moment estimation

Energy management of plug-in hybrid electric vehicles based on speed prediction fused driving intention and LIDAR

Driving intention and speed prediction are essential factors in the energy management of plug-in hybrid electric vehicles (PHEVs). This paper proposes an improved energy management strategy for the subject vehicle by speed prediction fused with driving intention and LIDAR data in a vehicle-following scenario. A driving intention recognition model is developed based on the gated recurrent unit (GRU), which takes the vehicle speed, throttle opening, and brake pedal force of the subject vehicle as input. Then integrating the LIDAR point cloud data and driving intention result of the subject vehicle to achieve more accurate speed prediction, where joint probabilistic data association and interacting multiple models methods are used to process LIDAR data. The more accurate speed prediction is then applied to design a prediction-informed adaptive equivalent consumption minimization strategy (PIA-ECMS) for real-time energy management optimization. Experimental results demonstrate the recognition accuracy of up to 88%, indicating that the driver’s driving intention can be identified effectively. The speed prediction has an error margin of no more than 5.9 km/h. Compared with existing adaptive ECMS without speed prediction, the proposed PIA-ECMS can enhance fuel economy by 1.3–2.7% while achieving better SOC charge sustainability.

Study on Multi-Heterogeneous Sensor Data Fusion Method Based on Millimeter-Wave Radar and Camera.

This study presents a novel multimodal heterogeneous perception cross-fusion framework for intelligent vehicles that combines data from millimeter-wave radar and camera to enhance target tracking accuracy and handle system uncertainties. The framework employs a multimodal interaction strategy to predict target motion more accurately and an improved joint probability data association method to match measurement data with targets. An adaptive root-mean-square cubature Kalman filter is used to estimate the statistical characteristics of noise under complex traffic scenarios with varying process and measurement noise. Experiments conducted on a real vehicle platform demonstrate that the proposed framework improves reliability and robustness in challenging environments. It overcomes the challenges of insufficient data fusion utilization, frequent leakage, and misjudgment of dangerous obstructions around vehicles, and inaccurate prediction of collision risks. The proposed framework has the potential to advance the state of the art in target tracking and perception for intelligent vehicles.

A novel joint multi-target detection and tracking approach based on Bayes joint decision and estimation

This paper proposes a novel joint decision and estimation (JDE) solution for the multi-target detection and tracking (MDT) problem. MDT aims to jointly detect the number of targets and estimate their states, which is essentially a JDE problem since detection and tracking are highly coupled. Thus, a joint solution which can utilize the coupling is preferable. However, the existing JDE approach has either poor performance or excessive design parameters without considering the MDT problem realities, i.e., the losses that different decisions may lead to. Therefore, we propose a compact conditional JDE (CCJDE)-based MDT method with less design parameters but superior performance. Specifically, we propose a CCJDE-based MDT risk which unifies the detection and tracking risks in a compact way. Then, we derive the joint detection and tracking solution accounting for their couplings, where the joint probabilistic data association filter is adopted due to its advantageous performance and the adaptability to the JDE framework. Then, an efficient CCJDE-MDT algorithm is developed. Besides, some parameter designing guidelines are presented by considering the MDT realities. Simulation results verify the effectiveness of the proposed CCJDE-MDT method, which outperforms the traditional decision-then-estimation in joint performance and also beats the existing recursive joint decision and estimation(RJDE) method in many cases.

An Overview of the PAKF-JPDA Approach for Elliptical Multiple Extended Target Tracking Using High-Resolution Marine Radar Data

Ground radar stations observing specific regions of interest nowadays provide detections in the form of point-clouds. This article focuses on a framework that consists of an elliptical multitarget tracker, referred to as Principal-Axes based Kalman Filter (PAKF)-based Joint Probabilistic Data Association (JPDA) (PAKF-JPDA), to enable maritime traffic monitoring. The framework touches on two major stages, target detection and target tracking. For the former, we employed a clustering approach and for the latter, we presented a data-association-based version of the PAKF tracker with an automatic track management functionality. The framework’s benefits are demonstrated when it is applied to the radar streaming in a harbor setting based on a homogeneous multisensor tracking system by comparing our results against their corresponding reference data with visualizations, including performance measures.

Correlation Filtering Algorithm of Infrared Spectral Data for Dim Target Tracking

The correlation filtering algorithm of infrared spectral data for dim and small target tracking is studied to improve the tracking accuracy of small and weak targets and to track small and weak targets in real time. After the image noise reduction processing by the mean shift filtering algorithm, the infrared small and weak target image data model is constructed by using the denoised infrared small and weak target image. And the brightness value and position of unknown small and weak targets are obtained. The tracking and measurement model of small and weak targets is built. The joint probabilistic data association algorithm is used to calculate the probability that each measurement is associated with its possible source targets, and the particle filter is used to update the tracking status of small and weak targets to achieve real-time tracking of small and weak targets. The experimental results show that the algorithm can enhance the edge contour information of small and weak images, so as to accurately track small and weak targets moving in any track, and has good real-time tracking and accuracy. There is a small deviation between the tracking track of weak and small targets tracked by the algorithm and the actual track, and the root mean square difference of tracking weak and small targets is within 2 pixels. In addition, the detection probability of detecting weak and small targets is less affected by SNR environmental factors.

Multiple target data-association algorithm based on Takagi–Sugeno intuitionistic fuzzy model

To solve the problem of data association in multiple target tracking in a densely cluttered environment, a multiple target data association algorithm based on Takagi–Sugeno (T-S) intuitionistic fuzzy model is proposed. In the proposed algorithm, the new multiple target T-S intuitionistic fuzzy model (MTTS-IFM) is constructed by incorporating intuitionistic fuzzy sets and intuitionistic fuzzy numbers. To identify the premise parameters for MTTS-IFM, a new intuitionistic index is defined and a modified form of the intuitionistic fuzzy C-means clustering algorithm is proposed. Furthermore, to solve the multiple target data association for the MTTS-IFM approach, the input variables of MTTS-IFM are defined using the maximum intuitionistic fuzzy entropy, and a data association approach is proposed to deal with the uncertainty between targets and measurements. Finally, simulation results show that the performance of the proposed algorithm is better than that of algorithms designed for multiple target tracking in either a clutter-free or cluttered environment. Moreover, the real-time performance was clearly better than that of the joint probabilistic data association filter (JPDAF) and slightly better than that of maximum entropy fuzzy (MEF-JPDAF), Fitzgerald-JPDAF, and a T-S intuitionistic fuzzy model (TS-FM).

Feature-Aided Passive Tracking of Noncooperative Multiple Targets Based on the Underwater Sensor Networks

Passive detection can work for a long time with low energy consumption in underwater surveillance. However, tracking unknown noncooperative targets with only direction angles is challenging, and the tracking performance of multiple targets is poor. Based on several passive sensors in the underwater sensor network (UWSN), a feature-aided state estimation method is used to start tracking unknown targets. The feature-aided joint probabilistic data association combined with the particle filter method is also proposed to improve the passive tracking performance of multiple targets. The track management and the fusion strategy are given to remove fake tracks and obtain correct trajectories of unknown targets. The simulation results show that the feature-aided method can quickly start and effectively track multiple noncooperative targets with passive sensors. Compared with other methods, the proposed method can track targets more accurately with the advantages of low energy consumption and less exposure in various environments.

Multiple targets video tracking based on extended kalman filter in combination with particle swarm optimization for intelligent applications

Multiple targets tracking is a major issue in the intelligent applications. Numerous methods have been presented for the multiple targets tracking to capture the targets trajectory in a video sequence in order to increase intelligence and reduce human error. In this paper, a method is proposed based on combining the Extended Kalman Filter (EKF) and Particle Swarm Optimization (PSO) to construct an intermediate tracker and track targets more accurately. The EKF solves targets collision problem, and PSO reduces the covariance of measured noise. Finally, the Joint Probabilistic Data Association (JPDA) filter is used to reduce the number of multiple hypotheses and create a one-to-one correspondence between targets and measurements. To detect targets, frames subtracting along with background modeling and canny edge detector are used. To reduce running time of the proposed method, number of video frames per second (fps) is reduced from 30 to 10 and the sampling rate is also reduced. Despite of this reduction, simulation reults of the proposed method show the multiple targets tracking with 98% accuracy at an acceptable running time compared to the similar methods. In addition, by using the proposed method, the number of assignment states is reduced in the targets tracking process. Overall, the proposed method not only can be used in the intelligent applications, but also in the video compression applications as well.

Design of Dynamic Multi-Obstacle Tracking Algorithm for Intelligent Vehicle

Environmental perception forms the basis of intelligent driving systems and is a prerequisite for path planning and vehicle control. Among them, dynamic multi-obstacle tracking is the key to environmental perception. In order to solve the problem of a large amount of correlation calculations and false correlations in the process of dynamic multi-obstacle tracking, and to obtain more accurate surrounding environment information, this paper first designs an obstacle data correlation algorithm based on improving the joint probabilistic data-association algorithm. Then, in order to solve the problem of obstacle movement mobility and the poor filtering effect of a single model, the interacting multiple model is designed to complete the filtering of multiple behavior patterns of obstacles. An obstacle state estimation algorithm based on the unscented Kalman filter is designed to solve the nonlinear problem of obstacle motion. Finally, an experimental prototype is built and tested. The results show that the data association algorithm designed in this paper can complete the data association of obstacles at different times, and there is no problem of obstacle loss and association error. The average running time of each frame is 51.63 ms. The result comparison between the proposed method and the traditional method shows that the proposed method is more effective.

Multiple User Intent Prediction Using Interacting Multiple Model Joint Probabilistic Data Association Filter

This paper presents a novel method for multi-user motion intent estimation when the motion is observed by a single sensor. A motion model is associated with each of the activities carried out by the operator and the end location of which is termed as a motion intent. Such modeling of intent is useful in human-robot collaborative tasks. The appropriate model selection is achieved via an interacting multiple model (IMM) filter. When the position measurements of multiple users originating from one sensor are close to each other, then the measurement to operator association becomes challenging. A joint probabilistic data association (JPDA) filter is employed to address this issue. The combined IMM and JPDA filter provides a way to infer the motion intent of each operator. Simulation results show that the IMM-JPDA filter tracks two target states reaching toward goal intent in the presence of clutter measurements originating from the Kinect sensor.

Plot Quality Aided Plot-to-Track Association in Dense Clutter for Compact High-Frequency Surface Wave Radar

Due to high false alarm rate and low positioning accuracy of compact high-frequency surface wave radar in moving vessel detection, false plot-to-track association often occurs during moving vessel tracking, thus leading to track fragmentation and false tracking. In order to address this problem, a plot quality evaluation method is proposed and applied to plot-to-track association. Firstly, the differences in spatial correlation of echo spectrum amplitudes and position among moving vessels, clutters, and noise on a range-Doppler map are analyzed, and a plot quality index integrating multi-directional gradient, local variance, and plot position probability is developed. Then, the plots labeled as low quality are removed to reduce both the negative impact of false alarms on plot-to-track association and the computational burden. Eventually, both plot quality index and kinematic parameters are used to calculate the association cost and determine the plot-track pairs during the plot-to-track association procedure. Experimental results with field data demonstrate that the proposed plot quality index can effectively distinguish moving vessel and other plots. Compared with both the nearest neighbor data association method and the joint probability data association method, the association accuracy of the proposed method is greatly improved and, thus, the tracking continuity is enhanced in dense clutter scenarios.