3D Target Tracking Research Articles

Radar-Based Target Tracking Using Deep Learning Approaches with Unscented Kalman Filter

Machine learning, a rapidly growing field, has attracted numerous researchers for its ability to automatically learn from and make predictions based on data. This manuscript presents an innovative approach to estimating the covariance matrix of noise in radar measurements for target tracking, resulting from collaborative efforts. Traditionally, researchers have assumed that the covariance matrix of noise in sonar measurements is present in the vast majority of literature related to target tracking. On the other hand, this research aims to estimate it by employing deep learning algorithms with noisy measurements in range, bearing, and elevation from radar sensors. This collaborative approach, involving multiple disciplines, provides a more precise and accurate covariance matrix estimate. Additionally, the unscented Kalman filter was combined with the gated recurrent unit, multilayer perceptron, convolutional neural network, and long short-term memory to accomplish the task of 3D target tracking in an airborne environment. The quantification of the results was achieved through the use of Monte Carlo simulations, which demonstrated that the convolutional neural network performed better than any other approach. The system was simulated using a Python program, and the proposed method offers higher accuracy and faster convergence time than conventional target tracking methods. This is a demonstration of the potential that collaboration can have in research.

Vehicle Behavior Discovery and Three-Dimensional Object Detection and Tracking Based on Spatio-Temporal Dependency Knowledge and Artificial Fish Swarm Algorithm.

In complex traffic environments, 3D target tracking and detection are often occluded by various stationary and moving objects. When the target is occluded, its apparent characteristics change, resulting in a decrease in the accuracy of tracking and detection. In order to solve this problem, we propose to learn the vehicle behavior from the driving data, predict and calibrate the vehicle trajectory, and finally use the artificial fish swarm algorithm to optimize the tracking results. The experiments show that compared with the CenterTrack method, the proposed method improves the key indicators of MOTA (Multi-Object Tracking Accuracy) in 3D object detection and tracking on the nuScenes dataset, and the frame rate is 26 fps.

Online 3D behavioral tracking of aquatic model organism with a dual-camera system

Behavioral tracking system of aquatic model organism is crucial for applications in aquaculture, environment and biomedicine, as it facilitates human to monitor subject states by automatically recognizing individual identities, and quantify their movement trajectories. Previous research has been devoted to this topic, but they are still not simple and effective enough. Therefore, this work introduces a novel online monitoring system implemented by dual-camera equipment and software modules consisting of an object detector and a multi-view multi-target tracker. The tracker provides the abilities of cross-view matching, underwater 3D reconstruction, and 3D target tracking. Specifically, our solution adopts a new paradigm, called tracking by early-reconstruction, which prioritizes the 3D reconstruction of targets’ coordinates on a frame-by-frame basis and then tracks them directly in 3D space rather than in a 2D image plane. This paradigm simplifies the complex multi-view tracking problem into a series of local association procedures, allowing us to achieve an online resolution through the iterative approach. To verify the effectiveness of the system, we employ zebrafish as the research subject, and evaluate the accuracy and robustness of the system on tracking benchmark, behavioral tasks and simulated data. Finally, we conducted extensive experiments and demonstrated the efficiency and effectiveness of the proposed system.

Optimal design and performance analysis of secure estimator for cyber‐physical systems against deception attacks

AbstractCyber‐physical systems (CPSs) are increasingly vulnerable to the threat from malicious adversaries owing to the deep integration of networks and facilities; therefore, CPSs' security against cyber attacks has attracted extensive attention. The deception attack is one of the typical cyber attacks targeting CPSs, and a deception attack mode is established to describe the simultaneous occurrence of actuator and sensor attacks in this paper. Compared with single‐target deception attacks with constant value, the designed attack mode that can describe deception attacks against multiple components and embedded random characteristics has higher applicability in practical engineering processes. To counter such attack behavior, a secure estimator is proposed based on the ideology of augmented state to achieve the optimal design of secure state estimation and deception attack identification. Taking the energy shortage problem of CPSs into account, a novel event‐triggered communication mechanism is introduced on the basis of measurement residuals to improve resource efficiency by reducing low‐value information transmission. The communication scheme and the augmented state estimator are co‐designed by adopting the implicit information of the event‐triggered mechanism. Then, the performance of the co‐designed estimator is analyzed by the upper bound of the estimation error covariance matrix. Besides, the reconciliation between estimation performance and communication energy consumption is achieved by the derivation of Pareto equilibrium. Finally, applications of our approaches to a 3D target tracking system are provided to illustrate the effectiveness of the proposed method.

On the Stability of Nonlinear Model Predictive Control for 3D Target Tracking

In this study, we present the stability analysis of a Nonlinear Model Predictive Control (NMPC) of a 3D ground target tracking system model. The system model is derived considering the 3D kinematic system model referenced to the ground target. The NMPC adeptly track errors across five states: range, bearing angle, altitude, pitch angle, and speed. This objective is achieved through the utilization of yaw angle, pitch angle, and acceleration as control inputs. The contribution of this paper is the establishment of stability conditions within the Lyapunov framework for the closed-loop system. To demonstrate the efficacy of our proposed model, we conducted simulations, which shows high accuracy in tracking all designated states.

Fusion Architectures for 3D Target Tracking Using IRST and Radar Measurements

Seven different architectures are presented to fuse IRST and radar data to track the target in 3D Cartesian coordinates, with the measurements available in polar coordinates. Performance of these architectures is checked with simulated data. Detailed mathematical expressions are provided which could be useful for algorithm implementation. From this study, it is concluded that CM (Common Measurements) architecture gives state estimates with relatively less uncertainty followed by SVF (State Vector Fusion). IRST gives target states with relatively high uncertainty followed by radar. This shows the necessity of the fusion in tracking system. In all, CM architecture is very simple, easy to implement and can be used in real time.

3D target tracking and shape reconstruction in clutter using Gaussian process and point completion network

AbstractExisting problems for 3D extended target tracking and contour reconstruction under clutter environment are thoroughly investigated in this article. Due to the sparsity of available point cloud data and interference from clutter, the shape completion method, such as Point Completion Network (PCN), cannot reconstruct 3D contour directly. Moreover, the traditional Gaussian process (GP) model suffered from computing overhead and cannot handle irregular non‐convex shapes. Here, a two‐stage tracking algorithm is proposed, which firstly combines the 3D GP measurement model with a probability data association filter to jointly estimate the kinematics and 3D Gaussian basis points, which contain partial shape information. Afterwards, this method input produced Gaussian basis points to a deep learning‐based PCN to reconstruct complete geometry of the contour. The effectiveness of the proposed algorithm is verified by extended target tracking with both simple and complex geometric shapes. The simulation results show that the proposed algorithm obtains accurate kinematic state estimation and produces complete point cloud estimation for 3D contour.

Passive Radar Tracking in Clutter Using Range and Range-Rate Measurements

Passive bistatic radar research is essential for accurate 3D target tracking, especially in the presence of missing or low-quality bearing information. Traditional extended Kalman filter (EKF) methods often introduce bias in such scenarios. To overcome this limitation, we propose employing the unscented Kalman filter (UKF) for handling the nonlinearities in 3D tracking, utilizing range and range-rate measurements. Additionally, we incorporate the probabilistic data association (PDA) algorithm with the UKF to handle cluttered environments. Through extensive simulations, we demonstrate a successful implementation of the UKF-PDA framework, showing that the proposed method effectively reduces bias and significantly advances tracking capabilities in passive bistatic radars.

Sensor Fusion with Asynchronous Decentralized Processing for 3D Target Tracking with a Wireless Camera Network.

We present a method to acquire 3D position measurements for decentralized target tracking with an asynchronous camera network. Cameras with known poses have fields of view with overlapping projections on the ground and 3D volumes above a reference ground plane. The purpose is to track targets in 3D space without constraining motion to a reference ground plane. Cameras exchange line-of-sight vectors and respective time tags asynchronously. From stereoscopy, we obtain the fused 3D measurement at the local frame capture instant. We use local decentralized Kalman information filtering and particle filtering for target state estimation to test our approach with only local estimation. Monte Carlo simulation includes communication losses due to frame processing delays. We measure performance with the average root mean square error of 3D position estimates projected on the image planes of the cameras. We then compare only local estimation to exchanging additional asynchronous communications using the Batch Asynchronous Filter and the Sequential Asynchronous Particle Filter for further fusion of information pairs' estimates and fused 3D position measurements, respectively. Similar performance occurs in spite of the additional communication load relative to our local estimation approach, which exchanges just line-of-sight vectors.

Autonomous 3D Target Tracking via UAV

In recent years, unmanned aerial vehicles (UAVs) have been integrated into many applications. These vehicles are used by the governments and the private sector for many different purposes. Some of the popular applications are field scanning, target tracking, defense markets, search and rescue, map creation, underground resource search, photography or entertainment. Recently many academic institutions have shown significant amount of interest in this field and many of them have established aviation departments and societies. In addition, many thesis studies and academic publications related to UAVs have been presented. The aim of this particular study is to detect a small object identified as a target by UAV and to provide autonomous tracking of the detected target in three axes. As the outcome of this work, the UAV is coded to lock on the target and maintain a fixed distance on the x, y and z axes after the initial lock. Within the scope of the study, a functional system was designed by evaluating the most important parameters for UAVs such as weight, cost and flight time. An algorithm was designed that uses the image processing methods to detect the target and movement direction of the target, and utilizes MAVLink communication protocol to control the UAV. In addition, two different tests and test results corresponding to two distinct methods were reported which were done to keep the distance constant on the z-axis by using an external distance sensor or comparison algorithm. The project was successfully implemented using single camera only.

FDA-SSD: Fast Depth-Assisted Single-Shot MultiBox Detector for 3D Tracking Based on Monocular Vision

In this study, a set of benchmarks for object tracking with motion parameters (OTMP) was first designed. The sample images were matched with the spatial depth of the camera, the pose of the camera, and other spatial parameters for the training of the detection model. Then, a Fast Depth-Assisted Single-Shot MultiBox Detector (FDA-SSD) algorithm suitable for 3D target tracking was proposed by combining the depth information of the sample into the original Single-Shot MultiBox Detector (SSD). Finally, an FDA-SSD-based monocular motion platform target detection and tracking algorithm framework were established. Specifically, the spatial geometric constraints of the target were adapted to solve the target depth information, which was fed back to the detection model. Then, the normalized depth information of the target was employed to select the feature window of the convolutional layer for the detector at a specific scale. This significantly reduces the computational power for simultaneously calculating detectors of all scales. This framework effectively combines the two-dimensional detection model and the three-dimensional positioning algorithm. Compared with the original SSD method, the network model designed in this study has fewer actual operating parameters; the measured detection operation speed was increased by about 18.1% on average; the recognition rate was maintained at a high level consistent with that of the original SSD. Furthermore, several groups of experiments were conducted on target detection and target space tracking based on monocular motion platforms indoors. The root mean square error (RMSE) of the spatial tracking trajectory was less than 4.72 cm. The experimental results verified that the algorithm framework in this study can effectively realize tasks such as visual detection, classification, and spatial tracking based on a monocular motion platform.

Vision-Based 3D Aerial Target Detection and Tracking for Maneuver Decision in Close-Range Air Combat

Automatic maneuver decision in close-range air combat depends on the situation awareness of the 3D aerial space. Optimal decision could only be made when the 3D state (e.g. 3D position, orientation and velocity) of the target aircraft is accurately provided. Together with the state of the aircraft in our side, optimal maneuver decision could be made by maximizing the situation advantage or utilizing deep reinforcement learning. On the other hand, vision-based 3D sensing methods are ideal for acquiring the 3D state of the target aircraft in close-range air combat, since radar and other sensors work badly in such short range. In this paper, we propose a novel pipeline for vision-based maneuver decision in close-range air combat. The proposed pipeline contains three main modules: 3D target detection based on Augmented Autoencoder, 3D target tracking based on segmentation and optimization, and maneuver decision based on advantage maximization and Deep Q Networks (DQN). The proposed method effectively handles the difficulties in air combat environment, such as fast movement, occlusion from cloud, etc. Experiments demonstrate that our method could robustly detect and track the target aircraft in complex environment, which provides strong priors for maneuver decision and helps to significantly improve the winning rate of short-range air combat.

Pervasive underwater passive target tracking for the computation of standard deviation solution in a 3D environment

PurposeNowadays advancement in acoustic technology can be explored with marine assets. The purpose of the paper is pervasive computing underwater target tracking has aroused military and civilian interest as a key component of ocean exploration. While many pervasive techniques are currently found in the literature, there is little published research on the effectiveness of these paradigms in the target tracking context.Design/methodology/approachThe unscented Kalman filter (UKF) provides good results for bearing and elevation angles only tracking. Detailed methodology and mathematical modeling are carried out and used to analyze the performance of the filter based on the Monte Carlo simulation.FindingsDue to the intricacy of maritime surroundings, tracking underwater targets using acoustic signals, without knowing the range parameter is difficult. The intention is to find out the solution in terms of standard deviation in a three-dimensional (3D) space.Originality/valueA new method is found for the acceptance criteria for range, course, speed and pitch based on the standard deviation for bearing and elevation 3D target tracking using the unscented Kalman filter covariance matrix. In the Monte Carlo simulation, several scenarios are used and the results are shown.

Event-based distributed bias compensation pseudomeasurement information filter for 3D bearing-only target tracking

This paper investigates the distributed 3D bearing-only target tracking problem in UAV network. Because of the limited computation capability and constrained power supply of UAV, traditional time-triggered nonlinear filters would inevitably cause serious network congestions. Thus, we proposed an efficient yet effective algorithm named event-based distributed pseudomeasurement information filter (EB-PMIF) by combining the deterministic Send-on-Delta triggering event-based mechanism and pseudomeasurement information filter in order to reduce the computation load and communication requirements. Since the correlated pseudomeasurement matrix and pseudomeasurement noise in EB-PMIF cause bias estimation, we analyzed the root cause of the bias and proposed a bias compensation filter named event-based distributed bias compensation pseudomeasurement information filter (EB-PMIF-BC) to eliminate the bias and improve the estimation accuracy. The proposed EB-PMIF and EB-PMIF-BC are demonstrated with an illustrative 3D bearing-only target tracking example. Simulation results verify the efficiency and effectiveness of the proposed methods.

Predictive online 3D target tracking with population-based generative networks for image-guided radiotherapy.

Respiratory motion of thoracic organs poses a severe challenge for the administration of image-guided radiotherapy treatments. Providing online and up-to-date volumetric information during free breathing can improve target tracking, ultimately increasing treatment efficiency and reducing toxicity to surrounding healthy tissue. In this work, a novel population-based generative network is proposed to address the problem of 3D target location prediction from 2D image-based surrogates during radiotherapy, thus enabling out-of-plane tracking of treatment targets using images acquired in real time. The proposed model is trained to simultaneously create a low-dimensional manifold representation of 3D non-rigid deformations and to predict, ahead of time, the motion of the treatment target. The predictive capabilities of the model allow correcting target location errors that can arise due to system latency, using only a baseline volume of the patient anatomy. Importantly, the method does not require supervised information such as ground-truth registration fields, organ segmentation, or anatomical landmarks. The proposed architecture was evaluated on both free-breathing 4D MRI and ultrasound datasets. Potential challenges present in a realistic therapy, like different acquisition protocols, were taken into account by using an independent hold-out test set. Our approach enables 3D target tracking from single-view slices with a mean landmark error of 1.8mm, 2.4mm and 5.2mm in volunteer MRI, patient MRI and US datasets, respectively, without requiring any prior subject-specific 4D acquisition. This model presents several advantages over state-of-the-art approaches. Namely, it benefits from an explainable latent space with explicit respiratory phase discrimination. Thanks to the strong generalization capabilities of neural networks, it does not require establishing inter-subject correspondences. Once trained, it can be quickly deployed with an inference time of only 8ms. The results show the capability of the network to predict future anatomical changes and track tumors in real time, yielding statistically significant improvements over related methods.

Experiment of Stereo Matching Algorithm Based on Binocular Vision

Binocular stereo matching is a hot topic in 3D reconstruction, target detection and target tracking. Through experiments, this paper analyzes and compares several classic binocular stereo matching algorithms, and finally selects SGBM with good comprehensive performance. Moreover, based on this, it gets accurate window size parameters through experiments, which provides a basis for the following 3D modeling.

Indoor Multi-Floor 3D Target Tracking Based on the Multi-Sensor Fusion

In recent years, indoor target tracking based on pedestrian dead reckoning (PDR) and the built-in inertial sensors of smartphone has become a research hotspot in location-based services (LBS). However, indoor 3D target tracking using smartphone inertial measurement unit (IMU) mainly face challenges of error accumulation caused by the heading drift of sensors and height fluctuation caused by the instability of barometer. This paper proposes a multi-floor 3D target tracking system based on the built-in inertial sensors of smartphone. We first establish a 2D PDR movement model by raw sensor data and then 2D PDR position is corrected in real time by rebuilding the particle filter model and refining the calculation method of particle weight to reduce the accumulated errors. A height displacement measurement method based on Kalman filter and floor change detection (FCD) algorithm is proposed to extend the 2D PDR tracking to 3D space. We first skillfully use Kalman filter to fuse the data of barometer and accelerometer, after that, the FCD algorithm is proposed to evaluate and modify the output height of the Kalman filter, evaluating the floor change state and maintaining steady height. The 3D target tracking, which consists of real-time 2D trajectory information and height information, is provided to the pedestrian. Experimental results demonstrate that the proposed method based on the fusion of various technologies can effectively maintain track stability and smoothness with low cost and high positioning accuracy. Moreover, additional peripheral devices need not be arranged in advance.

A three dimensional tracking scheme for underwater non-cooperative objects in mixed LOS and NLOS environment

Underwater positioning and tracking scheme for non-cooperative objects is of great essence to explore unknown fields. Due to the high response time and non-line-of-sight(NLOS) propagation in the underwater acoustic sensor networks (UASNs), the existed range-based 3D target tracking algorithms are generally inaccurate on detecting underwater non-cooperative objects. In order to solve the problems above, the corresponding solutions are presented respectively in this paper. Although it is hard to change the inherent property of the underwater acoustic propagation, reducing the communication time is another way to solve the problem indirectly. Since the ranging phase and synchronize phase occupy most of the communication time, the presented novel ranging scheme for non-cooperative objects reduces the redundant time consumption, and further eliminates the necessity of synchronization process in advanced. For NLOS propagation, a distributed residual weighting discrimination (DRWD) algorithm based on grouping strategy is proposed for non-cooperative objects. The position estimations of the groups containing the NLOS link error are always distributed in isolation, and the estimations without the NLOS link errors are always concentrated in a small range. According to this feature, a low computational complexity approach namely two-step least square (LS) is proposed to determine the best location by analyzing the distribution of estimated coordinates. Meanwhile, a parameterized selection strategy is proposed first time to evaluate the construction of reference nodes in 3D target tracking. We provide a mathematical proof for our strategy, which avoids the ambiguity occurrence caused by the distribution of reference nodes. The new scheme provided for underwater acoustic tracking (UWAT) greatly improves the positioning accuracy in mixed LOS/NLOS environment. At the end of the paper, simulations are illustrated to evaluate and validate the algorithmic superiority and effectiveness.

Use of continuous 3D target-tracking in VR to measure response latency to changes in depth

Use of continuous 3D target-tracking in VR to measure response latency to changes in depth

A Method to Track Targets in Three-Dimensional Space Using an Imaging Sonar.

This paper introduces a methodology applying an imaging sonar for three-dimensional (3D) target tracking underwater. The key process in this work involves obtaining the target’s position in space using two images of the same scene, acquired by an adaptive resolution imaging sonar (ARIS) at different positions. A data association algorithm was designed to connect the same target in image sequences. The goal of this work was to track multiple targets in 3D space. The ARIS provides sequences of bi-dimensional images from the backscattered energy according to the range and azimuth. The challenge involved determining the missing elevation information for the observed object within the sonar detection range. By computing the geometrical transformation between the acquisition planar images and the cubical space, using only the sonar information that included the posture and moving speed of the ARIS, the target’s elevation information was obtained. To evaluate the performance of the proposed method, an indoor experiment was conducted using the ARIS. On the basis of the experimental results, we confirmed that the proposed method effectively obtained the target’s position in 3D space. A moving target simulation was also conducted, and the results showed that this method was effective for moving targets. Finally, a field experiment was performed to obtain the vertical distribution and track the 3D trajectories of fish.