Dynamic Object Tracking Research Articles

Seal Pipeline: Enhancing Dynamic Object Detection and Tracking for Autonomous Unmanned Surface Vehicles in Maritime Environments

This study addresses the dynamic object detection problem for Unmanned Surface Vehicles (USVs) in marine environments, which is complicated by boat tilting and camera illumination sensitivity. A novel pipeline named “Seal” is proposed to enhance detection accuracy and reliability. The approach begins with an innovative preprocessing stage that integrates data from the Inertial Measurement Unit (IMU) with LiDAR sensors to correct tilt-induced distortions in LiDAR point cloud data and reduce ripple effects around objects. The adjusted data are grouped using clustering algorithms and bounding boxes for precise object localization. Additionally, a specialized Kalman filter tailored for maritime environments mitigates object discontinuities between successive frames and addresses data sparsity caused by boat tilting. The methodology was evaluated using the VRX simulator, with experiments conducted on the Volga River using real USVs. The preprocessing effectiveness was assessed using the Root Mean Square Error (RMSE) and tracking accuracy was evaluated through detection rate metrics. The results demonstrate a 25% to 30% improvement in detection accuracy and show that the pipeline can aid industry even with sparse object representation across different frames. This study highlights the potential of integrating sensor fusion with specialized tracking for accurate dynamic object detection in maritime settings, establishing a new benchmark for USV navigation systems’ accuracy and reliability.

DOT-SLAM: A Stereo Visual Simultaneous Localization and Mapping (SLAM) System with Dynamic Object Tracking Based on Graph Optimization.

Most visual simultaneous localization and mapping (SLAM) systems are based on the assumption of a static environment in autonomous vehicles. However, when dynamic objects, particularly vehicles, occupy a large portion of the image, the localization accuracy of the system decreases significantly. To mitigate this challenge, this paper unveils DOT-SLAM, a novel stereo visual SLAM system that integrates dynamic object tracking through graph optimization. By integrating dynamic object pose estimation into the SLAM system, the system can effectively utilize both foreground and background points for ego vehicle localization and obtain a static feature points map. To rectify the inaccuracies in depth estimation from stereo disparity directly on the foreground points of dynamic objects due to their self-similarity characteristics, a coarse-to-fine depth estimation method based on camera-road plane geometry is presented. This method uses rough depth to guide fine stereo matching, thereby obtaining the 3 dimensions (3D)spatial positions of feature points on dynamic objects. Subsequently, by establishing constraints on the dynamic object's pose using the road plane and non-holonomic constraints (NHCs) of the vehicle, reducing the initial pose uncertainty of dynamic objects leads to more accurate dynamic object initialization. Finally, by considering foreground points, background points, the local road plane, the ego vehicle pose, and dynamic object poses as optimization nodes, through the establishment and joint optimization of a nonlinear model based on graph optimization, accurate six degrees of freedom (DoFs) pose estimations are obtained for both the ego vehicle and dynamic objects. Experimental validation on the KITTI-360 dataset demonstrates that DOT-SLAM effectively utilizes features from the background and dynamic objects in the environment, resulting in more accurate vehicle trajectory estimation and a static environment map. Results obtained from a real-world dataset test reinforce the effectiveness.

Kalman Filter for a Particular Class of Dynamic Object Images

We discuss the problem of estimating the state of a dynamic object by using observed images generated by an optical system. The work aims to implement a novel approach that would ensure improved accuracy of dynamic object tracking using a sequence of images. We utilize a vector model that describes the object image as a limited number of vertexes (reference points). Upon imaging, the object of interest is assumed to be retained at the center of each frame, so that the motion parameters can be considered as projections onto the axes of a coordinate system matched with the camera's optical axis. The novelty of the approach is that the observed parameters (the distance along the optical axis and angular attitude) of the object are calculated using the coordinates of specified points in the object images. For estimating the object condition, a Kalman-Bucy filter is constructed on the assumption that the dynamic object motion is described by a set of equations for the translational motion of the center of mass along the optical axis and variations in the angular attitude relative to the image plane. The efficiency of the proposed method is illustrated by an example of estimating the object's angular attitude.

DMOT-SLAM: visual SLAM in dynamic environments with moving object tracking

Visual simultaneous localization and mapping (SLAM) in dynamic environments has received significant attention in recent years, and accurate segmentation of real dynamic objects is the key to enhancing the accuracy of pose estimation in such environments. In this study, we propose a visual SLAM approach based on ORB-SLAM3, namely dynamic multiple object tracking SLAM (DMOT-SLAM), which can accurately estimate the camera’s pose in dynamic environments while tracking the trajectories of moving objects. We introduce a spatial point correlation constraint and combine it with instance segmentation and epipolar constraint to identify dynamic objects. We integrate the proposed motion check method into DeepSort, an object tracking algorithm, to facilitate inter-frame tracking of dynamic objects. This integration not only enhances the stability of dynamic features detection but also enables the estimation of global motion trajectories for dynamic objects and the construction of object-level semi-dense semantic maps. We evaluate our approach on the public TUM, Bonn, and KITTI dataset, and the results show that our approach has a significant improvement over ORB-SLAM3 in dynamic scenes and performs better compared to other state-of-the-art SLAM approaches. Moreover, experiments in real-world scenarios further substantiate the effectiveness of our approach.

Adaptive iterative learning control of soft robot for beating heart tracking

PurposeSoft robots are known for their excellent safe interaction ability and promising in surgical applications for their lower risks of damaging the surrounding organs when operating than their rigid counterparts. To explore the potential of soft robots in cardiac surgery, this paper aims to propose an adaptive iterative learning controller for tracking the irregular motion of the beating heart.Design/methodology/approachIn continuous beating heart surgery, providing a relatively stable operating environment for the operator is crucial. It is highly necessary to use position-tracking technology to keep the target and the surgical manipulator as static as possible. To address the position tracking and control challenges associated with dynamic targets, with a focus on tracking the motion of the heart, control design work has been carried out. Considering the lag error introduced by the material properties of the soft surgical robotic arm and system delays, a controller design incorporating iterative learning control with parameter estimation was used for position control. The stability of the controller was analyzed and proven through the construction of a Lyapunov function, taking into account the unique characteristics of the soft robotic system.FindingsThe tracking performance of both the proportional-derivative (PD) position controller and the adaptive iterative learning controller are conducted on the simulated heart platform. The results of these two methods are compared and analyzed. The designed adaptive iterative learning control algorithm for position control at the end effector of the soft robotic system has demonstrated improved control precision and stability compared with traditional PD controllers. It exhibits effective compensation for periodic lag caused by system delays and material characteristics.Originality/valueTracking the beating heart, which undergoes quasi-periodic and complex motion with varying accelerations, poses a significant challenge even for rigid mechanical arms that can be precisely controlled and makes tracking targets located at the surface of the heart with the soft robot fraught with considerable difficulties. This paper originally proposes an adaptive interactive learning control algorithm to cope with the dynamic object tracking problem. The algorithm has theoretically proved its convergence and experimentally validated its performance at the cable-driven soft robot test bed.

CTO-SLAM: Contour Tracking for Object-Level Robust 4D SLAM

The demand for 4D ( 3D+time ) SLAM system is increasingly urgent, especially for decision-making and scene understanding. However, most of the existing simultaneous localization and mapping ( SLAM ) systems primarily assume static environments. They fail to represent dynamic scenarios due to the challenge of establishing robust long-term spatiotemporal associations in dynamic object tracking. We address this limitation and propose CTO-SLAM, a monocular and RGB-D object-level 4D SLAM system to track moving objects and estimate their motion simultaneously. In this paper, we propose contour tracking, which introduces contour features to enhance the keypoint representation of dynamic objects and coupled with pixel tracking to achieve long-term robust object tracking. Based on contour tracking, we propose a novel sampling-based object pose initialization algorithm and the following adapted bundle adjustment ( BA ) optimization algorithm to estimate dynamic object poses with high accuracy. The CTO-SLAM system is verified on both KITTI and VKITTI datasets. The experimental results demonstrate that our system effectively addresses cumulative errors in long-term spatiotemporal association and hence obtains substantial improvements over the state-of-the-art systems. The source code is available at https://github.com/realXiaohan/CTO-SLAM.

DL-SLOT: Tightly-Coupled Dynamic LiDAR SLAM and 3D Object Tracking Based on Collaborative Graph Optimization

DL-SLOT: Tightly-Coupled Dynamic LiDAR SLAM and 3D Object Tracking Based on Collaborative Graph Optimization

Dynamic Object Detection and Tracking in Vision SLAM

Abstract Addressing the trade-off between operational efficiency and localization accuracy in visual SLAM, this paper introduces a monocular visual-inertial SLAM algorithm that integrates point and line features. To construct the point-line reprojection error and optimize the observation volume in front-end vision initialization, the motion recovery structure method (SFM) is employed through 3D reconstruction with a sliding window. The marginalization method uses the removed keyframe information as a priori constraint for nonlinear optimization in the back-end. In addition, the loopback detection algorithm is optimized in combination with the bag-of-words model and four-degree-of-freedom global bitmap to improve the accuracy of dynamic object detection, and the performance of the algorithm is tested. The results show that in the case of no closed loop, the absolute root mean square error of the algorithm proposed in this paper is lower than that of VINS-Mono (0.0625), PL-VIO (0.0401), and PL-VINS (0.0554) for the majority of sequences. In the case of closed loops, the absolute root mean square error of the proposed algorithm in this paper is reduced by 0.0395 and 0.0139 on average over most sequences compared to VINS-Mono and PL-VINS. The proposed algorithm in this paper demonstrates higher accuracy and robustness for improved detection and tracking of dynamic objects.

Dynamic Object Tracking Based on Triplet Relationship Guided Sampling Consensus Algorithm

Compared with the traditional methods based on the prediction of moving object trajectory, the method based on image registration has the advantage of not relying on the object motion equation to track the object. However, in the case of a large number of outliers, the image matching method has the defects of inadequate filtering of outliers or erroneous filtering of incomers due to the limitations of spatial constraints of feature matching pairs, which will make the tracking of flying objects inaccurate or impossible. During the flight, the object may be in a similar background and the image shooting angle is different, which makes the object imaging angle change, resulting in a large number of outlier interference. Because of the above situation, this paper adopted an improved TRESAC algorithm based on RANSAC, which used a feature matching method based on a triplet relationship to effectively filter outliers and then adopted an initial data subset selection strategy to increase its robustness. Experimental results show that the TRESAC algorithm can filter outliers quickly and accurately, and the object is still tracked effectively under the condition of similar background and pose change.

A Visual-Inertial Dynamic Object Tracking SLAM Tightly Coupled System

A Visual-Inertial Dynamic Object Tracking SLAM Tightly Coupled System

Perception-Aware Image-Based Visual Servoing of Aggressive Quadrotor UAVs

The maintenance of visual features within the sensor field of view poses a significant challenge for underactuated aerial vehicles such as quadrotors, especially during aggressive maneuvers. However, the existing image-based visual servo (IBVS) control methods rely on strict target visibility assumptions or impose excessive constraints on the quadrotor's agility to meet this requirement. Furthermore, the effectiveness of the visibility constraint defined in prior works remains unverified in aggressive flight tests. To address these issues, we present a robust IBVS scheme for quadrotors to perform aggressive maneuvers while ensuring target visibility. Based on the nonlinear model-predictive control framework, we propose a novel underactuation compensation scheme to eliminate the need for a virtual camera frame, which enables us to formulate the true target visibility constraint. We then introduce a quaternion-based representation of spherical visual features to handle the nonsmooth vector field problem on the 2-sphere and derive its corresponding image kinematics. We validate our method through three challenging visual servo tasks where agile maneuvers are desired: fast landing, aggressive long-distance flight, and dynamic object tracking. Extensive simulation and experiment show that our method consistently achieves a target-visible rate of 100% in all the image frames, even under a maximum pitch of 21.04 <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$^{\circ }$</tex-math></inline-formula> . The results validate the effectiveness of our visibility constraint under large robot rotations and underscore its importance in enabling robust and aggressive flights.

A novel multidimensional uncalibration method applied to six-axis manipulators.

This study proposes a multidimensional uncalibrated technique for tracking and grasping dynamic targets by a robotic arm in the eye-in-hand mode. This method avoids complex and cumbersome calibration processes, enabling machine vision tasks to be adaptively applied in a variety of complex environments, which solved the problem of traditional calibration methods being unstable in complex environments. The specific method used in this study is first, in the eye-in-hand mode, the robotic arm moves along the x, y, and z axes in sequence, and images are taken before and after each movement. Thereafter, the image Jacobian matrix is calculated from the three (or more) sets of images collected. Finally, the robotic arm converts the target coordinates in the real-time captured images by the camera into coordinates in the robotic arm coordinate system through the image Jacobian matrix and performs real-time tracking. This study tests the dynamic quasi-Newton method for estimating the Jacobian matrix and optimizes the initialization coupling problem using the orthogonal moving method. This optimization scheme significantly shortens the iteration process, making the uncalibrated technology more fully applied in the field of dynamic object tracking. In addition, this study proposes a servo control algorithm with predictive compensation to mitigate or even eliminate the systematic error caused by time delay in dynamic target tracking in robot visual servo systems.

PointSLOT: Real-Time Simultaneous Localization and Object Tracking for Dynamic Environment

The applicability of SLAM algorithms is largely limited to the assumption of scene rigidity. Moreover, dynamic object perception is an essential technique for many robotic applications such as autonomous driving and multi-robot collaboration. In this letter, we present PointSLOT, an online, real-time and general stereo system that simultaneously performs SLAM and dynamic object tracking without any artificial priors. Unlike previous SLOT systems, we explicitly identify moving and stationary objects by computing the object motion probabilities, such that features from static objects are utilized to improve the camera pose estimation instead of treating all object regions as outliers. Based on the camera-centric parameterization for the object pose, the trajectories and dynamic landmarks of multi-keyframe objects are efficiently computed through an object-based bundle adjustment approach. The evaluations on KITTI datasets and real-world experiments show that the proposed system achieves comparable results to state-of-the-art solutions on both camera motion estimation and dynamic object tracking. The open-source code is available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/pkzhou/PointSLOT</uri> .

UAV Dynamic Object Tracking with Lightweight Deep Vision Reinforcement Learning

Several approaches have applied Deep Reinforcement Learning (DRL) to Unmanned Aerial Vehicles (UAVs) to do autonomous object tracking. These methods, however, are resource intensive and require prior knowledge of the environment, making them difficult to use in real-world applications. In this paper, we propose a Lightweight Deep Vision Reinforcement Learning (LDVRL) framework for dynamic object tracking that uses the camera as the only input source. Our framework employs several techniques such as stacks of frames, segmentation maps from the simulation, and depth images to reduce the overall computational cost. We conducted the experiment with a non-sparse Deep Q-Network (DQN) (value-based) and a Deep Deterministic Policy Gradient (DDPG) (actor-critic) to test the adaptability of our framework with different methods and identify which DRL method is the most suitable for this task. In the end, a DQN is chosen for several reasons. Firstly, a DQN has fewer networks than a DDPG, hence reducing the computational resources on physical UAVs. Secondly, it is surprising that although a DQN is smaller in model size than a DDPG, it still performs better in this specific task. Finally, a DQN is very practical for this task due to the ability to operate in continuous state space. Using a high-fidelity simulation environment, our proposed approach is verified to be effective.

Dynamic Object Tracking by Multi-UAV with Time-variant Radio Maps

Dynamic Object Tracking by Multi-UAV with Time-variant Radio Maps

3D Multi-Object Tracking With Adaptive Cubature Kalman Filter for Autonomous Driving

A crucial and challenging issue in autonomous driving is dynamic road environment detection and 3D multi object tracking. In this article, we propose a novel framework for online 3D multi-object tracking to eliminate the influence of inherent uncertainty and unknown biases in point cloud. A constant turn rate and velocity (CTRV) motion model is employed to estimate the future motion state, which are smoothed by a cubature Kalman filter (CKF) algorithm. A new affinity model is introduced to evaluate the similarity between trajectories and candidate detections for accurate and reliable data association which can be formulated as a bipartite matching problem. An adaptive cubature Kalman filter (ACKF) is given to remove the influence of unknown bias and to robustly update the tracked state. Accuracy and speed of the proposed tracking method are evaluated on the KITTI 3D multi-object tracking dataset, showing superior performance than the baselines.

Doppler Velocity-based Dynamic Object Tracking and Rejection for Increasing Reliability of Radar Ego-Motion Estimation

Doppler Velocity-based Dynamic Object Tracking and Rejection for Increasing Reliability of Radar Ego-Motion Estimation

EVtracker: An Event-Driven Spatiotemporal Method for Dynamic Object Tracking.

An event camera is a novel bio-inspired sensor that effectively compensates for the shortcomings of current frame cameras, which include high latency, low dynamic range, motion blur, etc. Rather than capturing images at a fixed frame rate, an event camera produces an asynchronous signal by measuring the brightness change of each pixel. Consequently, an appropriate algorithm framework that can handle the unique data types of event-based vision is required. In this paper, we propose a dynamic object tracking framework using an event camera to achieve long-term stable tracking of event objects. One of the key novel features of our approach is to adopt an adaptive strategy that adjusts the spatiotemporal domain of event data. To achieve this, we reconstruct event images from high-speed asynchronous streaming data via online learning. Additionally, we apply the Siamese network to extract features from event data. In contrast to earlier models that only extract hand-crafted features, our method provides powerful feature description and a more flexible reconstruction strategy for event data. We assess our algorithm in three challenging scenarios: 6-DoF (six degrees of freedom), translation, and rotation. Unlike fixed cameras in traditional object tracking tasks, all three tracking scenarios involve the simultaneous violent rotation and shaking of both the camera and objects. Results from extensive experiments suggest that our proposed approach achieves superior accuracy and robustness compared to other state-of-the-art methods. Without reducing time efficiency, our novel method exhibits a increase in accuracy over other recent models. Furthermore, results indicate that event cameras are capable of robust object tracking, which is a task that conventional cameras cannot adequately perform, especially for super-fast motion tracking and challenging lighting situations.

A Quantitative Validation of Multi-Modal Image Fusion and Segmentation for Object Detection and Tracking

In previous works, we have shown the efficacy of using Deep Belief Networks, paired with clustering, to identify distinct classes of objects within remotely sensed data via cluster analysis and qualitative analysis of the output data in comparison with reference data. In this paper, we quantitatively validate the methodology against datasets currently being generated and used within the remote sensing community, as well as show the capabilities and benefits of the data fusion methodologies used. The experiments run take the output of our unsupervised fusion and segmentation methodology and map them to various labeled datasets at different levels of global coverage and granularity in order to test our models’ capabilities to represent structure at finer and broader scales, using many different kinds of instrumentation, that can be fused when applicable. In all cases tested, our models show a strong ability to segment the objects within input scenes, use multiple datasets fused together where appropriate to improve results, and, at times, outperform the pre-existing datasets. The success here will allow this methodology to be used within use concrete cases and become the basis for future dynamic object tracking across datasets from various remote sensing instruments.

Detecting Anonymous Target and Predicting Target Trajectories in Wireless Sensor Networks

Target Tracking (TT) is an application of Wireless Sensor Networks (WSNs) which necessitates constant assessment of the location of a target. Any change in position of a target and the distance from each intermediate sensor node to the target is passed on to base station and these factors play a crucial role in further processing. The drawback of WSN is that it is prone to numerous constraints like low power, faulty sensors, environmental noises, etc. The target should be detected first and its path should be tracked continuously as it moves around the sensing region. This problem of detecting and tracking a target should be conducted with maximum accuracy and minimum energy consumption in each sensor node. In this paper, we propose a Target Detection and Target Tracking (TDTT) model for continuously tracking the target. This model uses prelocalization-based Kalman Filter (KF) for target detection and clique-based estimation for tracking the target trajectories. We evaluated our model by calculating the probability of detecting a target based on distance, then estimating the trajectory. We analyzed the maximum error in position estimation based on density and sensing radius of the sensors. The results were found to be encouraging. The proposed KF-based target detection and clique-based target tracking reduce overall expenditure of energy, thereby increasing network lifetime. This approach is also compared with Dynamic Object Tracking (DOT) and face-based tracking approach. The experimental results prove that employing TDTT improves energy efficiency and extends the lifetime of the network, without compromising the accuracy of tracking.