Dynamic Object Detection Research Articles

Seal Pipeline: Enhancing Dynamic Object Detection and Tracking for Autonomous Unmanned Surface Vehicles in Maritime Environments

This study addresses the dynamic object detection problem for Unmanned Surface Vehicles (USVs) in marine environments, which is complicated by boat tilting and camera illumination sensitivity. A novel pipeline named “Seal” is proposed to enhance detection accuracy and reliability. The approach begins with an innovative preprocessing stage that integrates data from the Inertial Measurement Unit (IMU) with LiDAR sensors to correct tilt-induced distortions in LiDAR point cloud data and reduce ripple effects around objects. The adjusted data are grouped using clustering algorithms and bounding boxes for precise object localization. Additionally, a specialized Kalman filter tailored for maritime environments mitigates object discontinuities between successive frames and addresses data sparsity caused by boat tilting. The methodology was evaluated using the VRX simulator, with experiments conducted on the Volga River using real USVs. The preprocessing effectiveness was assessed using the Root Mean Square Error (RMSE) and tracking accuracy was evaluated through detection rate metrics. The results demonstrate a 25% to 30% improvement in detection accuracy and show that the pipeline can aid industry even with sparse object representation across different frames. This study highlights the potential of integrating sensor fusion with specialized tracking for accurate dynamic object detection in maritime settings, establishing a new benchmark for USV navigation systems’ accuracy and reliability.

Infield disease detection in citrus plants: integrating semantic segmentation and dynamic deep learning object detection model for enhanced agricultural yield

Infield disease detection in citrus plants: integrating semantic segmentation and dynamic deep learning object detection model for enhanced agricultural yield

Sequential Point Clouds: A Survey.

Point clouds have garnered increasing research attention and found numerous practical applications. However, many of these applications, such as autonomous driving and robotic manipulation, rely on sequential point clouds, essentially adding a temporal dimension to the data (i.e., four dimensions) because the information of the static point cloud data could provide is still limited. Recent research efforts have been directed towards enhancing the understanding and utilization of sequential point clouds. This paper offers a comprehensive review of deep learning methods applied to sequential point cloud research, encompassing dynamic flow estimation, object detection & tracking, point cloud segmentation, and point cloud forecasting. This paper further summarizes and compares the quantitative results of the reviewed methods over the public benchmark datasets. Ultimately, the paper concludes by addressing the challenges in current sequential point cloud research and pointing towards promising avenues for future research.

A dynamic object removing 3D reconstruction system based on multi-sensor fusion

Abstract Currently, one of the key technologies for autonomous navigation of unmanned mobile robots is SLAM, which faces many challenges in practical applications. These challenges include a lack of texture, deterioration in sensor performance, and interference from moving objects in dynamic outdoor environments, all of which have an impact on the mapping system. To address these issues, this paper proposes a framework for lidar, vision camera, and inertial navigation data, resulting in fusion and dynamic object removing. The system consists of three sub-modules: the Lidar-Inertial Module (LIM), the Visual-Inertial Module (VIM), and the Dynamic-Object-Removing Module (DORM). LIM and VIM assist each other, with lidar point clouds providing three-dimensional information for the global voxel map and the camera providing pixel-level color information. At the same time, the DORM performs synchronous dynamic object detection to remove dynamic objects from the global map. The system constructs a multi-sensor factor graph using the state and observation models, and the optimal solution is obtained using least squares. Furthermore, this paper employs triangle descriptors and bundle adjustment methods for loop closure detection in order to reduce accumulated errors and maintain consistency. Experimental results demonstrate that the system can perform clean state estimation, dynamic removing and scene reconstruction in a variety of complex scenarios.

Dynamic object detection using sparse LiDAR data for autonomous machine driving and road safety applications

Light Detection And Ranging (LiDAR) sensor offers solutions to extract real-time information of vehicle’s surroundings and can provide a new opportunity to improve the road safety related issues in mixed traffic conditions. This paper explores fundamental principles and combination of algorithms, to accurately detect, classify and track surrounding vehicles on expressways using a vehicle mounted cost-effective LiDAR sensor in dynamic conditions. The proposed approach employs algorithms such as Density-Based Spatial Clustering of Applications with Noise, Random sample consensus, and Kalman filter on 3D LiDAR data for ground and non-ground points segmentation, object clustering, road boundary identification, vehicle classification and tracking. Trajectory data of the tracked vehicles is used to estimate their relative positions and speeds. The surrounding vehicles were classified into three categories: two-wheelers, four-wheelers and heavy-vehicles. The proposed method achieves an accuracy, precision, recall, and F1-score above 94 %, demonstrating its robustness. The resilience and versatility of the proposed algorithm were validated through comprehensive evaluations on a significant dataset spanning over 8000 km (around 3.6 million frames) of driving data collected under varied environmental scenarios on expressway. Further, various applications of proposed methodology in road safety studies are discussed. The approach can extract real-time micro-level trajectory data for surrounding vehicles on expressways in any lighting condition, which is useful for researchers and can be used in connected and autonomous vehicles. It also provides valuable information for assessing surrogate safety measures and identifying road safety issues and suggesting countermeasures, including advanced and cost-effective driver assistance system applications.

Advancing real-world visual SLAM: Integrating adaptive segmentation with dynamic object detection for enhanced environmental perception

The field of Visual Simultaneous Localization and Mapping (VSLAM) has long grappled with the limitations imposed by static environmental assumptions, a challenge that becomes increasingly pronounced in dynamic settings. Addressing this gap, our study introduces a pioneering technique, Accurate and Dynamic Reliable SLAM (YoloV8-SLAM), which represents a paradigm shift in VSLAM technology. This novel approach combines the prowess of dynamic object detection with advanced multi-view geometry, thereby elevating the system’s accuracy, robustness, and efficiency in varied dynamic scenarios. At the core of YoloV8-SLAM is an innovative adaptive segmentation process that skilfully merges geometric motion details from successive frames. This integration not only isolates dynamic objects with heightened precision but also ensures consistent mapping accuracy. The system employs the latest YoloV8 model, synergizing it with an Enhanced Multi-View Geometry algorithm. This combination is adept at handling a spectrum of dynamic conditions, further bolstered by a precise point selection mechanism that swiftly retrieves motion information, crucial for real-time applications. Our extensive evaluations demonstrate that YoloV8-SLAM significantly outperforms existing methods such as including DynaSLAM (Bescos et al., 2018), DM-SLAM (Cheng et al., 2020), RDS-SLAM (Liu et al., 2021), Blitz-SLAM (Fan et al., 2022), and OVD-SLAM (He et al., 2023). Notably, it achieves a 38–45% reduction in Absolute Trajectory Error and a 66–72% decrease in Relative Pose Error, underscoring its superior performance. The breakthrough of YoloV8-SLAM lies in its seamless amalgamation of cutting-edge machine learning with robust geometric principles. This fusion not only facilitates effective dynamic segmentation but also ensures stable structural reconstruction, even in highly dynamic environments. This work marks a substantial advancement in SLAM technology, particularly in addressing the complexities of dynamic and unrestricted settings. YoloV8-SLAM sets a new benchmark in the field and paves the way for future explorations in dynamic SLAM solutions, promising to revolutionize applications requiring high fidelity in real-world, ever-changing environments.

FFD-SLAM: A Real-Time Visual SLAM Toward Dynamic Scenes with Semantic and Optical Flow Information

To solve the problem of poor localization accuracy and robustness of visual simultaneous localization and mapping (SLAM) systems in highly dynamic environments, this paper proposes a dynamic visual SLAM algorithm called FFD-SLAM that fuses the target detection network with the optical flow method. The algorithm considers ORB-SLAM2 as the basic framework, joins the semantic thread in parallel with its tracking thread, initially obtains the set of feature points through the real-time detection of dynamic objects in the environment through YOLOv5 in the semantic thread, then filters the set of feature points obtained in the semantic thread through the optical flow module, and finally utilizes the remaining static feature points for the matching calculation. Experiments showed that the proposed algorithm showed an improvement of approximately 97% in the localization accuracy compared with the ORB-SLAM2 algorithm in a highly dynamic environment, which effectively improves the localization accuracy and robustness of the system. The proposed algorithm also showed a higher real-time performance compared with some excellent dynamic SLAM algorithms.

Visual Reinforcement Learning for Dynamic Object Detection

Abstract. Object detection is a widely studied task in computer vision. Current methods often focus on images captured from appropriate viewpoints. However, there is a large disparity between objects observed from different viewpoints in the real world. Dynamic Object Detection (DOD) method automatically adjusts the camera viewpoint in a visual scene to sequentially find optimal viewpoints. Currently, the DOD tasks are usually modeled as a sequential decision-making problem and solved using reinforcement learning methods. Existing approaches face challenges with sparse rewards and training instability. To tackle these issues, we proposed a single-step reward function and a lightweight network, respectively. The single-step reward function, which provides timely feedback, gives an efficient training process for DOD tasks. The lightweight network with few parameters can ensure the stability of the training process. To evaluate the effectiveness of our method, we developed a simulation dataset based on UE4, which consists of 1800 training images and 450 testing images. The dataset includes five object categories: vans, cars, trailers, box trucks and SUVs. Experiments demonstrate that our method outperforms SOTA object detectors on our simulation dataset. Specifically, the average precisions(APs) are improved from 89.1% to 96.0% when using the YOLOv8 object detector.

PSMD-SLAM: Panoptic Segmentation-Aided Multi-Sensor Fusion Simultaneous Localization and Mapping in Dynamic Scenes

Multi-sensor fusion is pivotal in augmenting the robustness and precision of simultaneous localization and mapping (SLAM) systems. The LiDAR–visual–inertial approach has been empirically shown to adeptly amalgamate the benefits of these sensors for SLAM across various scenarios. Furthermore, methods of panoptic segmentation have been introduced to deliver pixel-level semantic and instance segmentation data in a single instance. This paper delves deeper into these methodologies, introducing PSMD-SLAM, a novel panoptic segmentation assisted multi-sensor fusion SLAM approach tailored for dynamic environments. Our approach employs both probability propagation-based and PCA-based clustering techniques, supplemented by panoptic segmentation. This is utilized for dynamic object detection and the removal of visual and LiDAR data, respectively. Furthermore, we introduce a module designed for the robust real-time estimation of the 6D pose of dynamic objects. We test our approach on a publicly available dataset and show that PSMD-SLAM outperforms other SLAM algorithms in terms of accuracy and robustness, especially in dynamic environments.

A structural description of pedestrian movement behavior in multiple surveillance videos

This study purpose of recording pedestrians' activities in all surveillance videos and then convert these records into structured descriptions to analyze a large number of surveillance videos. This paper proposes a four-stage processing method. The first stage uses convolutional neural network (CNN) to detect objects in the surveillance video and classify them. The second stage finds the correlation of pedestrians in continuous images and describes multiple objects' trajectories in the surveillance video. The third stage calculates the similarity of the pedestrian object sets of the surveillance video at different times as the basis for pedestrian re-identification. In the fourth stage, extract the walking path of the pedestrian in each surveillance video. Each pedestrian's actions in each surveillance video, including whether to stand, walking path, direction, clothing color and walking time. To construct a correlation table of pedestrian motion in multiple surveillance videos.This experiment uses two image datasets for experimentation and analysis to evaluate this method's effectiveness for pedestrian movement in multiple surveillance videos. The image datasets are the Multi-Camera Object Tracking (MCT) Challenge and PETS 2009 Benchmark Data. This experiment includes pedestrian detection, pedestrian identification, and multi-pedestrian tracking of pedestrians in single and multiple surveillance cameras. The results show that in the pedestrian detection experiment, the convolutional neural network can effectively solve the problems caused by static objects, light changes, similar colors, crowded pedestrians, and incomplete objects. In the pedestrian recognition and re-recognition experiments, the accuracy is reduced due to the influence of low-resolution surveillance videos. This detection method of the pedestrian is more accurate than traditional dynamic object detection methods. After the pedestrian tracking and re-identification are completed, the pedestrian relationships in multiple surveillance cameras are integrated, and a pedestrian behavior association table is built for surveillance videos.

Dynamic Object Detection in Surveillance Videos using Temporal Convolutional Networks and Federated Learning in Edge Computing Environments

This research addresses the importance of advancing dynamic object detection in surveillance videos by introducing a novel framework that integrates Temporal Convolutional Networks (TCNs) and Federated Learning (FL) within edge computing environments. This research is motivated by the critical need for real-time threat response, enhanced security measures, and privacy preservation in dynamic surveillance scenarios. Leveraging TCNs, the system captures temporal dependencies, providing a comprehensive understanding of object movements. FL ensures decentralized model training, mitigating privacy concerns associated with centralized approaches. Current challenges in real-time processing, privacy preservation, and adaptability to dynamic environments are addressed through innovative solutions. Model optimization techniques optimize TCN efficiency, ensuring real-time processing. Advanced privacy-preserving mechanisms secure FL, addressing privacy concerns. Transfer learning and data augmentation enhance adaptability to dynamic scenarios. The proposed system not only addresses existing challenges but also contributes to the evolution of surveillance technology. Comprehensive metrics, including accuracy, sensitivity, specificity, and real-time processing metrics, provide a thorough evaluation of the system's performance. This research introduces an approach to dynamic object detection, combining TCN and FL in edge computing environments. Results show accuracy exceeding 97%, sensitivity and specificity at 97% and 98%, and F1 score reaching 96%.

A feature enhancement FCOS algorithm for dynamic traffic object detection

The development of object detection plays an important role in the realisation of fully autonomous driving, and the feature extraction is the key step for object detection. There has been significant difference and scale variation of object features for different road traffic participants (RTPs), meanwhile traditional Convolutional Neural Networks (CNNs) was difficult to extract object features efficiently for small targets. In order to improve the ability of feature extraction, a RTP object detection method combining dynamic convolution and feature enhancement was proposed. The Fully Convolutional One-Stage (FCOS) object detection algorithm was used as baseline. First, the dynamic convolution module was designed in the backbone network to identify different object features to the maximum extent. Second, a dual attention module was designed to filter object feature information while reducing the amount of computation. Finally, in the detection part, the feature expression ability of shallow network was further enhanced by multi-scale feature fusion module, and the effectiveness of the proposed algorithm was verified using Cityscapes dataset. The experimental result indicated that mAP increased by 2.3% compared with baseline. This study can improve the efficiency of RTP detection and contribute to the industrialisation of intelligent connected vehicles.

Efficient LiDAR-Trajectory Affinity Model for Autonomous Vehicle Orchestration

Computation and memory resource management strategies are the backbone of continuous object tracking in intelligent vehicle orchestration. Multi-object tracking generates enormous measurements of targets and extended object positions using light detection and ranging (Lidar) sensors. Designing an adequate object-tracking system is a global challenge because of dynamic object detection and data association uncertainties during scene understanding. In this regard, we develop an intelligent multi-objective tracking (IMOT) system with a novel measurement model, called the box data association inflate (BDAI) model, to assess each target’s object state and trajectory without noise by using the Bayesian approach. The box object filter method filters ambiguous detection responses during data association. The theoretical proof of the box object filter is derived based on binomial expansion. Prognosticating a lower-dimension object than the original point object reduces the computational complexity of vehicle orchestration. Two datasets (NuScenes dataset and our lab dataset) are considered during the simulations, and our approach measures the kinematic states adequately with reduced computation complexity compared to state-of-the-art methods. The simulation outcomes show that our proposed method is effective and works well to detect and track objects. The NuScenes dataset contains 28130 samples for training, 6019 examples for validation and 6008 samples for testing. IMOT achieves 58.09% tracking accuracy and 71% mAP with 5 ms pre-processing time. The Jetson Xavier NX consumes 49.63% GPU and 9.37% average power and exhibits 25.32 ms latency compared to other approaches. Our system trains a single pair frame in 169.71 ms with affinity estimation time of 12.19 ms, track association time of 0.19 ms and mATE of 0.245 compared to state-of-the-art approaches

Surgical robot navigation based on SLAM technology

With the widespread application of surgical robots and the development of computer vision, SLAM-applicated surgery is receiving increasing attention. However, due to the unique surgical environment, SLAM faces some challenges. Two key issues will be discussed in this article: dynamic object detection and image segmentation, as well as scene reconstruction under data scarcity. Firstly, dynamic object detection and image segmentation is an important issue in SLAM applications. During the surgical process, doctors often use surgical instruments, which may partially or completely obscure the object, making it difficult to detect the target. Methods based on traditional feature matching may not be able to accurately detect dynamic targets perform image segmentation. Therefore, this article will combine semantic networks for analysis to improve the performance. In addition, scene reconstruction under data scarcity is another challenge in SLAM applications. Traditional SLAM methods typically rely on a large amount of feature points or map data. But in surgery, due to the complexity of occlusion and geometric structure, reliable data may not be easily obtained. This article will develop with the steps of reconstruction and analyze feasible methods that can improve the accuracy and stability of reconstruction. To conclude, this article will concentrate on these two issues, analyze recent papers, and ultimately summarize some feasible solutions, providing ideas and references for other researchers in this field.

Semantic geometric fusion multi-object tracking and lidar odometry in dynamic environment

AbstractSimultaneous localization and mapping systems based on rigid scene assumptions cannot achieve reliable positioning and mapping in a complex environment with many moving objects. To solve this problem, this paper proposes a novel dynamic multi-object lidar odometry (MLO) system based on semantic object recognition technology. The proposed system enables the reliable localization of robots and semantic objects and the generation of long-term static maps in complex dynamic scenes. For ego-motion estimation, the proposed system extracts environmental features that take into account both semantic and geometric consistency constraints. Then, the filtered features can be robust to the semantic movable and unknown dynamic objects. In addition, we propose a new least-squares estimator that uses geometric object points and semantic box planes to realize the multi-object tracking (SGF-MOT) task robustly and precisely. In the mapping module, we implement dynamic semantic object detection using the absolute trajectory tracking list. By using static semantic objects and environmental features, the system eliminates accumulated localization errors and produces a purely static map. Experiments on the public KITTI dataset show that the proposed MLO system provides more accurate and robust object tracking performance and better real-time localization accuracy in complex scenes compared to existing technologies.

Dynamic Object Detection and Tracking in Vision SLAM

Abstract Addressing the trade-off between operational efficiency and localization accuracy in visual SLAM, this paper introduces a monocular visual-inertial SLAM algorithm that integrates point and line features. To construct the point-line reprojection error and optimize the observation volume in front-end vision initialization, the motion recovery structure method (SFM) is employed through 3D reconstruction with a sliding window. The marginalization method uses the removed keyframe information as a priori constraint for nonlinear optimization in the back-end. In addition, the loopback detection algorithm is optimized in combination with the bag-of-words model and four-degree-of-freedom global bitmap to improve the accuracy of dynamic object detection, and the performance of the algorithm is tested. The results show that in the case of no closed loop, the absolute root mean square error of the algorithm proposed in this paper is lower than that of VINS-Mono (0.0625), PL-VIO (0.0401), and PL-VINS (0.0554) for the majority of sequences. In the case of closed loops, the absolute root mean square error of the proposed algorithm in this paper is reduced by 0.0395 and 0.0139 on average over most sequences compared to VINS-Mono and PL-VINS. The proposed algorithm in this paper demonstrates higher accuracy and robustness for improved detection and tracking of dynamic objects.

DLOAM: Real-time and Robust LiDAR SLAM System Based on CNN in Dynamic Urban Environments

Dynamic object detection, state estimation, and map-building are crucial for autonomous robot systems and intelligent transportation applications in urban scenarios. Most current LiDAR Simultaneous Localization and Mapping (SLAM) systems operate on the assumption that the observed environment is static. However, the overall accuracy and robustness of a SLAM system can be compromised by dynamic objects in the environment. Aiming at the problem of inaccurate odometry estimation and wrong mapping caused by the existing LiDAR SLAM method which cannot detect the dynamic objects, we study the SLAM problem of robots and unmanned vehicles equipped with LiDAR traveling in the dynamic urban scenes. We propose a fast LiDAR-only model-free dynamic objects detection method, which uses the spatial and temporal information of point cloud through a convolutional neural network (CNN), and the detection accuracy is improved by 35 use spatial information. We further integrate it into a state-of-the-art LiDAR SLAM framework to improve the SLAM performance. Firstly, the range image constructed by LiDAR point cloud is used for ground extraction and non-ground point clustering. Then, the motion of objects in the scene is estimated by the difference between adjacent frames, and the segmented objects are further divided into dynamic objects and static objects by their motion features. After that, the stable feature points are extracted from the static objects. Finally, the pose transformation of adjacent frames is solved by matching feature point pairs. We evaluated the accuracy and robustness of our system on datasets with different challenging dynamic environments, and the results show our system has significant improvements in accuracy and robustness of odometry and mapping, while still maintain real-time performance, which is sufficient for autonomous robot systems and intelligent transportation applications in urban scenarios.

Application of a modified neural network YOLOv5 for recognition of indoors moving people

It is proposed to improve the visual dynamic SLAM system by combining the YOLOv5 neural network with the attention mechanism modules CBAM and BiFPN. Special attention is paid to the effective detection of dynamic objects (people), solving problems of image occlusion and small object detection, and increase of overall system performance in dynamic environments. The obtained results demonstrate the effectiveness of the method and its potential for application in robotics and autonomous vehicles. Keywords CBAM attention mechanism, bidirectional feature pyramid network, person detection, occlusion, dynamic SLAM, YOLO detector

Dynamic Object Detection Revolution: Deep Learning with Attention, Semantic Understanding, and Instance Segmentation for Real-World Precision

Dynamic Object Detection Revolution: Deep Learning with Attention, Semantic Understanding, and Instance Segmentation for Real-World Precision

Dynamic and Real-Time Object Detection Based on Deep Learning for Home Service Robots

Home service robots operating indoors, such as inside houses and offices, require the real-time and accurate identification and location of target objects to perform service tasks efficiently. However, images captured by visual sensors while in motion states usually contain varying degrees of blurriness, presenting a significant challenge for object detection. In particular, daily life scenes contain small objects like fruits and tableware, which are often occluded, further complicating object recognition and positioning. A dynamic and real-time object detection algorithm is proposed for home service robots. This is composed of an image deblurring algorithm and an object detection algorithm. To improve the clarity of motion-blurred images, the DA-Multi-DCGAN algorithm is proposed. It comprises an embedded dynamic adjustment mechanism and a multimodal multiscale fusion structure based on robot motion and surrounding environmental information, enabling the deblurring processing of images that are captured under different motion states. Compared with DeblurGAN, DA-Multi-DCGAN had a 5.07 improvement in Peak Signal-to-Noise Ratio (PSNR) and a 0.022 improvement in Structural Similarity (SSIM). An AT-LI-YOLO method is proposed for small and occluded object detection. Based on depthwise separable convolution, this method highlights key areas and integrates salient features by embedding the attention module in the AT-Resblock to improve the sensitivity and detection precision of small objects and partially occluded objects. It also employs a lightweight network unit Lightblock to reduce the network’s parameters and computational complexity, which improves its computational efficiency. Compared with YOLOv3, the mean average precision (mAP) of AT-LI-YOLO increased by 3.19%, and the detection precision of small objects, such as apples and oranges and partially occluded objects, increased by 19.12% and 29.52%, respectively. Moreover, the model inference efficiency had a 7 ms reduction in processing time. Based on the typical home activities of older people and children, the dataset Grasp-17 was established for the training and testing of the proposed method. Using the TensorRT neural network inference engine of the developed service robot prototype, the proposed dynamic and real-time object detection algorithm required 29 ms, which meets the real-time requirement of smooth vision.