Real-time Simultaneous Localization And Mapping Research Articles

AFO-SLAM: an improved visual SLAM in dynamic scenes using acceleration of feature extraction and object detection

Abstract In visual simultaneous localization and mapping (SLAM) systems, traditional methods often excel due to rigid environmental assumptions, but face challenges in dynamic environments. To address this, learning-based approaches have been introduced, but their expensive computing costs hinder real-time performance, especially on embedded mobile platforms. In this article, we propose a robust and real-time visual SLAM method towards dynamic environments using acceleration of feature extraction and object detection (AFO-SLAM). First, AFO-SLAM employs an independent object detection thread that utilizes YOLOv5 to extract semantic information and identify the bounding boxes of moving objects. To preserve the background points within these boxes, depth information is utilized to segment target foreground and background with only a single frame, with the points of the foreground area considered as dynamic points and then rejected. To optimize performance, CUDA program accelerates feature extraction preceding point removal. Finally, extensive evaluations are performed on both TUM RGB-D dataset and real scenes using a low-power embedded platform. Experimental results demonstrate that AFO-SLAM offers a balance between accuracy and real-time performance on embedded platforms, and enables the generation of dense point cloud maps in dynamic scenarios.

FFD-SLAM: A Real-Time Visual SLAM Toward Dynamic Scenes with Semantic and Optical Flow Information

To solve the problem of poor localization accuracy and robustness of visual simultaneous localization and mapping (SLAM) systems in highly dynamic environments, this paper proposes a dynamic visual SLAM algorithm called FFD-SLAM that fuses the target detection network with the optical flow method. The algorithm considers ORB-SLAM2 as the basic framework, joins the semantic thread in parallel with its tracking thread, initially obtains the set of feature points through the real-time detection of dynamic objects in the environment through YOLOv5 in the semantic thread, then filters the set of feature points obtained in the semantic thread through the optical flow module, and finally utilizes the remaining static feature points for the matching calculation. Experiments showed that the proposed algorithm showed an improvement of approximately 97% in the localization accuracy compared with the ORB-SLAM2 algorithm in a highly dynamic environment, which effectively improves the localization accuracy and robustness of the system. The proposed algorithm also showed a higher real-time performance compared with some excellent dynamic SLAM algorithms.

A Compact Handheld Sensor Package with Sensor Fusion for Comprehensive and Robust 3D Mapping.

This paper introduces an innovative approach to 3D environmental mapping through the integration of a compact, handheld sensor package with a two-stage sensor fusion pipeline. The sensor package, incorporating LiDAR, IMU, RGB, and thermal cameras, enables comprehensive and robust 3D mapping of various environments. By leveraging Simultaneous Localization and Mapping (SLAM) and thermal imaging, our solution offers good performance in conditions where global positioning is unavailable and in visually degraded environments. The sensor package runs a real-time LiDAR-Inertial SLAM algorithm, generating a dense point cloud map that accurately reconstructs the geometric features of the environment. Following the acquisition of that point cloud, we post-process these data by fusing them with images from the RGB and thermal cameras and produce a detailed, color-enriched 3D map that is useful and adaptable to different mission requirements. We demonstrated our system in a variety of scenarios, from indoor to outdoor conditions, and the results showcased the effectiveness and applicability of our sensor package and fusion pipeline. This system can be applied in a wide range of applications, ranging from autonomous navigation to smart agriculture, and has the potential to make a substantial benefit across diverse fields.

Real-time visual SLAM based YOLO-Fastest for dynamic scenes

Within the realm of autonomous robotic navigation, simultaneous localization and mapping (SLAM) serves as a critical perception technology, drawing heightened attention in contemporary research. The traditional SLAM systems perform well in static environments, but in the real physical world, dynamic objects can destroy the static geometric constraints of the SLAM system, further limiting its practical application in the real world. In this paper, a robust dynamic RGB-D SLAM system is proposed to expand the number of static points in the scene by combining with YOLO-Fastest to ensure the effectiveness of the geometric constraints model construction, and then based on that, a new thresholding model is designed to differentiate the dynamic features in the objection bounding box, which takes advantage of the double polyline constraints and the residuals after reprojection to filter the dynamic feature points. In addition, two Gaussian models are constructed to segment the moving objects in the bounding box in the depth image to achieve the effect similar to the instance segmentation under the premise of ensuring the computational speed. In this paper, experiments are conducted on dynamic sequences provided by the TUM dataset to evaluate the performance of the proposed method, and the results show that the root mean squared error metric of the absolute trajectory error of the algorithm of this paper has at least 80% improvement compared to ORB-SLAM2. Higher robustness in dynamic environments with both high and low dynamic sequences compared to DS-SLAM and Dynaslam, and can effectively provide intelligent localization and navigation for mobile robots.

Hierarchical fusion based high precision SLAM for solid-state lidar

Abstract Solid-state LiDARs have become an important perceptual device for simultaneous localization and mapping (SLAM) due to its low-cost and high-reliability compared to mechanical LiDARs. Nevertheless, existing solid-state LiDARs-based SLAM methods face challenges, including drift and mapping inconsistency, when operating in dynamic environments over extended periods and long distances. To this end, this paper proposes a robust, high-precision, real-time LiDAR-inertial SLAM method for solid-state LiDARs. At the front-end, the raw point cloud is segmented to filter dynamic points in preprocessing process. Subsequently, features are extracted using a combination of Principal Component Analysis (PCA) and Mean Clustering to reduce redundant points and improve data processing efficiency. At the back-end, a hierarchical fusion method is proposed to improve the accuracy of the system by fusing the feature information to iteratively optimize the LiDAR frames, and then adaptively selecting the LiDAR keyframes to be fused with the IMU. The proposed method is extensively evaluated using a Livox Avia solid-state LiDAR collecting datasets on two different platforms. In experiments, the end-to-end error is reduced by 35% and the single-frame operational efficiency is improved by 12% compared to LiLi-OM.

UDS-SLAM: real-time robust visual SLAM based on semantic segmentation in dynamic scenes

PurposeExisting Simultaneous Localization and Mapping (SLAM) algorithms have been relatively well developed. However, when in complex dynamic environments, the movement of the dynamic points on the dynamic objects in the image in the mapping can have an impact on the observation of the system, and thus there will be biases and errors in the position estimation and the creation of map points. The aim of this paper is to achieve more accurate accuracy in SLAM algorithms compared to traditional methods through semantic approaches.Design/methodology/approachIn this paper, the semantic segmentation of dynamic objects is realized based on U-Net semantic segmentation network, followed by motion consistency detection through motion detection method to determine whether the segmented objects are moving in the current scene or not, and combined with the motion compensation method to eliminate dynamic points and compensate for the current local image, so as to make the system robust.FindingsExperiments comparing the effect of detecting dynamic points and removing outliers are conducted on a dynamic data set of Technische Universität München, and the results show that the absolute trajectory accuracy of this paper's method is significantly improved compared with ORB-SLAM3 and DS-SLAM.Originality/valueIn this paper, in the semantic segmentation network part, the segmentation mask is combined with the method of dynamic point detection, elimination and compensation, which reduces the influence of dynamic objects, thus effectively improving the accuracy of localization in dynamic environments.

DLD-SLAM: RGB-D Visual Simultaneous Localisation and Mapping in Indoor Dynamic Environments Based on Deep Learning

This work presents a novel RGB-D dynamic Simultaneous Localisation and Mapping (SLAM) method that improves the precision, stability, and efficiency of localisation while relying on lightweight deep learning in a dynamic environment compared to the traditional static feature-based visual SLAM algorithm. Based on ORB-SLAM3, the GCNv2-tiny network instead of the ORB method, improves the reliability of feature extraction and matching and the accuracy of position estimation; then, the semantic segmentation thread employs the lightweight YOLOv5s object detection algorithm based on the GSConv network combined with a depth image to determine potentially dynamic regions of the image. Finally, to guarantee that the static feature points are used for position estimation, dynamic probability is employed to determine the true dynamic feature points based on the optical flow, semantic labels, and the state in last frame. We have performed experiments on the TUM datasets to verify the feasibility of the algorithm. Compared with the classical dynamic visual SLAM algorithm, the experimental results demonstrate that the absolute trajectory error is greatly reduced in dynamic environments, and that the computing efficiency is improved by 31.54% compared with the real-time dynamic visual SLAM algorithm with close accuracy, demonstrating the superiority of DLD-SLAM in accuracy, stability, and efficiency.

DLOAM: Real-time and Robust LiDAR SLAM System Based on CNN in Dynamic Urban Environments

Dynamic object detection, state estimation, and map-building are crucial for autonomous robot systems and intelligent transportation applications in urban scenarios. Most current LiDAR Simultaneous Localization and Mapping (SLAM) systems operate on the assumption that the observed environment is static. However, the overall accuracy and robustness of a SLAM system can be compromised by dynamic objects in the environment. Aiming at the problem of inaccurate odometry estimation and wrong mapping caused by the existing LiDAR SLAM method which cannot detect the dynamic objects, we study the SLAM problem of robots and unmanned vehicles equipped with LiDAR traveling in the dynamic urban scenes. We propose a fast LiDAR-only model-free dynamic objects detection method, which uses the spatial and temporal information of point cloud through a convolutional neural network (CNN), and the detection accuracy is improved by 35 use spatial information. We further integrate it into a state-of-the-art LiDAR SLAM framework to improve the SLAM performance. Firstly, the range image constructed by LiDAR point cloud is used for ground extraction and non-ground point clustering. Then, the motion of objects in the scene is estimated by the difference between adjacent frames, and the segmented objects are further divided into dynamic objects and static objects by their motion features. After that, the stable feature points are extracted from the static objects. Finally, the pose transformation of adjacent frames is solved by matching feature point pairs. We evaluated the accuracy and robustness of our system on datasets with different challenging dynamic environments, and the results show our system has significant improvements in accuracy and robustness of odometry and mapping, while still maintain real-time performance, which is sufficient for autonomous robot systems and intelligent transportation applications in urban scenarios.

SCE-SLAM: a real-time semantic RGBD SLAM system in dynamic scenes based on spatial coordinate error

Simultaneous localization and mapping (SLAM) is one of the prerequisite technologies for intelligent mobile robots to accomplish various tasks in unknown environments. In recent years, many excellent SLAM systems have emerged, but most of them have a basic assumption that the environment is static, which results in their poor performance in dynamic environments. To solve this problem, this paper presents SCE-SLAM: a novel real-time semantic RGB-D SLAM system that is built on the RGB-D mode of ORB-SLAM3. SCE-SLAM tightly combines semantic and geometric information. Considering the real-time requirements, the semantic module provides semantic prior knowledge for the geometric module using the latest and fastest object detection network YOLOv7. Then, a new geometric constraint method is proposed to filter dynamic feature points. This method takes full advantage of depth images and semantic information to recover three-dimensional (3D) feature points and the initial camera pose. A 3D coordinate error is used as a threshold, and SCE-SLAM removes dynamic points using the K-means clustering algorithm. In this way, SCE-SLAM effectively reduces the impact of dynamic points. Furthermore, we validate SCE-SLAM with challenging dynamic sequences of the TUM dataset. The results demonstrate that SCE-SLAM significantly improves the localization accuracy and system robustness in all kinds of dynamic environments.

AGRI-SLAM: a real-time stereo visual SLAM for agricultural environment

In this research, we proposed a stereo visual simultaneous localisation and mapping (SLAM) system that efficiently works in agricultural scenarios without compromising the performance and accuracy in contrast to the other state-of-the-art methods. The proposed system is equipped with an image enhancement technique for the ORB point and LSD line features recovery, which enables it to work in broader scenarios and gives extensive spatial information from the low-light and hazy agricultural environment. Firstly, the method has been tested on the standard dataset, i.e., KITTI and EuRoC, to validate the localisation accuracy by comparing it with the other state-of-the-art methods, namely VINS-SLAM, PL-SLAM, and ORB-SLAM2. The experimental results evidence that the proposed method obtains superior localisation and mapping accuracy than the other visual SLAM methods. Secondly, the proposed method is tested on the ROSARIO dataset, our low-light agricultural dataset, and O-HAZE dataset to validate the performance in agricultural environments. In such cases, while other methods fail to operate in such complex agricultural environments, our method successfully operates with high localisation and mapping accuracy.

A Real-Time Stereo Visual-Inertial SLAM System Based on Point-and-Line Features

Simultaneous localization and mapping (SLAM) is widely used in various fields, such as unmanned driving, robotics, and VR. The SLAM system with multiple landmarks is also a research hotspot currently. In this study, a real-time stereo visual-inertial SLAM system based on point-and-line features is proposed; the system is studied based on VINS-fusion. It improves the front-end process of VINS fusion. First, we proposed an IMU-assisted hierarchical grid optical flow tracking method that can more accurately and quickly tracks points between frames. Second, we add line features on the basis of existing point features. To match line features in real time, we only select the best line features to participate in optimization. We combine line segments by geometric relation to reduce the line segment splitting and redundancy brought by LSD algorithm, and further predict line features by IMU-assisted optical flow tracking to achieve higher precision matching. In the back-end optimization process, we used a redundant structure to avoid the failure of stereo constraints in a highly dynamic environment. The proposed method outperformed the state-of-the-art methods (VINS-fusion and PL-VINS) on the EuRoC MAV dataset. The system also achieved good performance in a real environment.

A real-time semantic visual SLAM for dynamic environment based on deep learning and dynamic probabilistic propagation

Most existing visual simultaneous localization and mapping (SLAM) algorithms rely heavily on the static world assumption. Combined with deep learning, semantic SLAM has become a popular solution for dynamic scenes. However, most semantic SLAM methods show poor real-time performance when dealing with dynamic scenes. To handle this problem, a real-time semantic SLAM method is proposed in this paper, combining knowledge distillation and dynamic probability propagation strategy. First, to improve the execution speed, a multi-level knowledge distillation method is adopted to obtain a lightweight segmentation model, which is more suitable for continuous frames to create an independent semantic segmentation thread. This segmentation thread only accepts keyframes as input so that the system can avoid time delay caused by processing each frame. Second, a static semantic keyframe selection strategy is proposed based on the segmentation results. In this way, those keyframes containing more static information will be selected to reduce the participation of dynamic objects. By combining segmentation results and data matching algorithm, our system can realize the update and propagation of dynamic probability, reducing the influence of dynamic points in the pose optimization process. Validation results based on the KITTI and TUM datasets show that our method can effectively deal with dynamic feature points and improve running speed simultaneously.

MOP-SLAM: A real time SLAM system based on multi-head optical flow estimation network

SLAM (simultaneous localization and mapping) is becoming a significant technology in driverless vehicles nowadays. Sometimes the optical flow method can be used to track feature points to reduce the amount of calculation. But the traditional optical flow method is based on the assumption that the luminosity will not change and has high requirements for the environment. Therefore, an optical flow calculation method based on neural network called Optical-Net is proposed in this paper to match and track feature and corner points. We use the multi-head attention module to extract and fuse the features of different scales. The training results show that the Optical-Net designed in this study effectively improves the robustness and accuracy of optical flow estimation when the object has large displacement and small displacement. Meanwhile, to address the problem of reduced matching accuracy during video input due to dynamic blurring, a video dynamic deblurring algorithm based on the one-dimensional Wineman filter is adopted to preprocess the input video. Finally, we replaced the tracking thread in traditional SLAM. The performance of the improved algorithm is verified by experiments on our own driverless car.

Improved Real-Time Monocular SLAM Using Semantic Segmentation on Selective Frames

Monocular simultaneous localization and mapping (SLAM) is emerging in advanced driver assistance systems and autonomous driving, because a single camera is cheap and easy to install. Conventional monocular SLAM has two major challenges leading inaccurate localization and mapping. First, it is challenging to estimate scales in localization and mapping. Second, conventional monocular SLAM uses inappropriate mapping factors such as dynamic objects and low-parallax areas in mapping. This paper proposes an improved real-time monocular SLAM that resolves the aforementioned challenges by efficiently using deep learning-based semantic segmentation. To achieve the real-time execution of the proposed method, we apply semantic segmentation only to downsampled keyframes in parallel with mapping processes. In addition, the proposed method corrects scales of camera poses and three-dimensional (3D) points, using estimated ground plane from road-labeled 3D points and the real camera height. The proposed method also removes inappropriate corner features labeled as moving objects and low parallax areas. Experiments with eight video sequences demonstrate that the proposed monocular SLAM system achieves significantly improved and comparable trajectory tracking accuracy, compared to existing state-of-the-art monocular and stereo SLAM systems, respectively. The proposed system can achieve real-time tracking on a standard CPU potentially with a standard GPU support, whereas existing segmentation-aided monocular SLAM does not.

An Improved Visual SLAM Based on Map Point Reliability under Dynamic Environments

The visual simultaneous localization and mapping (SLAM) method under dynamic environments is a hot and challenging issue in the robotic field. The oriented FAST and Rotated BRIEF (ORB) SLAM algorithm is one of the most effective methods. However, the traditional ORB-SLAM algorithm cannot perform well in dynamic environments due to the feature points of dynamic map points at different timestamps being incorrectly matched. To deal with this problem, an improved visual SLAM method built on ORB-SLAM3 is proposed in this paper. In the proposed method, an improved new map points screening strategy and the repeated exiting map points elimination strategy are presented and combined to identify obvious dynamic map points. Then, a concept of map point reliability is introduced in the ORB-SLAM3 framework. Based on the proposed reliability calculation of the map points, a multi-period check strategy is used to identify the unobvious dynamic map points, which can further deal with the dynamic problem in visual SLAM, for those unobvious dynamic objects. Finally, various experiments are conducted on the challenging dynamic sequences of the TUM RGB-D dataset to evaluate the performance of our visual SLAM method. The experimental results demonstrate that our SLAM method can run at an average time of 17.51 ms per frame. Compared with ORB-SLAM3, the average RMSE of the absolute trajectory error (ATE) of the proposed method in nine dynamic sequences of the TUM RGB-D dataset can be reduced by 63.31%. Compared with the real-time dynamic SLAM methods, the proposed method can obtain state-of-the-art performance. The results prove that the proposed method is a real-time visual SLAM, which is effective in dynamic environments.

HFNet-SLAM: An Accurate and Real-Time Monocular SLAM System with Deep Features.

Image tracking and retrieval strategies are of vital importance in visual Simultaneous Localization and Mapping (SLAM) systems. For most state-of-the-art systems, hand-crafted features and bag-of-words (BoW) algorithms are the common solutions. Recent research reports the vulnerability of these traditional algorithms in complex environments. To replace these methods, this work proposes HFNet-SLAM, an accurate and real-time monocular SLAM system built on the ORB-SLAM3 framework incorporated with deep convolutional neural networks (CNNs). This work provides a pipeline of feature extraction, keypoint matching, and loop detection fully based on features from CNNs. The performance of this system has been validated on public datasets against other state-of-the-art algorithms. The results reveal that the HFNet-SLAM achieves the lowest errors among systems available in the literature. Notably, the HFNet-SLAM obtains an average accuracy of 2.8 cm in EuRoC dataset in pure visual configuration. Besides, it doubles the accuracy in medium and large environments in TUM-VI dataset compared with ORB-SLAM3. Furthermore, with the optimisation of TensorRT technology, the entire system can run in real-time at 50 FPS.

SG-SLAM: A Real-Time RGB-D Visual SLAM Toward Dynamic Scenes With Semantic and Geometric Information

Simultaneous localization and mapping (SLAM) is one of the fundamental capabilities for intelligent mobile robots to perform state estimation in unknown environments. However, most visual SLAM systems rely on the static scene assumption and consequently have severely reduced accuracy and robustness in dynamic scenes. Moreover, the metric maps constructed by many systems lack semantic information, so the robots cannot understand their surroundings at a human cognitive level. In this article, we propose SG-SLAM, which is a real-time RGB-D semantic visual SLAM system based on the ORB-SLAM2 framework. First, SG-SLAM adds two new parallel threads: an object detecting thread to obtain 2-D semantic information and a semantic mapping thread. Then, a fast dynamic feature rejection algorithm combining semantic and geometric information is added to the tracking thread. Finally, they are published to the robot operating system (ROS) system for visualization after generating 3-D point clouds and 3-D semantic objects in the semantic mapping thread. We performed an experimental evaluation on the TUM dataset, the Bonn dataset, and the OpenLORIS-Scene dataset. The results show that SG-SLAM is not only one of the most real-time, accurate, and robust systems in dynamic scenes but also allows the creation of intuitive semantic metric maps.

GMP-SLAM: A real-time RGB-D SLAM in Dynamic Environments using GPU Dynamic Points Detection Method

Simultaneous Localization and Mapping (SLAM) is a fundamental technology for robotics. Vision-based SLAM has been developed for many years, but it is still difficult for SLAM system to handle dynamic environments. In this paper, we present GMP-SLAM, a realtime RGB-D SLAM system for highly dynamic environments with the help of GPU Grid Map Projection— a GPU dynamic points detection method we design. SLAM is a time-sensitive system for robotics, and it is hard to reach real time, especially in dynamic environments because it is necessary to track moving objects and it takes a lot of time. GMP-SLAM is based on ORB-SLAM2, which is one of the best feature-based SLAM frameworks and can reach real time just in CPU. But ORB-SLAM2 cannot handle highly dynamic environments very well, and most work focus on tracking moving objects with a neural network, which cannot reach real time even with the help of GPU. To solve real-time problem, we propose an all-in-parallel dynamic points detection framework for visual simultaneous localization and mapping (VSLAM) in dynamic environments based on 3D occupancy grid maps. Our SLAM system can provide not only higher trajectory accuracy but also a 3D grid map for navigation. We test our SLAM system in our real-world datasets we record and get higher trajectory accuracy than ORB-SLAM2. At the same time, our system can run nearly in 20Hz, which is much better than existing VSLAM framework in dynamic environments.

YPD-SLAM: A Real-Time VSLAM System for Handling Dynamic Indoor Environments

Aiming at the problem that Simultaneous localization and mapping (SLAM) is greatly disturbed by many dynamic elements in the actual environment, this paper proposes a real-time Visual SLAM (VSLAM) algorithm to deal with a dynamic indoor environment. Firstly, a lightweight YoloFastestV2 deep learning model combined with NCNN and Mobile Neural Network (MNN) inference frameworks is used to obtain preliminary semantic information of images. The dynamic feature points are removed according to epipolar constraint and dynamic properties of objects between consecutive frames. Since reducing the number of feature points after rejection affects the pose estimation, this paper innovatively combines Cylinder and Plane Extraction (CAPE) planar detection. We generate planes from depth maps and then introduce planar and in-plane point constraints into the nonlinear optimization of SLAM. Finally, the algorithm is tested on the publicly available TUM (RGB-D) dataset, and the average improvement in localization accuracy over ORB-SLAM2, DS-SLAM, and RDMO-SLAM is about 91.95%, 27.21%, and 30.30% under dynamic sequences, respectively. The single-frame tracking time of the whole system is only 42.68 ms, which is 44.1%, being 14.6-34.33% higher than DS-SLAM, RDMO-SLAM, and RDS-SLAM respectively. The system that we proposed significantly increases processing speed, performs better in real-time, and is easily deployed on various platforms.

OG-SLAM: A real-time and high-accurate monocular visual SLAM framework

The challenge of improving the accuracy of monocular Simultaneous Localization and Mapping (SLAM) is considered, which widely appears in computer vision, autonomous robotics, and remote sensing. A new framework (ORB-GMS-SLAM (or OG-SLAM)) is proposed, which introduces the region-based motion smoothness into a typical Visual SLAM (V-SLAM) system. The region-based motion smoothness is implemented by integrating the Oriented Fast and Rotated Brief (ORB) features and the Grid-based Motion Statistics (GMS) algorithm into the feature matching process. The OG-SLAM significantly reduces the absolute trajectory error (ATE) on the key-frame trajectory estimation without compromising the real-time performance. This study compares the proposed OG-SLAM to an advanced V-SLAM system (ORB-SLAM2). The results indicate the highest accuracy improvement of almost 75% on a typical RGB-D SLAM benchmark. Compared with other ORB-SLAM2 settings (1800 key points), the OG-SLAM improves the accuracy by around 20% without losing performance in real-time. The OG-SLAM framework has a significant advantage over the ORB-SLAM2 system in that it is more robust for rotation, loop-free, and long ground-truth length scenarios. Furthermore, as far as the authors are aware, this framework is the first attempt to integrate the GMS algorithm into the V-SLAM.