Real-time SLAM Research Articles

EGLT-SLAM: Real-Time Visual-Inertial SLAM Based on Entropy-Guided Line Tracking

EGLT-SLAM: Real-Time Visual-Inertial SLAM Based on Entropy-Guided Line Tracking

Strong-SLAM: real-time RGB-D visual SLAM in dynamic environments based on StrongSORT

Abstract The assumptions of a static environment and scene rigidity are important theoretical underpinnings of traditional visual simultaneous localization and mapping (SLAM) algorithms. However, these assumptions are difficult to work in dynamic environments containing non-rigid objects, and cannot effectively handle the characteristics of local areas of non-rigid moving objects, seriously affecting the robustness and accuracy of the SLAM system in localization and mapping. To address these problems, we improved ORB-SLAM3 and proposed a real-time RGB-D visual SLAM framework for dynamic environments based on StrongSORT—Strong-SLAM. First, we combine YOLOv7-tiny with StrongSORT to match the semantic information of dynamic targets. Optical flow and epipolar constraints are then used to initially extract geometric and motion information between adjacent frames. Subsequently, based on an improved adaptive threshold segmentation algorithm and geometric residuals, a background model and a Gaussian residual model are constructed to further extract the geometric information of dynamic targets. Finally, semantic and geometric information are integrated to perform global feature motion level classification, and motion probabilities and optimization weights are defined to participate in global pose estimation and optimization. Experimental results on the publicly available TUM RGB-D dataset show that Strong-SLAM reduces the absolute trajectory error and relative pose error by at least 90% compared to ORB-SLAM3, achieving performance comparable to the most advanced dynamic SLAM solutions.

TrinitySLAM: On-board Real-time Event-image Fusion SLAM System for Drones

TrinitySLAM: On-board Real-time Event-image Fusion SLAM System for Drones

A real-time visual SLAM based on semantic information and geometric information in dynamic environment

A real-time visual SLAM based on semantic information and geometric information in dynamic environment

Parallel implementation for real-time visual SLAM systems based on heterogeneous computing

Simultaneous localization and mapping has become rapidly developed and plays an indispensable role in intelligent vehicles. However, many state-of-the-art visual simultaneous localization and mapping (VSLAM) frameworks are very time-consuming both in front-end and back-end, especially for large-scale scenes. Nowadays, the increasingly popular use of graphics processors for general-purpose computing, and the progressively mature high-performance programming theory based on compute unified device architecture (CUDA) have given the possibility for large-scale VSLAM to solve the conflict between limited computing power and excessive computing tasks. The paper proposes a full-flow optimal parallelization scheme based on heterogeneous computing to speed up the time-consuming modules in VSLAM. Firstly, a parallel strategy for feature extraction and matching is designed to reduce the time consumption arising from multiple data transfers between devices. Secondly, a bundle adjustment method based solely on CUDA is developed. By fully optimizing memory scheduling and task allocation, a large increase in speed is achieved while maintaining accuracy. Besides, CUDA heterogeneous acceleration is fully utilized for tasks such as error computation and linear system construction in the VSLAM back-end to enhance the operation speed. Our proposed method is tested on numerous public datasets on both computer and embedded sides, respectively. A number of qualitative and quantitative experiments are performed to verify its superiority in terms of speed compared to other states-of-the-art.

Real-Time Dense Visual SLAM with Neural Factor Representation

Developing a high-quality, real-time, dense visual SLAM system poses a significant challenge in the field of computer vision. NeRF introduces neural implicit representation, marking a notable advancement in visual SLAM research. However, existing neural implicit SLAM methods suffer from long runtimes and face challenges when modeling complex structures in scenes. In this paper, we propose a neural implicit dense visual SLAM method that enables high-quality real-time reconstruction even on a desktop PC. Firstly, we propose a novel neural scene representation, encoding the geometry and appearance information of the scene as a combination of the basis and coefficient factors. This representation allows for efficient memory usage and the accurate modeling of high-frequency detail regions. Secondly, we introduce feature integration rendering to significantly improve rendering speed while maintaining the quality of color rendering. Extensive experiments on synthetic and real-world datasets demonstrate that our method achieves an average improvement of more than 60% for Depth L1 and ATE RMSE compared to existing state-of-the-art methods when running at 9.8 Hz on a desktop PC with a 3.20 GHz Intel Core i9-12900K CPU and a single NVIDIA RTX 3090 GPU. This remarkable advancement highlights the crucial importance of our approach in the field of dense visual SLAM.

CMDS-SLAM: real-time efficient centralized multi-robot dense surfel SLAM

Abstract Real-time dense mapping technology for multi-robot systems is crucial in scenarios like search and rescue. This paper presents CMDS-SLAM, a centralized multi-robot dense surfel SLAM system aimed at overcoming limitations in hardware constraints, data transmission, and real-time creation and updating of dense maps in multi-robot SLAM. CMDS-SLAM reduces the transmission of dense information by employing a dense information filtering mechanism based on co-visual keyframes, in conjunction with the extraction and compression of superpixels. Additionally, the method employs a three-stage superpixel segmentation approach to optimize transmission and enhance the efficiency of surfel map generation. Finally, a surfel co-visibility graph is established, and multi-robot surfel map maintenance and updates are achieved through co-visibility graph and map optimization. A comprehensive evaluation of CMDS-SLAM indicates that the method enables multi-robot surfel mapping and significantly alleviates data transmission pressures while achieving real-time updates and maintenance of the surfel map.

X-SLAM: Scalable Dense SLAM for Task-aware Optimization using CSFD

We present X-SLAM, a real-time dense differentiable SLAM system that leverages the complex-step finite difference (CSFD) method for efficient calculation of numerical derivatives, bypassing the need for a large-scale computational graph. The key to our approach is treating the SLAM process as a differentiable function, enabling the calculation of the derivatives of important SLAM parameters through Taylor series expansion within the complex domain. Our system allows for the real-time calculation of not just the gradient, but also higher-order differentiation. This facilitates the use of high-order optimizers to achieve better accuracy and faster convergence. Building on X-SLAM, we implemented end-to-end optimization frameworks for two important tasks: camera relocalization in wide outdoor scenes and active robotic scanning in complex indoor environments. Comprehensive evaluations on public benchmarks and intricate real scenes underscore the improvements in the accuracy of camera relocalization and the efficiency of robotic navigation achieved through our task-aware optimization. The code and data are available at https://gapszju.github.io/X-SLAM.

Real-Time SLAM and Faster Object Detection on a Wheeled Lifting Robot with Mobile-ROS Interaction

Wheeled lifting robots have found widespread applications in various industrial and logistical environments. However, traditional robots are far from adequate in terms of visual perception capabilities. Additionally, their remote control methods suffer from inefficiencies, which tend to bring safety concerns. To address these issues, this work proposes an autonomous multi-sensor-enabled wheeled lifting robot system, i.e., AMSeWL-R, to facilitate remote autonomous operations. Specifically, AMSeWL-R integrates real-time simultaneous localization and mapping with object detection on a wheeled lifting robot. Additionally, a novel mobile-ROS interaction method is proposed to achieve real-time communication and control between a mobile device and a ROS host. Furthermore, a lightweight object detection algorithm based on YOLOv8, i.e., YOLOv8-R, is proposed to achieve faster detection. Experimental results validate the effectiveness of the AMSeWL-R system for accurately detecting objects and mapping its surroundings. Furthermore, TensorRT acceleration is employed during practical testing on a Jetson Nano to achieve real-time detection using the proposed YOLOv8-R, demonstrating its efficacy in real-world scenarios.

Real-time deformable SLAM with geometrically adapted template for dynamic monocular laparoscopic scenes.

Intraoperative reconstruction of endoscopic scenes is a key technology for surgical navigation systems. The accuracy and efficiency of 3D reconstruction directly determine the effectiveness of navigation systems in a variety of clinical applications. While current deformable SLAM algorithms can meet real-time requirements, their underlying reliance on regular templates still makes it challenging to efficiently capture abrupt geometric features within scenes, such as organ contours and surgical margins. We propose a novel real-time monocular deformable SLAM algorithm with geometrically adapted template. To ensure real-time performance, the proposed algorithm consists of two threads: a deformation mapping thread updates the template at keyframe rate and a deformation tracking thread estimates the camera pose and the deformation at frame rate. To capture geometric features more efficiently, the algorithm first detects salient edge features using a pre-trained contour detection network and then constructs the template through a triangulation method with guidance of the salient features. We thoroughly evaluated this method on Mandala and Hamlyn datasets in terms of accuracy and performance. The results demonstrated that the proposed method achieves better accuracy with 0.75-7.95% improvement and achieves consistent effectiveness in data association compared with the closest method. This study verified an adaptive template does improve the performance of reconstruction of dynamic laparoscopic Scenes with abrupt geometric features. However, further exploration is needed for applications in laparoscopic surgery with incisal margins caused by surgical instruments. This research serves as a crucial step toward enhanced automatic computer-assisted navigation in laparoscopic surgery. Code is available at https://github.com/Tang257/SLAM-with-geometrically-adapted-template .

VIS-SLAM: A Real-Time Dynamic SLAM Algorithm Based on the Fusion of Visual, Inertial, and Semantic Information

A deep learning-based Visual Inertial SLAM technique is proposed in this paper to ensure accurate autonomous localization of mobile robots in environments with dynamic objects. Addressing the limitations of real-time performance in deep learning algorithms and the poor robustness of pure visual geometry algorithms, this paper presents a deep learning-based Visual Inertial SLAM technique. Firstly, a non-blocking model is designed to extract semantic information from images. Then, a motion probability hierarchy model is proposed to obtain prior motion probabilities of feature points. For image frames without semantic information, a motion probability propagation model is designed to determine the prior motion probabilities of feature points. Furthermore, considering that the output of inertial measurements is unaffected by dynamic objects, this paper integrates inertial measurement information to improve the estimation accuracy of feature point motion probabilities. An adaptive threshold-based motion probability estimation method is proposed, and finally, the positioning accuracy is enhanced by eliminating feature points with excessively high motion probabilities. Experimental results demonstrate that the proposed algorithm achieves accurate localization in dynamic environments while maintaining real-time performance.

Real-Time Dynamic SLAM Using Moving Probability Based on IMU and Segmentation

Real-Time Dynamic SLAM Using Moving Probability Based on IMU and Segmentation

IPR-VINS: Real-time monocular visual-inertial SLAM with implicit plane optimization

SLAM typically involves updating numerous landmarks to reconstruct scenes rather than generating a topological depiction of the actual environment. Planar features, prevalent in structured environments, can contribute to the development of more nuanced yet less cumbersome maps. The incorporation of fusions between feature points and planar facets gives rise to new discrepancies that must be meticulously addressed. To prevent the uncontrolled expansion of planar boundaries, a robust technique for boundary extraction is offered. Scene planes are fused using the boundary function, ensuring stable feature extraction and planar association. To leverage limited computational resources effectively, a streamlined back-end framework is constructed. Finally, factors are fine-tuned at a consistent scale, which not only elevates locational accuracy but also diminishes computational complexity. To reduce systematic errors, plane prior information is assimilated into global optimization. Compared with VINS-Mono, the average Absolute Trajectory Error of the proposed method on EuRoC is reduced by 44%.

Real-time Road Surface LiDAR SLAM based on Dead-Reckoning Assisted Registration

Real-time Road Surface LiDAR SLAM based on Dead-Reckoning Assisted Registration

RVD-SLAM: A Real-Time Visual SLAM Toward Dynamic Environments Based on Sparsely Semantic Segmentation and Outlier Prior

RVD-SLAM: A Real-Time Visual SLAM Toward Dynamic Environments Based on Sparsely Semantic Segmentation and Outlier Prior

DYS-SLAM: A real-time RGBD SLAM combined with optical flow and semantic information in a dynamic environment1

Traditional Simultaneous Localization and Mapping application in dynamic situations is constrained by static assumptions. However, the majority of well-known dynamic SLAM systems use deep learning to identify dynamic objects, which creates the issue of trade-offs between accuracy and real-time. To tackle this issue, this work suggests a unique dynamic semantics method(DYS-SLAM) for semantic simultaneous localization and mapping that strikes a trade-off between high accuracy and high real-time performance. We propose M-LK, an enhanced Lucas-Kanade(LK) optical flow method. This technique keeps the continuous motion and greyscale consistency assumptions from the original method while switching out the spatial consistency assumption for a motion consistency assumption, reducing sensitivity to image gradients to identify dynamic feature points and regions efficiently. In order to increase segmentation accuracy while maintaining real-time performance, we develop a segmentation refinement scheme that projects 3D point cloud segmentation results into 2D object detection zones. A dense semantic octree graph is built in the interim to expedite the high-level process. Compared to the four equivalent dynamic SLAM approaches, experiments on the publicly available TUM RGB-D dataset demonstrate that the DYS-SLAM method offers competitive localization accuracy and satisfactory real-time performance in both high and low-dynamic scenarios.

FPGA-Based Feature Extraction and Tracking Accelerator for Real-Time Visual SLAM.

Due to its advantages of low latency, low power consumption, and high flexibility, FPGA-based acceleration technology has been more and more widely studied and applied in the field of computer vision in recent years. An FPGA-based feature extraction and tracking accelerator for real-time visual odometry (VO) and visual simultaneous localization and mapping (V-SLAM) is proposed, which can realize the complete acceleration processing capability of the image front-end. For the first time, we implement a hardware solution that combines features from accelerated segment test (FAST) feature points with Gunnar Farneback (GF) dense optical flow to achieve better feature tracking performance and provide more flexible technical route selection. In order to solve the scale invariance and rotation invariance lacking problems of FAST features, an efficient pyramid module with a five-layer thumbnail structure was designed and implemented. The accelerator was implemented on a modern Xilinx Zynq FPGA. The evaluation results showed that the accelerator could achieve stable tracking of features of violently shaking images and were consistent with the results from MATLAB code running on PCs. Compared to PC CPUs, which require seconds of processing time, the processing latency was greatly reduced to the order of milliseconds, making GF dense optical flow an efficient and practical technical solution on the edge side.

Stereo SLAM in Dynamic Environments Using Semantic Segmentation

As we all know, many dynamic objects appear almost continuously in the real world that are immensely capable of impairing the performance of the majority of vision-based SLAM systems based on the static-world assumption. In order to improve the robustness and accuracy of visual SLAM in high-dynamic environments, a real-time and robust stereo SLAM system for dynamic scenes was proposed. To weaken the influence of dynamic content, the moving-object detection method was put forward in our visual odometry, and then the semantic segmentation network was combined in our stereo SLAM to extract pixel-level contours of dynamic objects. Then, the influences of dynamic objects were significantly weakened and the performance of our system increased markedly in dynamic, complex, and crowed city spaces. Following experiments with both the KITTI Odometry dataset and in a real-life scene, the results showed that our method could dramatically decrease the tracking error or drift, and improve the robustness and stability of our stereo SLAM in high dynamic outdoor scenarios.

A Real-Time Monocular Visual SLAM Based on the Bundle Adjustment with Adaptive Robust Kernel

A Real-Time Monocular Visual SLAM Based on the Bundle Adjustment with Adaptive Robust Kernel

Reconfigurable System-on-Chip Architectures for Robust Visual SLAM on Humanoid Robots

Visual Simultaneous Localization and Mapping (vSLAM) is the method of employing an optical sensor to map the robot’s observable surroundings while also identifying the robot’s pose in relation to that map. The accuracy and speed of vSLAM calculations can have a very significant impact on the performance and effectiveness of subsequent tasks that need to be executed by the robot, making it a key building component for current robotic designs. The application of vSLAM in the area of humanoid robotics is particularly difficult due to the robot’s unsteady locomotion. This paper introduces a pose graph optimization module based on RGB (ORB) features, as an extension of the KinectFusion pipeline (a well-known vSLAM algorithm), to assist in recovering the robot’s stance during unstable gait patterns when the KinectFusion tracking system fails. We develop and test a wide range of embedded MPSoC FPGA designs, and we investigate numerous architectural improvements, both precise and approximation, to study their impact on performance and accuracy. Extensive design space exploration reveals that properly designed approximations, which exploit domain knowledge and efficient management of CPU and FPGA fabric resources, enable real-time vSLAM at more than 30 fps in humanoid robots with high energy-efficiency and without compromising robot tracking and map construction. This is the first FPGA design to achieve robust, real-time dense SLAM operation targeting specifically humanoid robots. An open source release of our implementations and data can be found in [ 1 ].