Visual SLAM Research Articles

Multi-Sensor SLAM Positioning and Navigation

This paper provides a comprehensive review of advancements in Positioning and Navigation, focusing on the integration of key technologies to enhance operational efficiency and sustainability. The study examines the application of sensors in robotic systems, with particular emphasis on visual Simultaneous Localization and Mapping (SLAM) and LiDAR SLAM technologies. Visual SLAM, which uses monocular or binocular cameras to capture environmental images for navigation and mapping, offers cost-effective solutions but faces challenges such as sensitivity to lighting conditions and high computational demands. LiDAR SLAM, utilizing laser pulses to create precise 3D maps, complements visual SLAM by improving accuracy and robustness. The review also covers the integration of these technologies with deep learning and other advanced algorithms, highlighting their potential to enhance robotic performance and address practical limitations. The paper identifies future research directions aimed at advancing these technologies for more effective and sustainable practices.

Advanced obstacle recognition and trajectory prediction for autonomous vehicles using 3D CNN and GRU

Abstract. Recent years have seen a rapid advancement in new energy vehicle technology, which has increased consumer demand for autonomous driving systems. However, current self-driving cars still face many technical barriers. Obstacle detection and obstacle trajectory prediction in dynamic environments is one of the main difficulties of the technique. To improve self-driving cars' capacity for route planning and decision-making in dynamic contexts, this thesis proposes an obstacle identification and trajectory prediction model that combines a gated recurrent unit (GRU) network with a three-dimensional convolutional neural network (3D CNN). The integrated framework utilises the feature recognition capability of 3D CNN and the prediction capability of GRU to accurately predict dynamic obstacles, thereby improving the performance of the model in various types of environments (e.g., visual SLAM). Experimental results based on publicly available datasets (e.g., KITTI and Argoverse) show that the proposed method has significant advantages in dynamic obstacle identification and prediction as well as overall model operation efficiency. The findings show that the integrated framework is useful and successful in advancing the creation of self-driving automobiles.

Object detection and tracking for drones: A system design using dynamic visual SLAM

Abstract. Drones can play a quite crucial role in many walks of life today. Enhancing the visual perception ability of drones is crucial to their intelligence level. Among them, it is necessary to focus on strengthening the detection, tracking and mapping capabilities of drones for dynamic objects. However, the existing visual SLAM systems carried by drones do not perform well in dynamic environments. This project designs a monocular visual SLAM system specifically for drones, aiming to achieve efficient three-dimensional mapping and target tracking, surpassing the limitations of simple static mapping and positioning. Besides, this project constructs a drone dynamic SLAM system developed on the ORB-SLAM3 structure, uses drone images to detect, track and map object motion models, and reconstructs environmental maps to obtain motion parameters with real physical scales. This project strives to optimize the input pre-processing module, improve the validity of data and output environmental maps and raster maps. The outcomes demonstrate the system's strong accuracy and adaptability in dynamic installation procedures.

Intelligent assistive obstacle avoidance device based on SLAM and wearable technology

Abstract. This paper presents a conceptual design for an intelligent assistive device that combines visual simultaneous localisation and mapping (SLAM) with wearable technology. The device has been developed through the integration of two innovative fields: visual SLAM and wearable technology. The objective is to develop a device that is both user-friendly and safe. The design addresses the issue of safety for visually impaired individuals when they are outside the home. The objective is to provide a dependable, real-time, and resilient solution that can be utilised in intricate indoor and outdoor settings. The system is designed to provide reliable, real-time, and highly effective solutions in variable environments. The main components of the proposed system include visual SLAM, intelligent wearable devices (as carriers), and a comfortable and straightforward user feedback system (through haptic, auditory, or visual signals to provide feedback to the wearer), while simultaneously considering the safety and comfort of the device. This is due to the consideration of the prolonged and frequent periods of use by visually impaired individuals. This paper considers the latest advances in SLAM algorithms, improvements in wearable sensors and the latest developments in robotics for assisting visually impaired people. It also discusses the potential of these technologies in the future development of assistive devices. The aim is to provide a feasible, comfortable and safe solution that will enhance the safety and autonomy of visually impaired people when they are outside.

Advances and challenges in UAV navigation based on visual SLAM

Abstract. In recent years, unmanned aerial vehicles (UAVs) have seen extensive use in fields such as agriculture, search and rescue, commercial, and military operations, driving the demand for autonomous navigation capabilities. Though GPS is the traditional method used for navigation, it becomes problematic in harsh environments like deserts. The Visual Simultaneous Localization and Mapping technology offers a solution to enhance UAV navigation in complex environments by constructing maps and localizing the UAV simultaneously in real time. This paper presents state-of-the-art visual SLAM technology developed for UAV navigation regarding algorithms like Oriented FAST and Rotated BRIEF SLAM (ORB-SLAM) and Large-Scale Direct Monocular SLAM (LSD-SLAM), whereby their performance is also discussed, with its positives and negatives. In this regard, the latest progress and challenges in applications are reviewed and analyzed through relevant literature from the databases of PubMed, IEEE, and Google Scholar in the past five years. The novelty of this paper lies in the comprehensive evaluation of the application performance of different visual SLAM algorithms in UAV navigation and the proposal of future research directions.

Analysis of modeling nasal narrow space based on visual slam

Abstract. Now, surgeries are becoming more stable, safe, efficient and low-cost. During the surgical treatment of nasal diseases, surgical robotic robotics can help operate accurately and reduce the discomfort after surgery. However, due to the internal space of the nasal cavity being relatively narrow, it is difficult for the nasal surgical robot to contain multiple vision sensors and the monocular camera could not get information about the depth of the 3D objects in the scene, so the existing surgical robots cannot accomplish the three-dimensional modeling about the internal space of nasal cavity well. In practice, doctors still have to analyze pictures from the robotic, which may decrease the efficiency of the surgery and increase the risk to patients. This article designed a SLAM algorithm framework based on a depth estimation network, it can simulate the internal structure of the nasal cavity more accurately through pictures, which come from monocular endoscopic on surgical robotic. The insights gained in this study verify that the method of image segmentation can also make the depth representation of the nasal internal space more accurate and this method may help robots realize their self-position in the narrow area of the nasal cavity, which lays the foundations for the development of fully autonomous surgical robots.

D3L-SLAM: A Comprehensive Hybrid Simultaneous Location and Mapping System with Deep Keypoint, Deep Depth, Deep Pose, and Line Detection

Robust localization and mapping are crucial for autonomous systems, but traditional handcrafted feature-based visual SLAM often struggles in challenging, textureless environments. Additionally, monocular SLAM lacks scale-aware depth perception, making accurate scene scale estimation difficult. To address these issues, we propose D3L-SLAM, a novel monocular SLAM system that integrates deep keypoints, deep depth estimates, deep pose priors, and a line detector. By leveraging deep keypoints, which are more resilient to lighting variations, our system improves the robustness of visual SLAM. We further enhance perception in low-texture areas by incorporating line features in the front-end and mitigate scale degradation with learned depth estimates. Additionally, point-line feature constraints optimize pose estimation and mapping through a tightly coupled point-line bundle adjustment (BA). The learned pose estimates refine the feature matching process during tracking, leading to more accurate localization and mapping. Experimental results on public and self-collected datasets show that D3L-SLAM significantly outperforms both traditional and learning-based visual SLAM methods in localization accuracy.

Robust and accurate visual geo-localization using prior map constructed by handheld LiDAR SLAM with camera image and terrestrial LiDAR point cloud

ABSTRACT Visual geo-localization using prior map of known environments has extensive applications in the fields such as self-driving, augmented reality and navigation. Currently, such prior maps are usually constructed by visual SLAM or SFM. However, the recent advances in LiDAR SLAM technology with RGB camera demonstrate great potential in efficient and accurate prior map construction and visual geo-localization. In this research, we developed a novel prior map construction approach which seamlessly integrates a handheld LiDAR SLAM system, a global LiDAR localization algorithm and a sparse set of terrestrial LiDAR scans as accurate control within a factor graph optimization framework. The developed LiDAR SLAM system augmented by terrestrial LiDAR scans achieves an improved mapping accuracy of 8 cm under GNSS-denied conditions in the areas over 50,000 m2 while the state-of-art FAST-LIO2 alone fails and produces a mapping error over 30 m. The results show that a competitive accuracy of visual geo-localization on a mobile phone using the constructed prior map is achieved (i.e. 60 and 76 cm in two tests respectively). Finally, this work also demonstrates a novel LiDAR point cloud fusion method that produces optimized and consistent coarse-to-fine 3D reconstruction in large and complex scenes.

Visual SLAM system based on dynamic semantic features

Aiming at the problem that dynamic objects (such as pedestrians, vehicles, and animals) in real scenes affect the positioning and mapping accuracy of visual SLAM (Simultaneous Localization and Mapping), a YOLOv3-ORB-SLAM3 algorithm is proposed based on ORB-SLAM3 (Oriented FAST and Rotated BRIEF-Simultaneous Localization and Mapping 3). The algorithm adds a semantic thread on the basis of ORB-SLAM3, and adopts a dual-thread mechanism for dynamic and static scene feature extraction: the semantic thread uses YOLOv3 to perform semantic recognition target detection on dynamic objects in the scene, and removes outliers from the extracted dynamic area feature points; the tracking thread extracts scene area features through ORB features, and obtains static scene features combined with semantic information and sends them to the backend, thereby eliminating the interference of dynamic scenes on the system and improving the positioning accuracy of the visual SLAM algorithm. Verified using the TUM (Technical University of Munich) dataset, the results show that the ATE (Average Treatment Effect) index of the YOLOv3-ORB-SLAM3 algorithm in the monocular mode is about 30% lower than that of the ORB-SLAM3 algorithm, and the ATE index of the dynamic sequence in the RGB-D (Red, Green and Blue-Depth) mode is reduced 10%, while the static sequence has no significant decrease.

A Novel Fuzzy Image-Based UAV Landing Using RGBD Data and Visual SLAM

In this work, an innovative perception-guided approach is proposed for landing zone detection and realization of Unmanned Aerial Vehicles (UAVs) operating in unstructured environments ridden with obstacles. To accommodate secure landing, two well-established tools, namely fuzzy systems and visual Simultaneous Localization and Mapping (vSLAM), are implemented into the landing pipeline. Firstly, colored images and point clouds acquired by a visual sensory device are processed to serve as characterizing maps that acquire information about flatness, steepness, inclination, and depth variation. By leveraging these images, a novel fuzzy map infers the areas for risk-free landing on which the UAV can safely land. Subsequently, the vSLAM system is employed to estimate the platform’s pose and an additional set of point clouds. The vSLAM point clouds presented in the corresponding keyframe are projected back onto the image plane on which a threshold fuzzy landing score map is applied. In other words, this binary image serves as a mask for the re-projected vSLAM world points to identify the best subset for landing. Once these image points are identified, their corresponding world points are located, and among them, the center of the cluster with the largest area is chosen as the point to land. Depending on the UAV’s size, four synthesis points are added to the vSLAM point cloud to execute the image-based visual servoing landing using image moment features. The effectiveness of the landing package is assessed through the ROS Gazebo simulation environment, where comparisons are made with a state-of-the-art landing site detection method.

A Survey of Computer Vision Detection, Visual SLAM Algorithms, and Their Applications in Energy-Efficient Autonomous Systems

Within the area of environmental perception, automatic navigation, object detection, and computer vision are crucial and demanding fields with many applications in modern industries, such as multi-target long-term visual tracking in automated production, defect detection, and driverless robotic vehicles. The performance of computer vision has greatly improved recently thanks to developments in deep learning algorithms and hardware computing capabilities, which have spawned the creation of a large number of related applications. At the same time, with the rapid increase in autonomous systems in the market, energy consumption has become an increasingly critical issue in computer vision and SLAM (Simultaneous Localization and Mapping) algorithms. This paper presents the results of a detailed review of over 100 papers published over the course of two decades (1999–2024), with a primary focus on the technical advancement in computer vision. To elucidate the foundational principles, an examination of typical visual algorithms based on traditional correlation filtering was initially conducted. Subsequently, a comprehensive overview of the state-of-the-art advancements in deep learning-based computer vision techniques was compiled. Furthermore, a comparative analysis of conventional and novel algorithms was undertaken to discuss the future trends and directions of computer vision. Lastly, the feasibility of employing visual SLAM algorithms in the context of autonomous vehicles was explored. Additionally, in the context of intelligent robots for low-carbon, unmanned factories, we discussed model optimization techniques such as pruning and quantization, highlighting their importance in enhancing energy efficiency. We conducted a comprehensive comparison of the performance and energy consumption of various computer vision algorithms, with a detailed exploration of how to balance these factors and a discussion of potential future development trends.

Enhancing Real-Time Visual SLAM with Distant Landmarks in Large-Scale Environments

The efficacy of visual Simultaneous Localization and Mapping (SLAM) diminishes in large-scale environments due to challenges in identifying distant landmarks, leading to a limited perception range and trajectory drift. This paper presents a practical method to enhance the accuracy of feature-based real-time visual SLAM for compact unmanned vehicles by constructing distant map points. By tracking consecutive image features across multiple frames, remote map points are generated with sufficient parallax angles, extending the mapping scope to the theoretical maximum range. Observations of these landmarks from preceding keyframes are supplemented accordingly, improving back-end optimization and, consequently, localization accuracy. The effectiveness of this approach is ensured by the introduction of the virtual map point, a proposed data structure that links relational features to an imaginary map point, thereby maintaining the constrained size of local optimization during triangulation. Based on the ORB-SLAM3 code, a SLAM system incorporating the proposed method is implemented and tested. Experimental results on drone and vehicle datasets demonstrate that the proposed method outperforms ORB-SLAM3 in both accuracy and perception range with negligible additional processing time, thus preserving real-time performance. Field tests using a UGV further validate the efficacy of the proposed method.

Lightweight image denoising method for feature-based visual SLAM in γ radiation environments

In the γ radiation environments, high-energy photons impact image sensor pixels and induce speckle noise in the scene image, which influences the accuracy of feature detection and matching in the visual Simultaneous Localization and Mapping (SLAM) algorithms. To mitigate these effects, this paper presents a lightweight image denoising method based on super-neighborhood down-sampling. Firstly, the super-neighborhood down-sampling strategy is used to split the speckle noise in the camera images into isolated point noises in the sub-image sets. Secondly, these isolated flickering point noises are removed by the detection-based median filter. Finally, the denoised scene image is reconstructed by the super-neighborhood up-sampling. Extensive experiments were performed on the radiation environment datasets that were captured by the camera inside the Cobalt-60 hot cell. The results show that the proposed method can effectively filter out the speckle radiation noise induced by γ rays, significantly improve the quality of features and feature matching rate, and improve the performance of ORB-SLAM3, the state-of-the-art feature-based SLAM algorithm, in radiation environments.

4Seasons: Benchmarking Visual SLAM and Long-Term Localization for Autonomous Driving in Challenging Conditions

AbstractIn this paper, we present a novel visual SLAM and long-term localization benchmark for autonomous driving in challenging conditions based on the large-scale 4Seasons dataset. The proposed benchmark provides drastic appearance variations caused by seasonal changes and diverse weather and illumination conditions. While significant progress has been made in advancing visual SLAM on small-scale datasets with similar conditions, there is still a lack of unified benchmarks representative of real-world scenarios for autonomous driving. We introduce a new unified benchmark for jointly evaluating visual odometry, global place recognition, and map-based visual localization performance which is crucial to successfully enable autonomous driving in any condition. The data has been collected for more than one year, resulting in more than 300 km of recordings in nine different environments ranging from a multi-level parking garage to urban (including tunnels) to countryside and highway. We provide globally consistent reference poses with up to centimeter-level accuracy obtained from the fusion of direct stereo-inertial odometry with RTK GNSS. We evaluate the performance of several state-of-the-art visual odometry and visual localization baseline approaches on the benchmark and analyze their properties. The experimental results provide new insights into current approaches and show promising potential for future research. Our benchmark and evaluation protocols will be available at https://go.vision.in.tum.de/4seasons.

Balancing Efficiency and Accuracy: Enhanced Visual Simultaneous Localization and Mapping Incorporating Principal Direction Features

In visual Simultaneous Localization and Mapping (SLAM), operational efficiency and localization accuracy are equally crucial evaluation metrics. We propose an enhanced visual SLAM method to ensure stable localization accuracy while improving system efficiency. It can maintain localization accuracy even after reducing the number of feature pyramid levels by 50%. Firstly, we innovatively incorporate the principal direction error, which represents the global geometric features of feature points, into the error function for pose estimation, utilizing Pareto optimal solutions to improve the localization accuracy. Secondly, for loop-closure detection, we construct a feature matrix by integrating the grayscale and gradient direction of an image. This matrix is then dimensionally reduced through aggregation, and a multi-layer detection approach is employed to ensure both efficiency and accuracy. Finally, we optimize the feature extraction levels and integrate our method into the visual system to speed up the extraction process and mitigate the impact of the reduced levels. We comprehensively evaluate the proposed method on local and public datasets. Experiments show that the SLAM method maintained high localization accuracy after reducing the tracking time by 24% compared with ORB SLAM3. Additionally, the proposed loop-closure-detection method demonstrated superior computational efficiency and detection accuracy compared to the existing methods.

A semantic visual SLAM based on improved mask R-CNN in dynamic environment

A semantic visual SLAM based on improved mask R-CNN in dynamic environment

Evaluation of visual SLAM algorithms in unstructured planetary-like and agricultural environments

Given the significant advance in visual SLAM (VSLAM), it might be assumed that the location and mapping problem has been solved. Nevertheless, VSLAM algorithms may exhibit poor performance in unstructured environments. This paper addresses the problem of VSLAM in unstructured planetary-like and agricultural environments. A performance study of state-of-the-art algorithms in these environments was conducted to evaluate their robustness. Quantitative and qualitative results of the study are reported, which exposes that the impressive performance of most state-of-the-art VSLAM algorithms is not generally reflected in unstructured planetary-like and agricultural environments. Statistical scene analysis was performed on datasets from well-known structured environments as well as planetary-like and agricultural datasets to identify visual differences between structured and unstructured environments, which cause VSLAM algorithms to fail. In addition, strategies to overcome the VSLAM algorithm limitations in unstructured planetary-like and agricultural environments are suggested to guide future research on VSLAM in these environments.

DZ-SLAM: A SAM-based SLAM algorithm oriented to dynamic environments

Precise localization is a fundamental prerequisite for the effective operation of Simultaneous Localization and Mapping (SLAM) systems. Traditional visual SLAM is based on static environments and therefore performs poorly in dynamic environments. While numerous visual SLAM methods have been proposed to address dynamic environments, these approaches are typically based on certain prior knowledge. This paper introduces DZ-SLAM, a dynamic SLAM algorithm that does not require any prior knowledge, based on ORB-SLAM3, to handle unknown dynamic elements in the scene. This work first introduces the FastSAM to enable comprehensive image segmentation. It then proposes an adaptive threshold-based dense optical flow approach to identify dynamic elements within the environment. Finally, combining FastSAM with optical flow method and embedding it into the SLAM framework to eliminate dynamic objects and improve positioning accuracy in dynamic environments. The experiment shows that compared with the original ORB-SLAM3 algorithm, the algorithm proposed in this paper reduces the absolute trajectory error by up to 96%; Compared to the most advanced algorithms currently available, the absolute trajectory error of our algorithm can be reduced by up to 46%. In summary, the proposed dynamic object segmentation method without prior knowledge can significantly reduce the positioning error of SLAM algorithm in various dynamic environments.

RSO-SLAM: A Robust Semantic Visual SLAM With Optical Flow in Complex Dynamic Environments

RSO-SLAM: A Robust Semantic Visual SLAM With Optical Flow in Complex Dynamic Environments

Enhanced Visual SLAM for Collision-Free Driving with Lightweight Autonomous Cars.

The paper presents a vision-based obstacle avoidance strategy for lightweight self-driving cars that can be run on a CPU-only device using a single RGB-D camera. The method consists of two steps: visual perception and path planning. The visual perception part uses ORBSLAM3 enhanced with optical flow to estimate the car's poses and extract rich texture information from the scene. In the path planning phase, the proposed method employs a method combining a control Lyapunov function and control barrier function in the form of a quadratic program (CLF-CBF-QP) together with an obstacle shape reconstruction process (SRP) to plan safe and stable trajectories. To validate the performance and robustness of the proposed method, simulation experiments were conducted with a car in various complex indoor environments using the Gazebo simulation environment. The proposed method can effectively avoid obstacles in the scenes. The proposed algorithm outperforms benchmark algorithms in achieving more stable and shorter trajectories across multiple simulated scenes.