Visual SLAM Research Articles

A semantic SLAM-based method for navigation and landing of UAVs in indoor environments

Autonomous navigation and landing of Unmanned aerial vehicles (UAVs) are critical functions towards full intelligence in unknown environments. However, current methods heavily rely on positioning devices or geometric maps, leading to two limitations: (1) UAVs fail to work normally indoors when signals are blocked, and (2) Navigation systems are not capable of achieving high-level decision-making missions. In this paper, we propose a novel and complete framework to realize the autonomous landing of UAVs in unknown indoor scenes based on visual SLAM, semantic segmentation, terrain estimation, and a decision-making model. Firstly, our 3D map is built on top of the visual SLAM system with semantic features, and then multiple types of maps (e.g., texture map, octo-map, grid-map, and semantic topology) are created for different function requirements. To achieve high-level scene understanding, we design a data association rule to fuse semantic features extracted by a deep learning model into the mapping process of topology structure. Next, a terrain estimation strategy is performed in the lightweight grid-map, which is modeled only via low-level elevation representation. Through multiple terrain constraints factor, a few optimized landing sites are selected in safe regions via clustering algorithm. Finally, based on the guidance of semantic topology, we construct a decision-making model for UAV landing that effectively integrates terrain safety, environment perception, and path planning. We perform extensive autonomous landing experiments on multiple indoor scenes from the TUM RGB-D dataset, and the full unknown environment maps are created simultaneously. The experiment proves that the proposed framework can select an optimal landing site for the advanced mission of UAVs effectively.

RU-SLAM: A Robust Deep-Learning Visual Simultaneous Localization and Mapping (SLAM) System for Weakly Textured Underwater Environments.

Accurate and robust simultaneous localization and mapping (SLAM) systems are crucial for autonomous underwater vehicles (AUVs) to perform missions in unknown environments. However, directly applying deep learning-based SLAM methods to underwater environments poses challenges due to weak textures, image degradation, and the inability to accurately annotate keypoints. In this paper, a robust deep-learning visual SLAM system is proposed. First, a feature generator named UWNet is designed to address weak texture and image degradation problems and extract more accurate keypoint features and their descriptors. Further, the idea of knowledge distillation is introduced based on an improved underwater imaging physical model to train the network in a self-supervised manner. Finally, UWNet is integrated into the ORB-SLAM3 to replace the traditional feature extractor. The extracted local and global features are respectively utilized in the feature tracking and closed-loop detection modules. Experimental results on public datasets and self-collected pool datasets verify that the proposed system maintains high accuracy and robustness in complex scenarios.

OSPC: Online Sequential Photometric Calibration

Photometric calibration is essential to many computer vision applications. One of its key benefits is enhancing the performance of Visual SLAM, especially when it depends on a direct method for tracking, such as the standard KLT algorithm. Additionally, it proves valuable in extracting sensor irradiance values from measured intensities, serving as a pre-processing step for a number of vision algorithms such as shape-from-shading. Current photometric calibration systems rely on a joint optimization problem and encounter an ambiguity in the estimates, which can only be resolved using ground truth information. We propose a novel method that solves for photometric parameters using a sequential estimation approach. To enhance the decoupling of CRF and Vignette estimation, we strategically utilize keyframes with high exposure ratios and small displacements for the former, and keyframes with relatively large displacements for the latter. Our proposed method achieves high accuracy in estimating all parameters; furthermore, the formulations are linear and convex, which makes the solution fast and suitable for online applications. Experiments on a Visual Odometry system validate the proposed method and demonstrate its advantages.

SiLK-SLAM: accurate, robust and versatile visual SLAM with simple learned keypoints

PurposeWeak repeatability is observed in handcrafted keypoints, leading to tracking failures in visual simultaneous localization and mapping (SLAM) systems under challenging scenarios such as illumination change, rapid rotation and large angle of view variation. In contrast, learning-based keypoints exhibit higher repetition but entail considerable computational costs. This paper proposes an innovative algorithm for keypoint extraction, aiming to strike an equilibrium between precision and efficiency. This paper aims to attain accurate, robust and versatile visual localization in scenes of formidable complexity.Design/methodology/approachSiLK-SLAM initially refines the cutting-edge learning-based extractor, SiLK, and introduces an innovative postprocessing algorithm for keypoint homogenization and operational efficiency. Furthermore, SiLK-SLAM devises a reliable relocalization strategy called PCPnP, leveraging progressive and consistent sampling, thereby bolstering its robustness.FindingsEmpirical evaluations conducted on TUM, KITTI and EuRoC data sets substantiate SiLK-SLAM’s superior localization accuracy compared to ORB-SLAM3 and other methods. Compared to ORB-SLAM3, SiLK-SLAM demonstrates an enhancement in localization accuracy even by 70.99%, 87.20% and 85.27% across the three data sets. The relocalization experiments demonstrate SiLK-SLAM’s capability in producing precise and repeatable keypoints, showcasing its robustness in challenging environments.Originality/valueThe SiLK-SLAM achieves exceedingly elevated localization accuracy and resilience in formidable scenarios, holding paramount importance in enhancing the autonomy of robots navigating intricate environments. Code is available at https://github.com/Pepper-FlavoredChewingGum/SiLK-SLAM.

How Challenging is a Challenge? CEMS: a Challenge Evaluation Module for SLAM Visual Perception

Despite promising SLAM research in both vision and robotics communities, which fundamentally sustains the autonomy of intelligent unmanned systems, visual challenges still threaten its robust operation severely. Existing SLAM methods usually focus on specific challenges and solve the problem with sophisticated enhancement or multi-modal fusion. However, they are basically limited to particular scenes with a non-quantitative understanding and awareness of challenges, resulting in a significant performance decline with poor generalization and(or) redundant computation with inflexible mechanisms. To push the frontier of visual SLAM, we propose a fully computational reliable evaluation module called CEMS (Challenge Evaluation Module for SLAM) for general visual perception based on a clear definition and systematic analysis. It decomposes various challenges into several common aspects and evaluates degradation with corresponding indicators. Extensive experiments demonstrate our feasibility and outperformance. The proposed module has a high consistency of 88.298% compared with annotation ground truth, and a strong correlation of 0.879 compared with SLAM tracking performance. Moreover, we show the prototype SLAM based on CEMS with better performance and the first comprehensive CET (Challenge Evaluation Table) for common SLAM datasets (EuRoC, KITTI, etc.) with objective and fair evaluations of various challenges. We make it available online to benefit the community on our website.

Semisupervised Vector Quantization in Visual SLAM Using HGCN

We present a novel vector quantization (VQ) module for the two state-of-the-art long-range simultaneous localization and mapping (SLAM) algorithms. The VQ task in SLAM is generally performed using unsupervised methods. We provide an alternative approach trough embedding a semisupervised hyperbolic graph convolutional neural network (HGCN) in the VQ step of the SLAM processes. The SLAM platforms we have utilized for this purpose are fast appearance-based mapping (FABMAP) and oriented fast and rotated short (ORB), both of which rely on extracting the features of the captured images in their loop closure detection (LCD) module. For the first time, we have considered the space formed by these SURF features, robust image descriptors, as a graph, enabling us to apply an HGCN in the VQ section which results in an improved LCD performance. The HGCN vector quantizes the SURF feature space, leading to a bag-of-word (BoW) representation construction of the images. This representation is subsequently used to determine LCD accuracy and recall. Our approaches in this study are referred to as HGCN-FABMAP and HGCN-ORB. The main advantage of using HGCN in the LCD section is that it scales linearly when the features are accumulated. The benchmarking experiments show the superiority of our methods in terms of both trajectory generation accuracy in small-scale paths and LCD accuracy and recall for large-scale problems.

Open Dataset for Testing of Visual SLAM Algorithms under Different Weather Conditions

Existing datasets for testing SLAM algorithms in outdoor environments are not suitable for assessing the influence of weather conditions on localization accuracy. Obtaining a suitable dataset from the real world is difficult due to the long data collection period and the inability to exclude dynamic environmental factors. Artificially generated datasets make it possible to bypass the described limitations, but up to date, researchers have not identified testing SLAM algorithms under different weather conditions as a stand-alone task, despite the fact that it is one of the main aspects of the difference between outdoor and indoor environments. This work presents a new open dataset that consists of 36 sequences of robot movement in an urban environment or rough terrain, in the form of images from a stereo camera and the ground truth position of the robot, collected at a frequency of 30 Hz. Movement within one area occurs along a fixed route; the sequences are distinguished only by whether conditions, which can make it possible to correctly assess the influence of weather phenomena on the accuracy of localization.

A Synchronous Localization Method for Multiple Mobile Robots Based on Visual SLAM Algorithm in the Context of the Internet of Things

Abstract Aiming at the problem that traditional methods cannot meet the problem of synchronous positioning of multi-mobile robots in different motion states, a synchronous positioning method of multi-mobile robots based on visual simultaneous localization and mapping (SLAM) algorithm is proposed. Analyze the general model of SLAM problem, realize visual SLAM feature detection through scale invariant feature transform algorithm, and match image features based on active vision. A multi-mobile robot motion state estimation model is constructed, an octree model is used to construct a multi-robot synchronous localization map, and the camera pose tracking is realized by using the oriented fast accelerated segment test and rotated binary robust independent elementary features points that are not in the target bounding box. The three-dimensional points in the environment recovered by visual SLAM separate the background target and the moving target, and match the corresponding target in the previous frame through the feature to realize the synchronous positioning of the multi-mobile robot. The experimental results show that the method can realize the synchronous positioning of multi-robots under the state of multi-robot linear motion, curvilinear motion and mixed motion, and the positioning accuracy is high.

Visual Simultaneous Localization and Mapping Optimization Method Based on Object Detection in Dynamic Scene

SLAM (Simultaneous Localization and Mapping), as one of the basic functions of mobile robots, has become a hot topic in the field of robotics this year. The majority of SLAM systems in use today, however, disregard the impact of dynamic objects on the system by defining the external environment as static. A SLAM system suitable for dynamic scenes is proposed, aiming at the issue that dynamic objects in real scenes can affect the localization accuracy and map effect of traditional visual SLAM systems. Initially, the enhanced lightweight YOLOv5s target detection algorithm is employed to detect dynamic objects in each frame of the image. Simultaneously, an assessment is conducted on the feature points present on dynamic objects to determine their potential impact on system accuracy, subsequently guiding the decision to retain or exclude these feature points. The preserved static feature points are then utilized for pose estimation and map construction. Experiments on the publicly available TUM dataset and the KITTI dataset are conducted to compare the system in this paper with ORB-SLAM 3, DS-SLAM, and DynaSLAM, and the algorithm is verified to have better performance.

Optimization method to improve visual SLAM in dynamic environment

Optimization method to improve visual SLAM in dynamic environment

NAVS: A Neural Attention-Based Visual SLAM for Autonomous Navigation in Unknown 3D Environments

Navigation in unknown 3D environments aims to progressively find an efficient path to a given target goal in unseen scenarios. A challenge is how to explore the navigation quickly and effectively. An end-to-end learning approach has been proposed to extract geometric shapes from RGB images, but it is not suitable for large environments due to its exhaustive exploration with exponential search space. Active Neural SLAM (ANS) presents a Neural SLAM module to maximize the exploration coverage to tackle the active SLAM task. However, ANS still frequently visits the explored areas due to the inappropriate local target selection. In this paper, we propose a Neural Attention-based Visual SLAM (NAVS) model to explore unknown 3D environments. Spatial attention is provided to quickly identify obstacles (such as similarly colored tea table or floor). We also leverage the priority of unknown regions in the short-term goal decision to avoid frequent exploration with a channel attention. The experimental results show that our model can build a more accurate map than ANS and other baseline methods with less running time. In terms of relative coverage, NAVS achieves a 0.5%\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\%$$\\end{document} improvement over ANS in overall and a 1.1%\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\%$$\\end{document} improvement over ANS in large environments.

One step back, two steps forward: learning moves to recover from SLAM tracking failures

Tracking failure in state-of-the-art visual SLAM has been reported to be frequent and hampers real-world deployment of SLAM solutions. Very recently, efforts have been made to avoid tracking failure using various methods (e.g. using deep reinforcement learning to plan a path where the risk of tracking failure is minimal). The results of those approaches are encouraging but are far from producing a failure-free visual system. Failure is inevitable in vision-based systems. Therefore, developing recovery (post-failure) solutions might be a better approach in developing more reliable systems. To this end, we propose a novel, easily trainable, deep recovery maneuver algorithm. Instead of predicting tracking failures, our algorithm predicts the back stepping move, post failure, which helps regain tracking. With our proposed simple approach, state-of-the-art SLAM manages to complete 13 out of 50 navigation episodes (13x improvement over the baseline). Furthermore, even for cases where our algorithm fails to complete the route, it traverses 2x longer distances as compared to baseline methods.

DN-SLAM: A Visual SLAM With ORB Features and NeRF Mapping in Dynamic Environments

DN-SLAM: A Visual SLAM With ORB Features and NeRF Mapping in Dynamic Environments

Real-time visual SLAM based YOLO-Fastest for dynamic scenes

Within the realm of autonomous robotic navigation, simultaneous localization and mapping (SLAM) serves as a critical perception technology, drawing heightened attention in contemporary research. The traditional SLAM systems perform well in static environments, but in the real physical world, dynamic objects can destroy the static geometric constraints of the SLAM system, further limiting its practical application in the real world. In this paper, a robust dynamic RGB-D SLAM system is proposed to expand the number of static points in the scene by combining with YOLO-Fastest to ensure the effectiveness of the geometric constraints model construction, and then based on that, a new thresholding model is designed to differentiate the dynamic features in the objection bounding box, which takes advantage of the double polyline constraints and the residuals after reprojection to filter the dynamic feature points. In addition, two Gaussian models are constructed to segment the moving objects in the bounding box in the depth image to achieve the effect similar to the instance segmentation under the premise of ensuring the computational speed. In this paper, experiments are conducted on dynamic sequences provided by the TUM dataset to evaluate the performance of the proposed method, and the results show that the root mean squared error metric of the absolute trajectory error of the algorithm of this paper has at least 80% improvement compared to ORB-SLAM2. Higher robustness in dynamic environments with both high and low dynamic sequences compared to DS-SLAM and Dynaslam, and can effectively provide intelligent localization and navigation for mobile robots.

V-SLAM-AIDED PHOTOGRAMMETRY TO PROCESS FISHEYE MULTI-CAMERA SYSTEMS SEQUENCES

Abstract. The advent of mobile mapping systems (MMSs) and computer vision algorithms has enriched a wide range of navigation and mapping tasks such as localisation, 3D motion estimation and 3D mapping. This study focuses on Visual Simultaneous Localisation and Mapping (V-SLAM) in the context of two in-houses MMSs: Ant3D, a patented five-fisheye multi-camera rig and GeoRizon, a high-resolution stereo fisheye rig. The aim is to leverage V-SLAM to enhance the systems performance in near-real-time and non-real-time 3D reconstruction applications. The research investigates both Monocular and Stereo V-SLAM applied to both MMSs and tackles the challenge of combining the V-SLAM estimated trajectory of one or a pair of cameras with known multi-camera relative orientation. We propose a state-of-the-art code that serves as a flexible and extensible platform for MMSs image acquisition and processing, along with an adapted version of the well-established ORB-SLAM3.0. Evaluation is performed in a cultural heritage challenging setup: the Minguzzi spiral staircase in the Duomo di Milano Cathedral. Performed tests highlight that introducing V-SLAM trajectories as well as pre-calibrated interior orientation and multi-camera constraints improve speed, applicability and accuracy of 3D surveys.

A COMPARISON BETWEEN TERRESTRIAL LASER SCANNING AND HAND-HELD MOBILE MAPPING FOR THE DOCUMENTATION OF BUILT HERITAGE

Abstract. The increasing demand for 3D digital models has spurred the rapid evolution of technologies to acquire, process, and disseminate 3D data of physical artifacts. Current scanners meet building surveying needs, prompting the exploration of closed systems with multiple sensors, custom accessories, control systems, and varied data processing approaches. While efficient in specific conditions, these systems require careful consideration of factors like technical requirements, environmental conditions, and the intended use of the final model. This article compares a static terrestrial laser scanner (TLS) and a portable mobile mapping system (PMMS) for documenting built heritage, focusing on the City Hall of Montecatini Terme in Italy. The Leica Geosystems RTC360 scanner provides quick scanning times and pre-alignment during fieldwork, reducing post-processing efforts. The Leica BLK2GO, belonging to the PMMS family, utilizes laser SLAM, visual SLAM, and inertial IMU measurements for rapid large-area documentation. The comparison assesses accuracy, completeness, and detail recognition, revealing that both systems are suitable for heritage documentation requiring centimetre-level accuracy. However, differences in point density and roughness indicate that the RTC360 may be better for intricate details. The BLK2GO excels in efficiency but is more oriented toward general representations at the architectural scale. This study emphasizes the importance of evaluating different methodologies based on project objectives and desired levels of detail, providing insights into the strengths and limitations of each system for diverse applications in heritage documentation.

LIVE PROCESSING AND FEEDBACK ON A COMBINED LASERSCANNING AND PHOTOGRAMMETRIC DOCUMENTATION WORKFLOW

Abstract. The use of a combined terrestrial laserscanning (TLS) and structure-from-motion (SFM) surveying approach has proved to be able to deliver advantageous benefits in terms of coverage and surface rendition. The elaboration of such approaches entails a succession of processes and are without many options for on-site feedback largely dependent on the experience of the surveyor. In support of this approach a combined automated processing workflow with augmented reality feedback is introduced, allowing operators to get a better understanding of their progress and project success during surveying. To do so native TLS and SFM software is connected to a shared network and data storage environment and controlled within an active framework. Within this, scans and images are aligned within RealityCapture and resulting registration is visualised to the operator using augmented reality (AR). To achieve complete independence of pre-scanned environments, the augmented reality application presented within this research uses a GNSS supported visual SLAM algorithm. With this the application is able to position virtual objects – such as markers representing the positions of registered cameras – within a physical space using only their geoposition as reference. The workflow presented here should be understood as a proof of concept to determine possible improvements that would enable such interactive means of surveying and would allow deeper integration of components into a combined AR supported documentation workflow.

Research on Indoor 3D Reconstruction Technology Based on Semantic Visual Simultaneous Localization and Mapping

In response to the challenge that traditional visual simultaneous localization and mapping (SLAM) systems, based on the assumption of a static environment, struggle to achieve real-time indoor 3D reconstruction in complex dynamic scenes, this paper proposes a real-time indoor 3D reconstruction algorithm based on semantic visual SLAM. By leveraging object detection to obtain 2D semantic information and providing prior information for geometric methods, the fusion of the two effectively suppresses dynamic features, reduces reliance on deep learning methods, and ensures the algorithm's real-time performance. Experimental results on dynamic scenes in the TUM RGB-D dataset show that our algorithm maintains nearly unchanged real-time performance while achieving an average performance improvement of approximately 97.56% and 97.31% on the TUM dataset and Bonn dataset, respectively, compared to the ORB-SLAM2 system. Moreover, our algorithm can reconstruct more intuitive indoor global Octo-map and semantic metric maps compared to sparse point cloud maps, effectively enhancing the scene perception capability of mobile robots and laying the foundation for performing advanced tasks. Furthermore, our algorithm demonstrates a 3.5-10.5 times improvement in real-time performance compared to other mainstream semantic SLAM systems. Experimental results on the NVIDIA Jetson AGX Xavier confirm that our algorithm can run in real time on low-power platforms such as mobile robots or drones. However, the drawbacks of our algorithm include lower reconstruction accuracy in low-texture and large-scale scenes and ineffective suppression of dynamic features in low-dynamic scenes. Future work will consider replacing and improving deep learning methods and integrating IMU and other sensors to enhance system usability.

A dynamic detection and data association method based on probabilistic models for visual SLAM

Visual Simultaneous Localization and Mapping (VSLAM) is a critical foundation in mobile robotics and augmented reality (AR). However, VSLAM faces challenges in dynamic environments since both the camera and the object are in motion, which contradicts the classical static scene assumption. Generally, multi-view geometry is employed for static features to estimate camera pose and reconstruct environment maps. Hence, dynamic feature detection and data association become key issues in dynamic VSLAM. To solve these problems, we propose an innovative probability-based approach that combines instance segmentation and nonparametric Kolmogorov–Smirnov test approaches to detect the distribution of features on an object. Furthermore, we propose a data association algorithm based on the Bayesian model that comprehensively utilizes the descriptors of feature points and their spatial information. Experiments on the KITTI public dataset and the Oxford Multi-motion dataset validate the effectiveness of our method.

An Enhancement of Outdoor Location-Based Augmented Reality Anchor Precision through VSLAM and Google Street View.

Outdoor Location-Based Augmented Reality (LAR) applications require precise positioning for seamless integrations of virtual content into immersive experiences. However, common solutions in outdoor LAR applications rely on traditional smartphone sensor fusion methods, such as the Global Positioning System (GPS) and compasses, which often lack the accuracy needed for precise AR content alignments. In this paper, we introduce an innovative approach to enhance LAR anchor precision in outdoor environments. We leveraged Visual Simultaneous Localization and Mapping (VSLAM) technology, in combination with innovative cloud-based methodologies, and harnessed the extensive visual reference database of Google Street View (GSV), to address the accuracy limitation problems. For the evaluation, 10 Point of Interest (POI) locations were used as anchor point coordinates in the experiments. We compared the accuracies between our approach and the common sensor fusion LAR solution comprehensively involving accuracy benchmarking and running load performance testing. The results demonstrate substantial enhancements in overall positioning accuracies compared to conventional GPS-based approaches for aligning AR anchor content in the real world.