Visual SLAM Research Articles

Visual Simultaneous Localization and Mapping Optimization Method Based on Object Detection in Dynamic Scene

SLAM (Simultaneous Localization and Mapping), as one of the basic functions of mobile robots, has become a hot topic in the field of robotics this year. The majority of SLAM systems in use today, however, disregard the impact of dynamic objects on the system by defining the external environment as static. A SLAM system suitable for dynamic scenes is proposed, aiming at the issue that dynamic objects in real scenes can affect the localization accuracy and map effect of traditional visual SLAM systems. Initially, the enhanced lightweight YOLOv5s target detection algorithm is employed to detect dynamic objects in each frame of the image. Simultaneously, an assessment is conducted on the feature points present on dynamic objects to determine their potential impact on system accuracy, subsequently guiding the decision to retain or exclude these feature points. The preserved static feature points are then utilized for pose estimation and map construction. Experiments on the publicly available TUM dataset and the KITTI dataset are conducted to compare the system in this paper with ORB-SLAM 3, DS-SLAM, and DynaSLAM, and the algorithm is verified to have better performance.

Optimization method to improve visual SLAM in dynamic environment

Optimization method to improve visual SLAM in dynamic environment

NAVS: A Neural Attention-Based Visual SLAM for Autonomous Navigation in Unknown 3D Environments

Navigation in unknown 3D environments aims to progressively find an efficient path to a given target goal in unseen scenarios. A challenge is how to explore the navigation quickly and effectively. An end-to-end learning approach has been proposed to extract geometric shapes from RGB images, but it is not suitable for large environments due to its exhaustive exploration with exponential search space. Active Neural SLAM (ANS) presents a Neural SLAM module to maximize the exploration coverage to tackle the active SLAM task. However, ANS still frequently visits the explored areas due to the inappropriate local target selection. In this paper, we propose a Neural Attention-based Visual SLAM (NAVS) model to explore unknown 3D environments. Spatial attention is provided to quickly identify obstacles (such as similarly colored tea table or floor). We also leverage the priority of unknown regions in the short-term goal decision to avoid frequent exploration with a channel attention. The experimental results show that our model can build a more accurate map than ANS and other baseline methods with less running time. In terms of relative coverage, NAVS achieves a 0.5%\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\%$$\\end{document} improvement over ANS in overall and a 1.1%\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\%$$\\end{document} improvement over ANS in large environments.

One step back, two steps forward: learning moves to recover from SLAM tracking failures

Tracking failure in state-of-the-art visual SLAM has been reported to be frequent and hampers real-world deployment of SLAM solutions. Very recently, efforts have been made to avoid tracking failure using various methods (e.g. using deep reinforcement learning to plan a path where the risk of tracking failure is minimal). The results of those approaches are encouraging but are far from producing a failure-free visual system. Failure is inevitable in vision-based systems. Therefore, developing recovery (post-failure) solutions might be a better approach in developing more reliable systems. To this end, we propose a novel, easily trainable, deep recovery maneuver algorithm. Instead of predicting tracking failures, our algorithm predicts the back stepping move, post failure, which helps regain tracking. With our proposed simple approach, state-of-the-art SLAM manages to complete 13 out of 50 navigation episodes (13x improvement over the baseline). Furthermore, even for cases where our algorithm fails to complete the route, it traverses 2x longer distances as compared to baseline methods.

DN-SLAM: A Visual SLAM With ORB Features and NeRF Mapping in Dynamic Environments

DN-SLAM: A Visual SLAM With ORB Features and NeRF Mapping in Dynamic Environments

Real-time visual SLAM based YOLO-Fastest for dynamic scenes

Within the realm of autonomous robotic navigation, simultaneous localization and mapping (SLAM) serves as a critical perception technology, drawing heightened attention in contemporary research. The traditional SLAM systems perform well in static environments, but in the real physical world, dynamic objects can destroy the static geometric constraints of the SLAM system, further limiting its practical application in the real world. In this paper, a robust dynamic RGB-D SLAM system is proposed to expand the number of static points in the scene by combining with YOLO-Fastest to ensure the effectiveness of the geometric constraints model construction, and then based on that, a new thresholding model is designed to differentiate the dynamic features in the objection bounding box, which takes advantage of the double polyline constraints and the residuals after reprojection to filter the dynamic feature points. In addition, two Gaussian models are constructed to segment the moving objects in the bounding box in the depth image to achieve the effect similar to the instance segmentation under the premise of ensuring the computational speed. In this paper, experiments are conducted on dynamic sequences provided by the TUM dataset to evaluate the performance of the proposed method, and the results show that the root mean squared error metric of the absolute trajectory error of the algorithm of this paper has at least 80% improvement compared to ORB-SLAM2. Higher robustness in dynamic environments with both high and low dynamic sequences compared to DS-SLAM and Dynaslam, and can effectively provide intelligent localization and navigation for mobile robots.

V-SLAM-AIDED PHOTOGRAMMETRY TO PROCESS FISHEYE MULTI-CAMERA SYSTEMS SEQUENCES

Abstract. The advent of mobile mapping systems (MMSs) and computer vision algorithms has enriched a wide range of navigation and mapping tasks such as localisation, 3D motion estimation and 3D mapping. This study focuses on Visual Simultaneous Localisation and Mapping (V-SLAM) in the context of two in-houses MMSs: Ant3D, a patented five-fisheye multi-camera rig and GeoRizon, a high-resolution stereo fisheye rig. The aim is to leverage V-SLAM to enhance the systems performance in near-real-time and non-real-time 3D reconstruction applications. The research investigates both Monocular and Stereo V-SLAM applied to both MMSs and tackles the challenge of combining the V-SLAM estimated trajectory of one or a pair of cameras with known multi-camera relative orientation. We propose a state-of-the-art code that serves as a flexible and extensible platform for MMSs image acquisition and processing, along with an adapted version of the well-established ORB-SLAM3.0. Evaluation is performed in a cultural heritage challenging setup: the Minguzzi spiral staircase in the Duomo di Milano Cathedral. Performed tests highlight that introducing V-SLAM trajectories as well as pre-calibrated interior orientation and multi-camera constraints improve speed, applicability and accuracy of 3D surveys.

A COMPARISON BETWEEN TERRESTRIAL LASER SCANNING AND HAND-HELD MOBILE MAPPING FOR THE DOCUMENTATION OF BUILT HERITAGE

Abstract. The increasing demand for 3D digital models has spurred the rapid evolution of technologies to acquire, process, and disseminate 3D data of physical artifacts. Current scanners meet building surveying needs, prompting the exploration of closed systems with multiple sensors, custom accessories, control systems, and varied data processing approaches. While efficient in specific conditions, these systems require careful consideration of factors like technical requirements, environmental conditions, and the intended use of the final model. This article compares a static terrestrial laser scanner (TLS) and a portable mobile mapping system (PMMS) for documenting built heritage, focusing on the City Hall of Montecatini Terme in Italy. The Leica Geosystems RTC360 scanner provides quick scanning times and pre-alignment during fieldwork, reducing post-processing efforts. The Leica BLK2GO, belonging to the PMMS family, utilizes laser SLAM, visual SLAM, and inertial IMU measurements for rapid large-area documentation. The comparison assesses accuracy, completeness, and detail recognition, revealing that both systems are suitable for heritage documentation requiring centimetre-level accuracy. However, differences in point density and roughness indicate that the RTC360 may be better for intricate details. The BLK2GO excels in efficiency but is more oriented toward general representations at the architectural scale. This study emphasizes the importance of evaluating different methodologies based on project objectives and desired levels of detail, providing insights into the strengths and limitations of each system for diverse applications in heritage documentation.

LIVE PROCESSING AND FEEDBACK ON A COMBINED LASERSCANNING AND PHOTOGRAMMETRIC DOCUMENTATION WORKFLOW

Abstract. The use of a combined terrestrial laserscanning (TLS) and structure-from-motion (SFM) surveying approach has proved to be able to deliver advantageous benefits in terms of coverage and surface rendition. The elaboration of such approaches entails a succession of processes and are without many options for on-site feedback largely dependent on the experience of the surveyor. In support of this approach a combined automated processing workflow with augmented reality feedback is introduced, allowing operators to get a better understanding of their progress and project success during surveying. To do so native TLS and SFM software is connected to a shared network and data storage environment and controlled within an active framework. Within this, scans and images are aligned within RealityCapture and resulting registration is visualised to the operator using augmented reality (AR). To achieve complete independence of pre-scanned environments, the augmented reality application presented within this research uses a GNSS supported visual SLAM algorithm. With this the application is able to position virtual objects – such as markers representing the positions of registered cameras – within a physical space using only their geoposition as reference. The workflow presented here should be understood as a proof of concept to determine possible improvements that would enable such interactive means of surveying and would allow deeper integration of components into a combined AR supported documentation workflow.

Research on Indoor 3D Reconstruction Technology Based on Semantic Visual Simultaneous Localization and Mapping

In response to the challenge that traditional visual simultaneous localization and mapping (SLAM) systems, based on the assumption of a static environment, struggle to achieve real-time indoor 3D reconstruction in complex dynamic scenes, this paper proposes a real-time indoor 3D reconstruction algorithm based on semantic visual SLAM. By leveraging object detection to obtain 2D semantic information and providing prior information for geometric methods, the fusion of the two effectively suppresses dynamic features, reduces reliance on deep learning methods, and ensures the algorithm's real-time performance. Experimental results on dynamic scenes in the TUM RGB-D dataset show that our algorithm maintains nearly unchanged real-time performance while achieving an average performance improvement of approximately 97.56% and 97.31% on the TUM dataset and Bonn dataset, respectively, compared to the ORB-SLAM2 system. Moreover, our algorithm can reconstruct more intuitive indoor global Octo-map and semantic metric maps compared to sparse point cloud maps, effectively enhancing the scene perception capability of mobile robots and laying the foundation for performing advanced tasks. Furthermore, our algorithm demonstrates a 3.5-10.5 times improvement in real-time performance compared to other mainstream semantic SLAM systems. Experimental results on the NVIDIA Jetson AGX Xavier confirm that our algorithm can run in real time on low-power platforms such as mobile robots or drones. However, the drawbacks of our algorithm include lower reconstruction accuracy in low-texture and large-scale scenes and ineffective suppression of dynamic features in low-dynamic scenes. Future work will consider replacing and improving deep learning methods and integrating IMU and other sensors to enhance system usability.

A dynamic detection and data association method based on probabilistic models for visual SLAM

Visual Simultaneous Localization and Mapping (VSLAM) is a critical foundation in mobile robotics and augmented reality (AR). However, VSLAM faces challenges in dynamic environments since both the camera and the object are in motion, which contradicts the classical static scene assumption. Generally, multi-view geometry is employed for static features to estimate camera pose and reconstruct environment maps. Hence, dynamic feature detection and data association become key issues in dynamic VSLAM. To solve these problems, we propose an innovative probability-based approach that combines instance segmentation and nonparametric Kolmogorov–Smirnov test approaches to detect the distribution of features on an object. Furthermore, we propose a data association algorithm based on the Bayesian model that comprehensively utilizes the descriptors of feature points and their spatial information. Experiments on the KITTI public dataset and the Oxford Multi-motion dataset validate the effectiveness of our method.

An Enhancement of Outdoor Location-Based Augmented Reality Anchor Precision through VSLAM and Google Street View.

Outdoor Location-Based Augmented Reality (LAR) applications require precise positioning for seamless integrations of virtual content into immersive experiences. However, common solutions in outdoor LAR applications rely on traditional smartphone sensor fusion methods, such as the Global Positioning System (GPS) and compasses, which often lack the accuracy needed for precise AR content alignments. In this paper, we introduce an innovative approach to enhance LAR anchor precision in outdoor environments. We leveraged Visual Simultaneous Localization and Mapping (VSLAM) technology, in combination with innovative cloud-based methodologies, and harnessed the extensive visual reference database of Google Street View (GSV), to address the accuracy limitation problems. For the evaluation, 10 Point of Interest (POI) locations were used as anchor point coordinates in the experiments. We compared the accuracies between our approach and the common sensor fusion LAR solution comprehensively involving accuracy benchmarking and running load performance testing. The results demonstrate substantial enhancements in overall positioning accuracies compared to conventional GPS-based approaches for aligning AR anchor content in the real world.

NeoSLAM: Long-Term SLAM Using Computational Models of the Brain.

Simultaneous Localization and Mapping (SLAM) is a fundamental problem in the field of robotics, enabling autonomous robots to navigate and create maps of unknown environments. Nevertheless, the SLAM methods that use cameras face problems in maintaining accurate localization over extended periods across various challenging conditions and scenarios. Following advances in neuroscience, we propose NeoSLAM, a novel long-term visual SLAM, which uses computational models of the brain to deal with this problem. Inspired by the human neocortex, NeoSLAM is based on a hierarchical temporal memory model that has the potential to identify temporal sequences of spatial patterns using sparse distributed representations. Being known to have a high representational capacity and high tolerance to noise, sparse distributed representations have several properties, enabling the development of a novel neuroscience-based loop-closure detector that allows for real-time performance, especially in resource-constrained robotic systems. The proposed method has been thoroughly evaluated in terms of environmental complexity by using a wheeled robot deployed in the field and demonstrated that the accuracy of loop-closure detection was improved compared with the traditional RatSLAM system.

Adaptive Point-Line Fusion: A Targetless LiDAR-Camera Calibration Method with Scheme Selection for Autonomous Driving.

Accurate calibration between LiDAR and camera sensors is crucial for autonomous driving systems to perceive and understand the environment effectively. Typically, LiDAR-camera extrinsic calibration requires feature alignment and overlapping fields of view. Aligning features from different modalities can be challenging due to noise influence. Therefore, this paper proposes a targetless extrinsic calibration method for monocular cameras and LiDAR sensors that have a non-overlapping field of view. The proposed solution uses pose transformation to establish data association across different modalities. This conversion turns the calibration problem into an optimization problem within a visual SLAM system without requiring overlapping views. To improve performance, line features serve as constraints in visual SLAM. Accurate positions of line segments are obtained by utilizing an extended photometric error optimization method. Moreover, a strategy is proposed for selecting appropriate calibration methods from among several alternative optimization schemes. This adaptive calibration method selection strategy ensures robust calibration performance in urban autonomous driving scenarios with varying lighting and environmental textures while avoiding failures and excessive bias that may result from relying on a single approach.

Current study on multi-robot collaborative vision SLAM

Simultaneous Localization and Mapping (SLAM) stands as a vital technology for automatic control of robots. The significance of vision-based multi-robot collaborative SLAM technology is noteworthy in this domain, because visual SLAM uses cameras as the main sensor, which offers the benefits of easy access to environmental information and convenient installation. And the multi-robot system has the advantages of high efficiency, high fault tolerance, and high precision, so the multi-robot system can work in a complex environment and ensure its mapping efficiency, these may be a challenge for a single robot. This paper introduces the principles and common methods of visual SLAM, as well as the main algorithms of multi-robot collaborative SLAM. This paper analyzed the main problems existing in the current multi-robot collaborative visual SLAM technology: multi-robot SLAM task allocation, map fusion and back-end optimization. Then this paper listed different solutions, and analyzed their advantages and disadvantages. In addition, this paper also introduces some future research prospects of multi-robot collaborative visual SLAM technology, aiming to provide a reference direction for subsequent research in related fields.

Research on Inter-Frame Feature Mismatch Removal Method of VSLAM in Dynamic Scenes

Visual Simultaneous Localization and Mapping (VSLAM) estimates the robot’s pose in three-dimensional space by analyzing the depth variations of inter-frame feature points. Inter-frame feature point mismatches can lead to tracking failure, impacting the accuracy of the mobile robot’s self-localization and mapping. This paper proposes a method for removing mismatches of image features in dynamic scenes in visual SLAM. First, the Grid-based Motion Statistics (GMS) method was introduced for fast coarse screening of mismatched image features. Second, an Adaptive Error Threshold RANSAC (ATRANSAC) method, determined by the internal matching rate, was proposed to improve the accuracy of removing mismatched image features in dynamic and static scenes. Third, the GMS-ATRANSAC method was tested for removing mismatched image features, and experimental results showed that GMS-ATRANSAC can remove mismatches of image features on moving objects. It achieved an average error reduction of 29.4% and 32.9% compared to RANSAC and GMS-RANSAC, with a corresponding reduction in error variance of 63.9% and 58.0%, respectively. The processing time was reduced by 78.3% and 38%, respectively. Finally, the effectiveness of inter-frame feature mismatch removal in the initialization thread of ORB-SLAM2 and the tracking thread of ORB-SLAM3 was verified for the proposed algorithm.

RGB-D Based Visual SLAM Algorithm for Indoor Crowd Environment

Most current research on dynamic visual Simultaneous Localization and Mapping (SLAM) systems focuses on scenes where static objects occupy most of the environment. However, in densely populated indoor environments, the movement of the crowd can lead to the loss of feature information, thereby diminishing the system’s robustness and accuracy. This paper proposes a visual SLAM algorithm for dense crowd environments based on a combination of the ORB-SLAM2 framework and RGB-D cameras. Firstly, we introduced a dedicated target detection network thread and improved the performance of the target detection network, enhancing its detection coverage in crowded environments, resulting in a 41.5% increase in average accuracy. Additionally, we found that some feature points other than humans in the detection box were mistakenly deleted. Therefore, we proposed an algorithm based on standard deviation fitting to effectively filter out the features. Finally, our system is evaluated on the TUM and Bonn RGB-D dynamic datasets and compared with ORB-SLAM2 and other state-of-the-art visual dynamic SLAM methods. The results indicate that our system’s pose estimation error is reduced by at least 93.60% and 97.11% compared to ORB-SLAM2 in high dynamic environments and the Bonn RGB-D dynamic dataset, respectively. Our method demonstrates comparable performance compared to other recent visual dynamic SLAM methods.

Active View Planning for Visual SLAM in Outdoor Environments Based on Continuous Information Modeling

The visual simultaneous localization and mapping(vSLAM) is widely used in GPS-denied and open field environments for ground and surface robots. However, due to the frequent perception failures derived from lacking visual texture or the {swing} of robot view direction on rough terrains, the accuracy and robustness of vSLAM are still to be enhanced. The study develops a novel view planning approach of actively perceiving areas with maximal information to address the mentioned problem; a gimbal camera is used as the main sensor. Firstly, a map representation based on feature distribution-weighted Fisher information is proposed to completely and effectively represent environmental information richness. With the map representation, a continuous environmental information model is further established to convert the discrete information space into a continuous one for numerical optimization in real-time. Subsequently, the receding horizon optimization is utilized to obtain the optimal informative viewpoints with simultaneously considering the robotic perception, exploration and motion cost based on the continuous environmental model. Finally, several simulations and outdoor experiments are performed to verify the improvement of localization robustness and accuracy by the proposed approach.

UDS-SLAM: real-time robust visual SLAM based on semantic segmentation in dynamic scenes

PurposeExisting Simultaneous Localization and Mapping (SLAM) algorithms have been relatively well developed. However, when in complex dynamic environments, the movement of the dynamic points on the dynamic objects in the image in the mapping can have an impact on the observation of the system, and thus there will be biases and errors in the position estimation and the creation of map points. The aim of this paper is to achieve more accurate accuracy in SLAM algorithms compared to traditional methods through semantic approaches.Design/methodology/approachIn this paper, the semantic segmentation of dynamic objects is realized based on U-Net semantic segmentation network, followed by motion consistency detection through motion detection method to determine whether the segmented objects are moving in the current scene or not, and combined with the motion compensation method to eliminate dynamic points and compensate for the current local image, so as to make the system robust.FindingsExperiments comparing the effect of detecting dynamic points and removing outliers are conducted on a dynamic data set of Technische Universität München, and the results show that the absolute trajectory accuracy of this paper's method is significantly improved compared with ORB-SLAM3 and DS-SLAM.Originality/valueIn this paper, in the semantic segmentation network part, the segmentation mask is combined with the method of dynamic point detection, elimination and compensation, which reduces the influence of dynamic objects, thus effectively improving the accuracy of localization in dynamic environments.

Loop closure detection of visual SLAM based on variational autoencoder.

Loop closure detection is an important module for simultaneous localization and mapping (SLAM). Correct detection of loops can reduce the cumulative drift in positioning. Because traditional detection methods rely on handicraft features, false positive detections can occur when the environment changes, resulting in incorrect estimates and an inability to obtain accurate maps. In this research paper, a loop closure detection method based on a variational autoencoder (VAE) is proposed. It is intended to be used as a feature extractor to extract image features through neural networks to replace the handicraft features used in traditional methods. This method extracts a low-dimensional vector as the representation of the image. At the same time, the attention mechanism is added to the network and constraints are added to improve the loss function for better image representation. In the back-end feature matching process, geometric checking is used to filter out the wrong matching for the false positive problem. Finally, through numerical experiments, the proposed method is demonstrated to have a better precision-recall curve than the traditional method of the bag-of-words model and other deep learning methods and is highly robust to environmental changes. In addition, experiments on datasets from three different scenarios also demonstrate that the method can be applied in real-world scenarios and that it has a good performance.