Visual SLAM Algorithm Research Articles

Enhancing Autonomous Navigation: A Visual SLAM Approach

An autonomous vehicle can simultaneously map its environment and identify its own position by employing a technique called “Simultaneous Localisation And Mapping” (SLAM). Autonomous mobility requires identifying the locations of adjacent landmarks and objects, as well as the vehicle’s position, using an appropriate technique. Monocular SLAM systems often face challenges related to depth perception and scale ambiguity, leading to trajectory drift over time. In contrast, Stereo SLAM systems utilize dual cameras to overcome these limitations. The purpose of this work is to assess how well visual SLAM systems perform by contrasting trajectory estimates with ground truth information obtained from simulations. The findings indicate that stereo visual SLAM algorithms offer more accurate camera trajectory estimations than monocular SLAM, making them a preferable choice for applications demanding precise camera localization and mapping in autonomous vehicles.

Open Dataset for Testing of Visual SLAM Algorithms under Different Weather Conditions

Existing datasets for testing SLAM algorithms in outdoor environments are not suitable for assessing the influence of weather conditions on localization accuracy. Obtaining a suitable dataset from the real world is difficult due to the long data collection period and the inability to exclude dynamic environmental factors. Artificially generated datasets make it possible to bypass the described limitations, but up to date, researchers have not identified testing SLAM algorithms under different weather conditions as a stand-alone task, despite the fact that it is one of the main aspects of the difference between outdoor and indoor environments. This work presents a new open dataset that consists of 36 sequences of robot movement in an urban environment or rough terrain, in the form of images from a stereo camera and the ground truth position of the robot, collected at a frequency of 30 Hz. Movement within one area occurs along a fixed route; the sequences are distinguished only by whether conditions, which can make it possible to correctly assess the influence of weather phenomena on the accuracy of localization.

A Synchronous Localization Method for Multiple Mobile Robots Based on Visual SLAM Algorithm in the Context of the Internet of Things

Abstract Aiming at the problem that traditional methods cannot meet the problem of synchronous positioning of multi-mobile robots in different motion states, a synchronous positioning method of multi-mobile robots based on visual simultaneous localization and mapping (SLAM) algorithm is proposed. Analyze the general model of SLAM problem, realize visual SLAM feature detection through scale invariant feature transform algorithm, and match image features based on active vision. A multi-mobile robot motion state estimation model is constructed, an octree model is used to construct a multi-robot synchronous localization map, and the camera pose tracking is realized by using the oriented fast accelerated segment test and rotated binary robust independent elementary features points that are not in the target bounding box. The three-dimensional points in the environment recovered by visual SLAM separate the background target and the moving target, and match the corresponding target in the previous frame through the feature to realize the synchronous positioning of the multi-mobile robot. The experimental results show that the method can realize the synchronous positioning of multi-robots under the state of multi-robot linear motion, curvilinear motion and mixed motion, and the positioning accuracy is high.

LIVE PROCESSING AND FEEDBACK ON A COMBINED LASERSCANNING AND PHOTOGRAMMETRIC DOCUMENTATION WORKFLOW

Abstract. The use of a combined terrestrial laserscanning (TLS) and structure-from-motion (SFM) surveying approach has proved to be able to deliver advantageous benefits in terms of coverage and surface rendition. The elaboration of such approaches entails a succession of processes and are without many options for on-site feedback largely dependent on the experience of the surveyor. In support of this approach a combined automated processing workflow with augmented reality feedback is introduced, allowing operators to get a better understanding of their progress and project success during surveying. To do so native TLS and SFM software is connected to a shared network and data storage environment and controlled within an active framework. Within this, scans and images are aligned within RealityCapture and resulting registration is visualised to the operator using augmented reality (AR). To achieve complete independence of pre-scanned environments, the augmented reality application presented within this research uses a GNSS supported visual SLAM algorithm. With this the application is able to position virtual objects – such as markers representing the positions of registered cameras – within a physical space using only their geoposition as reference. The workflow presented here should be understood as a proof of concept to determine possible improvements that would enable such interactive means of surveying and would allow deeper integration of components into a combined AR supported documentation workflow.

RGB-D Based Visual SLAM Algorithm for Indoor Crowd Environment

Most current research on dynamic visual Simultaneous Localization and Mapping (SLAM) systems focuses on scenes where static objects occupy most of the environment. However, in densely populated indoor environments, the movement of the crowd can lead to the loss of feature information, thereby diminishing the system’s robustness and accuracy. This paper proposes a visual SLAM algorithm for dense crowd environments based on a combination of the ORB-SLAM2 framework and RGB-D cameras. Firstly, we introduced a dedicated target detection network thread and improved the performance of the target detection network, enhancing its detection coverage in crowded environments, resulting in a 41.5% increase in average accuracy. Additionally, we found that some feature points other than humans in the detection box were mistakenly deleted. Therefore, we proposed an algorithm based on standard deviation fitting to effectively filter out the features. Finally, our system is evaluated on the TUM and Bonn RGB-D dynamic datasets and compared with ORB-SLAM2 and other state-of-the-art visual dynamic SLAM methods. The results indicate that our system’s pose estimation error is reduced by at least 93.60% and 97.11% compared to ORB-SLAM2 in high dynamic environments and the Bonn RGB-D dynamic dataset, respectively. Our method demonstrates comparable performance compared to other recent visual dynamic SLAM methods.

Stereo vision based systems for sea-state measurement and floating structures monitoring

Using computer vision techniques such as stereo vision systems for sea state measurement or for offshore structures monitoring can improve the measurement fidelity and accuracy with no significant additional cost. In this paper, two experiments (in-lab/open-sea) are conducted to study the performance of stereo vision system to measure the water wave surface elevation and rigid body heaving motion. For the in-lab experiment, regular water waves are generated in a wave tank for different frequencies and wave heights, where the water surface is scanned by the stereo vision camera installed on the top of the tank. Surface elevation inferred by the stereo vision is verified by an installed stationary side camera that records the water surface through the tank transparent side window, water surface elevation measured by the side camera recordings is extracted using edge detection algorithm. During the in-lab experiment a heaving buoy is installed to test the performance of Visual Simultaneous Localization and Mapping (VSLAM) algorithm to monitor the buoy heave motion. The VSLAM algorithm fuses a buoy onboard stereo vision recordings with an embedded Inertial Measurement Unit (IMU) to estimate the 6-DOF of a rigid body. The Buoy motion VSLAM measurements are verified by a KLT tracking algorithm implemented on the video recordings of the stationary side camera. The open-sea experiment is implemented in Lake Somerville, Texas. The stereo vision system is installed to measure the water surface elevation and directional spectrum of the wind generated irregular waves. The open-sea wave measurements by the stereo vision are verified by a Sofar commercial wave buoys deployed in the testing location.

Visual simultaneous localization and mapping (vSLAM) algorithm based on improved Vision Transformer semantic segmentation in dynamic scenes

Abstract. Identifying dynamic objects in dynamic scenes remains a challenge for traditional simultaneous localization and mapping (SLAM) algorithms. Additionally, these algorithms are not able to adequately inpaint the culling regions that result from excluding dynamic objects. In light of these challenges, this study proposes a novel visual SLAM (vSLAM) algorithm based on improved Vision Transformer semantic segmentation in dynamic scenes (VTD-SLAM), which leverages an improved Vision Transformer semantic segmentation technique to address these limitations. Specifically, VTD-SLAM utilizes a residual dual-pyramid backbone network to extract dynamic object region features and a multiclass feature transformer segmentation module to enhance the pixel weight of potential dynamic objects and to improve global semantic information for precise identification of potential dynamic objects. The method of multi-view geometry is applied to judge and remove the dynamic objects. Meanwhile, according to static information in the adjacent frames, the optimal nearest-neighbor pixel-matching method is applied to restore the static background, where the feature points are extracted for pose estimation. With validation in the public dataset TUM (The Entrepreneurial University Dataset) and real scenarios, the experimental results show that the root-mean-square error of the algorithm is reduced by 17.1 % compared with dynamic SLAM (DynaSLAM), which shows better map composition capability.

VID-SLAM: A New Visual Inertial SLAM algorithm Coupling an RGB-D camera and IMU Based on Adaptive Point and Line Features

VID-SLAM: A New Visual Inertial SLAM algorithm Coupling an RGB-D camera and IMU Based on Adaptive Point and Line Features

Visual SLAM Algorithm Based on Weighted Static in Dynamic Environment

Visual SLAM Algorithm Based on Weighted Static in Dynamic Environment

A Semantics-Guided Visual Simultaneous Localization and Mapping with U-Net for Complex Dynamic Indoor Environments

Traditional simultaneous localization and mapping (SLAM) system tends to operate in small-area static environments, and its performance might degrade when moving objects appear in a highly dynamic environment. To address this issue, this paper proposes a dynamic object-aware visual SLAM algorithm specifically designed for dynamic indoor environments. The proposed method leverages a semantic segmentation architecture called U-Net, which is utilized in the tracking thread to detect potentially moving targets. The resulting output of semantic segmentation is tightly coupled with the geometric information extracted from the corresponding SLAM system, thus associating the feature points captured by images with the potentially moving targets. Finally, filtering out the moving feature points can greatly enhance localization accuracy in dynamic indoor environments. Quantitative and qualitative experiments were carried out on both the Technical University of Munich (TUM) public dataset and the real scenario dataset to verify the effectiveness and robustness of the proposed method. Results demonstrate that the semantics-guided approach significantly outperforms the ORB SLAM2 framework in dynamic indoor environments, which is crucial for improving the robustness and reliability of the SLAM system.

A Lightweight Visual Simultaneous Localization and Mapping Method with a High Precision in Dynamic Scenes.

Currently, in most traditional VSLAM (visual SLAM) systems, static assumptions result in a low accuracy in dynamic environments, or result in a new and higher level of accuracy but at the cost of sacrificing the real-time property. In highly dynamic scenes, balancing a high accuracy and a low computational cost has become a pivotal requirement for VSLAM systems. This paper proposes a new VSLAM system, balancing the competitive demands between positioning accuracy and computational complexity and thereby further improving the overall system properties. From the perspective of accuracy, the system applies an improved lightweight target detection network to quickly detect dynamic feature points while extracting feature points at the front end of the system, and only feature points of static targets are applied for frame matching. Meanwhile, the attention mechanism is integrated into the target detection network to continuously and accurately capture dynamic factors to cope with more complex dynamic environments. From the perspective of computational expense, the lightweight network Ghostnet module is applied as the backbone network of the target detection network YOLOv5s, significantly reducing the number of model parameters and improving the overall inference speed of the algorithm. Experimental results on the TUM dynamic dataset indicate that in contrast with the ORB-SLAM3 system, the pose estimation accuracy of the system improved by 84.04%. In contrast with dynamic SLAM systems such as DS-SLAM and DVO SLAM, the system has a significantly improved positioning accuracy. In contrast with other VSLAM algorithms based on deep learning, the system has superior real-time properties while maintaining a similar accuracy index.

YG-SLAM: GPU-Accelerated RGBD-SLAM Using YOLOv5 in a Dynamic Environment

Traditional simultaneous localization and mapping (SLAM) performs well in a static environment; however, with the abrupt increase of dynamic points in dynamic environments, the algorithm is influenced by a lot of meaningless information, leading to low precision and poor robustness in pose estimation. To tackle this problem, a new visual SLAM algorithm of dynamic scenes named YG-SLAM is proposed, which creates an independent dynamic-object-detection thread and adds a dynamic-feature-point elimination step in the tracking thread. The YOLOv5 algorithm is introduced in the dynamic-object-detection thread for target recognition and deployed on the GPU to speed up image frame identification. The optic-flow approach employs an optic flow to monitor feature points and helps to remove the dynamic points in different dynamic objects based on the varying speeds of pixel movement. While combined with the antecedent information of object detection, the system can eliminate dynamic feature points under various conditions. Validation is conducted in both TUM and KITTI datasets, and the results illustrate that YG-SLAM can achieve a higher accuracy in dynamic indoor environments, with the maximum accuracy augmented from 0.277 m to 0.014 m. Meanwhile, YG-SLAM requires less processing time than other dynamic-scene SLAM algorithms, indicating its positioning priority in dynamic situations.

Semantic information based solution for visual SLAM in dynamic environment

In recent years, the utilization of visual SLAM with a camera as a sensor has become increasingly widespread, particularly in the context of rapidly developing artificial intelligence such as mobile robots, VR, and AR. This approach is favored due to its affordability, lightweight design, ability to capture comprehensive information, and other advantages. The traditional slam technology has achieved a very mature effect with the static scene as the assumption condition, and the classic representatives are LSD-SLAM and ORB-SLAM which will be introduced in the following. However, since dynamic scenarios are unavoidable in the real world, overcoming the influence of dynamic objects becomes a challenge if researchers want to move forward with more applications. At present, under the dynamic environment of the visual slam algorithm faces positioning problems of low accuracy and poor robustness. To address the challenges posed by dynamic objects in a scene, many researchers are incorporating deep learning techniques and exploring the use of reference semantic information to collaboratively resolve the issue. This paper reviews this and summarizes the development process and important algorithms.

Research on visual SLAM for dynamic scenes

Simultaneous Localization and Mapping (SLAM) technology has many applications, such as intelligent robots, autonomous driving, and smart cities. It is a key technology for achieving autonomous robot navigation and environmental awareness. Robots need to perceive the surrounding environment in real-time and construct maps while conducting autonomous localization and path planning, which involves algorithms of SLAM technology. This article aims to introduce the research on visual SLAM algorithms in dynamic scenes. This article will first introduce the traditional SLAM algorithm, its research status, and its limitations. Next, introduce the research on the visual SLAM algorithm in dynamic scenes. Finally, the current visual SLAM algorithm's problems and future research directions were discussed. The research in this article will have significant value for the research and application of autonomous robot navigation, environmental perception, and smart cities.

Visual SLAM algorithm based on semantic information and geometric consistency1

Localization through intricate traffic scenes poses challenges due to their dynamic, light-variable, and low-textured nature. Existing visual Simultaneous Localization and Mapping (SLAM) methods, which are based on static and texture-rich assumptions, struggle with drift and tracking failures in such complex environments. To address this, we propose a visual SLAM algorithm based on semantic information and geometric consistency in order to solve the above issues and further realize autonomous driving applications in road environments. In dynamic traffic scenes, we employ an object detection network to identify moving objects and further classify them based on geometric consistency as dynamic objects or potential dynamic objects. This method permits us to preserve more reliable static feature points. In low-texture environments, we propose a method that employs key object categories and geometric parameters of static scene objects for object matching between consecutive frames, effectively resolving the problem of tracking failure in such scenarios. We conducted experiments on the KITTI and ApolloScape datasets for autonomous driving and compared them to current representative algorithms. The results indicate that in the dynamic environment of the KITTI dataset, our algorithm improves the compared metrics by an average of 29.68% . In the static environment of the KITTI dataset, our algorithm’s performance is comparable to that of the other compared algorithms. In the complex traffic scenario R11R003 from the ApolloScape dataset, our algorithm improves the compared metrics by an average of 25.27% . These results establish the algorithm’s exceptional localization accuracy in dynamic environments and its robust localization capabilities in environments with low texture. It provides development and support for the implementation of autonomous driving technology applications.

FirebotSLAM: Thermal SLAM to Increase Situational Awareness in Smoke-Filled Environments.

Operating in extreme environments is often challenging due to the lack of perceptual knowledge. During fire incidents in large buildings, the extreme levels of smoke can seriously impede a firefighter's vision, potentially leading to severe material damage and loss of life. To increase the safety of firefighters, research is conducted in collaboration with Dutch fire departments into the usability of Unmanned Ground Vehicles to increase situational awareness in hazardous environments. This paper proposes FirebotSLAM, the first algorithm capable of coherently computing a robot's odometry while creating a comprehensible 3D map solely using the information extracted from thermal images. The literature showed that the most challenging aspect of thermal Simultaneous Localization and Mapping (SLAM) is the extraction of robust features in thermal images. Therefore, a practical benchmark of feature extraction and description methods was performed on datasets recorded during a fire incident. The best-performing combination of extractor and descriptor is then implemented into a state-of-the-art visual SLAM algorithm. As a result, FirebotSLAM is the first thermal odometry algorithm able to perform global trajectory optimization by detecting loop closures. Finally, FirebotSLAM is the first thermal SLAM algorithm to be tested in a fiery environment to validate its applicability in an operational scenario.

Visual Simultaneous Localization and Mapping for Sewer Pipe Networks Leveraging Cylindrical Regularity

This work proposes a novel visual Simultaneous Localisation and Mapping (vSLAM) approach for robots in sewer pipe networks. One problem of vSLAM in pipes is that the scale drifts and accuracy degrades. We propose the use of structural information to mitigate this problem via cylindrical regularity. The main novelty consists of an approach for cylinder detection that is more robust than previous methods in non-smooth sewer pipe environments. Cylindrical regularity is then incorporated into both local bundle adjustment and pose graph optimisation. The approach adopts a minimal cylinder representation with only five parameters, avoiding constraints during the optimisation in vSLAM. A further novelty is that the estimated cylinder is part of the scale drift estimation, which enables a correction to the translation estimate and this further improves the accuracy. The approach, termed <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Cylindrical Regularity ORB-SLAM</i> (CRORB), is benchmarked and compared to leading visual SLAM algorithms ORB-SLAM2 and direct sparse odometry (DSO), as well as a vSLAM algorithm with cylindrical regularity developed for gas pipes, using real sewer pipe data and synthetic data generated with the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Gazebo</i> modelling software. The results demonstrate that CRORB improves substantially over the competitors, with a reduction of approximately 70% in error on real data.

VILO SLAM: Tightly Coupled Binocular Vision–Inertia SLAM Combined with LiDAR

For the existing visual–inertial SLAM algorithm, when the robot is moving at a constant speed or purely rotating and encounters scenes with insufficient visual features, problems of low accuracy and poor robustness arise. Aiming to solve the problems of low accuracy and robustness of the visual inertial SLAM algorithm, a tightly coupled vision-IMU-2D lidar odometry (VILO) algorithm is proposed. Firstly, low-cost 2D lidar observations and visual–inertial observations are fused in a tightly coupled manner. Secondly, the low-cost 2D lidar odometry model is used to derive the Jacobian matrix of the lidar residual with respect to the state variable to be estimated, and the residual constraint equation of the vision-IMU-2D lidar is constructed. Thirdly, the nonlinear solution method is used to obtain the optimal robot pose, which solves the problem of how to fuse 2D lidar observations with visual–inertial information in a tightly coupled manner. The results show that the algorithm still has reliable pose-estimation accuracy and robustness in many special environments, and the position error and yaw angle error are greatly reduced. Our research improves the accuracy and robustness of the multi-sensor fusion SLAM algorithm.

Increasing the localization accuracy of visual SLAM with semantic segmentation and motion consistency detection in dynamic scenes1

Visual Simultaneous Localization and Mapping (SLAM) is one of the key technologies for intelligent mobile robots. However, most of the existing SLAM algorithms have low localization accuracy in dynamic scenes. Therefore, a visual SLAM algorithm combining semantic segmentation and motion consistency detection is proposed. Firstly, the RGB images are segmented by SegNet network, the prior semantic information is established and the feature points of high-dynamic objects are removed; Secondly, motion consistency detection is carried out, the fundamental matrix is calculated by the improved Random Sample Consistency (RANSAC) algorithm, the abnormal feature points are output by the epipolar geometry method, and the feature points of low-dynamic objects are eliminated by combining the prior semantic information. Thirdly, the static feature points are used for pose estimation. Finally, the proposed algorithm is tested on the TUM dataset, the algorithm in this paper reduces the average RMSE of ORB-SLAM2 by 93.99% in highly dynamic scenes, which show that the algorithm can effectively improve the localization accuracy of the visual SLAM system in dynamic scenes.

AHY-SLAM: Toward Faster and More Accurate Visual SLAM in Dynamic Scenes Using Homogenized Feature Extraction and Object Detection Method.

At present, SLAM is widely used in all kinds of dynamic scenes. It is difficult to distinguish dynamic targets in scenes using traditional visual SLAM. In the matching process, dynamic points are incorrectly added to the pose calculation with the camera, resulting in low precision and poor robustness in the pose estimation. This paper proposes a new dynamic scene visual SLAM algorithm based on adaptive threshold homogenized feature extraction and YOLOv5 object detection, named AHY-SLAM. This new method adds three new modules based on ORB-SLAM2: a keyframe selection module, a threshold calculation module, and an object detection module. The optical flow method is used to screen keyframes for each frame input in AHY-SLAM. An adaptive threshold is used to extract feature points for keyframes, and dynamic points are eliminated with YOLOv5. Compared with ORB-SLAM2, AHY-SLAM has significantly improved pose estimation accuracy over multiple dynamic scene sequences in the TUM open dataset, and the absolute pose estimation accuracy can be increased by up to 97%. Compared with other dynamic scene SLAM algorithms, the speed of AHY-SLAM is also significantly improved under a guarantee of acceptable accuracy.