Relative Pose Error Research Articles

Advancing real-world visual SLAM: Integrating adaptive segmentation with dynamic object detection for enhanced environmental perception

The field of Visual Simultaneous Localization and Mapping (VSLAM) has long grappled with the limitations imposed by static environmental assumptions, a challenge that becomes increasingly pronounced in dynamic settings. Addressing this gap, our study introduces a pioneering technique, Accurate and Dynamic Reliable SLAM (YoloV8-SLAM), which represents a paradigm shift in VSLAM technology. This novel approach combines the prowess of dynamic object detection with advanced multi-view geometry, thereby elevating the system’s accuracy, robustness, and efficiency in varied dynamic scenarios. At the core of YoloV8-SLAM is an innovative adaptive segmentation process that skilfully merges geometric motion details from successive frames. This integration not only isolates dynamic objects with heightened precision but also ensures consistent mapping accuracy. The system employs the latest YoloV8 model, synergizing it with an Enhanced Multi-View Geometry algorithm. This combination is adept at handling a spectrum of dynamic conditions, further bolstered by a precise point selection mechanism that swiftly retrieves motion information, crucial for real-time applications. Our extensive evaluations demonstrate that YoloV8-SLAM significantly outperforms existing methods such as including DynaSLAM (Bescos et al., 2018), DM-SLAM (Cheng et al., 2020), RDS-SLAM (Liu et al., 2021), Blitz-SLAM (Fan et al., 2022), and OVD-SLAM (He et al., 2023). Notably, it achieves a 38–45% reduction in Absolute Trajectory Error and a 66–72% decrease in Relative Pose Error, underscoring its superior performance. The breakthrough of YoloV8-SLAM lies in its seamless amalgamation of cutting-edge machine learning with robust geometric principles. This fusion not only facilitates effective dynamic segmentation but also ensures stable structural reconstruction, even in highly dynamic environments. This work marks a substantial advancement in SLAM technology, particularly in addressing the complexities of dynamic and unrestricted settings. YoloV8-SLAM sets a new benchmark in the field and paves the way for future explorations in dynamic SLAM solutions, promising to revolutionize applications requiring high fidelity in real-world, ever-changing environments.

GY-SLAM: A Dense Semantic SLAM System for Plant Factory Transport Robots.

Simultaneous Localization and Mapping (SLAM), as one of the core technologies in intelligent robotics, has gained substantial attention in recent years. Addressing the limitations of SLAM systems in dynamic environments, this research proposes a system specifically designed for plant factory transportation environments, named GY-SLAM. GY-SLAM incorporates a lightweight target detection network, GY, based on YOLOv5, which utilizes GhostNet as the backbone network. This integration is further enhanced with CoordConv coordinate convolution, CARAFE up-sampling operators, and the SE attention mechanism, leading to simultaneous improvements in detection accuracy and model complexity reduction. While mAP@0.5 increased by 0.514% to 95.364, the model simultaneously reduced the number of parameters by 43.976%, computational cost by 46.488%, and model size by 41.752%. Additionally, the system constructs pure static octree maps and grid maps. Tests conducted on the TUM dataset and a proprietary dataset demonstrate that GY-SLAM significantly outperforms ORB-SLAM3 in dynamic scenarios in terms of system localization accuracy and robustness. It shows a remarkable 92.59% improvement in RMSE for Absolute Trajectory Error (ATE), along with a 93.11% improvement in RMSE for the translational drift of Relative Pose Error (RPE) and a 92.89% improvement in RMSE for the rotational drift of RPE. Compared to YOLOv5s, the GY model brings a 41.5944% improvement in detection speed and a 17.7975% increase in SLAM operation speed to the system, indicating strong competitiveness and real-time capabilities. These results validate the effectiveness of GY-SLAM in dynamic environments and provide substantial support for the automation of logistics tasks by robots in specific contexts.

ARD-SLAM: Accurate and robust dynamic SLAM using dynamic object identification and improved multi-view geometrical approaches

In the evolving landscape of autonomous navigation, traditional Visual Simultaneous Localization and Mapping (SLAM) systems often encounter challenges in dynamic environments, primarily due to their reliance on assumptions of static surroundings. In response to these limitations, we introduce ARD-SLAM, a groundbreaking approach to dynamic SLAM that innovatively combines global dense optical tracking with sophisticated geometric methodologies. The core innovation of ARD-SLAM lies in its dynamic object identification technique, which harmoniously integrates geometric motion information with prospective motion data. This integration facilitates effective segmentation of moving objects, thereby substantially diminishing their impact on camera ego-motion estimation. ARD-SLAM is further enhanced by an advanced multi-view geometry method that emphasizes the selection of well-matched feature points. This approach is instrumental in efficiently managing dynamic scenarios while also reducing computational load. Rigorous testing on the TUM RGB-D and Bonn RGB-D benchmark datasets has established ARD-SLAM's superiority over established techniques like ORB-SLAM2/3, DynaSLAM, SD-SLAM, DGS-SLAM, and OVD-SLAM. Notably, ARD-SLAM achieves a substantial average reduction in Absolute Trajectory Error (ATE) by 86.1% and in Relative Pose Error (RPE) by 88.0% compared to ORB-SLAM3. The results from the Bonn RGB-D Dataset further underscore ARD-SLAM's effectiveness. Compared to other SLAM methods, ARD-SLAM shows remarkable improvements: 37.8% and 66.4% over DynaSLAM, 41.2% and 73.1% over DGS-SLAM, and 48.9% and 79.7% over OVD-SLAM in ATE and RPE metrics, respectively. This robust performance in dynamically changing environments solidifies ARD-SLAM as a significant advancement in SLAM technology, offering a more precise and adaptable solution for the complex challenges of real-world autonomous navigation.

A Measure of Semantic Class Difference of Point Reprojection Pairs in Camera Pose Estimation

In this paper, we propose a new measure that evaluates the semantic errors of camera poses in visual odometry (VO) and visual simultaneous localization and mapping (VSLAM). Traditionally, VO/VSLAM methods have used photometric images to estimate camera poses, but they suffer from varying illumination and viewpoint changes. Thus, methods using semantic images have been an alternative to increase consistency, as semantic information has shown its robustness even in hostile environments. Our measure compares semantic classes of map point reprojection pairs between images to improve the camera pose estimation accuracy in VO/VSLAM. To evaluate the difference between semantic classes, we adopt the normalized information distance (NID) from information theory. Furthermore, we suggest a weight parameter to balance the existing error of VO/VSLAM with the semantic error introduced by our approach. Our experimental results, obtained from the VKITTI and KITTI benchmark datasets, show that the proposed semantic error measure reduces both the relative pose error (RPE) and absolute trajectory error (ATE) of camera pose estimation compared to the existing photometric image-based errors of indirect and direct VO/VSLAM.

DOE: a dynamic object elimination scheme based on geometric and semantic constraints

In this paper, we propose a dynamic object elimination algorithm that combines semantic and geometric constraints to address the problem of visual SLAM being easily affected by dynamic feature points in dynamic environments. This issue leads to the degradation of localisation accuracy and robustness. Firstly, we employ a lightweight YOLO-Tiny network to enhance both detection accuracy and system speed. Secondly, we integrate the YOLO-Tiny network into the ORB-SLAM3 system to extract semantic information from the images and initiate the elimination of dynamic feature points. Subsequently, we augment this approach by incorporating geometric constraints between neighbouring frames to further eliminate dynamic feature points. Then, the former is supplemented by combining the geometric constraints between neighbouring frames to further eliminate dynamic feature points. Experiments on the TUM dataset demonstrate that the algorithm in this paper can improve the Relative Pose Error (RPE) by up to 95.12% and the Absolute Trajectory Error (ATE) by up to 99.01% in high dynamic sequences compared to ORB-SLAM3. The effectiveness of dynamic feature point elimination is evident, leading to significantly improved localisation accuracy.

CONTINUOUS BIM ALIGNMENT FOR MIXED REALITY VISUALISATION

Abstract. Several methods exist that can be used to perform initial alignment of Building information models (BIMs) to the real building for Mixed Reality (MR) applications, such as marker-based or markerless visual methods, but this alignment is susceptible to drift over time. The existing model-based methods that can be used to maintain this alignment have multiple limitations, such as the use of iterative processes and poor performance in environments with either too many or not enough lines. To address these issues, we propose an end-to-end trainable Convolutional Neural Network (CNN) that takes a real and synthetic BIM image pair as input to regress the 6 DoF relative camera pose difference between them directly. By correcting the relative pose error we are able to considerably improve the alignment of the BIM to the real building. Furthermore, the results of our experiments demonstrate good performance in a challenging environment and high resilience to domain shift between synthetic and real images. A high localisation accuracy of approximately 7.0 cm and 0.9° is achieved which indicates the method can be used to reduce the camera tracking drift for MR applications.

Semantic visual simultaneous localization and mapping (SLAM) using deep learning for dynamic scenes.

Simultaneous localization and mapping (SLAM) is a fundamental problem in robotics and computer vision. It involves the task of a robot or an autonomous system navigating an unknown environment, simultaneously creating a map of the surroundings, and accurately estimating its position within that map. While significant progress has been made in SLAM over the years, challenges still need to be addressed. One prominent issue is robustness and accuracy in dynamic environments, which can cause uncertainties and errors in the estimation process. Traditional methods using temporal information to differentiate static and dynamic objects have limitations in accuracy and applicability. Nowadays, many research trends have leaned towards utilizing deep learning-based methods which leverage the capabilities to handle dynamic objects, semantic segmentation, and motion estimation, aiming to improve accuracy and adaptability in complex scenes. This article proposed an approach to enhance monocular visual odometry's robustness and precision in dynamic environments. An enhanced algorithm using the semantic segmentation algorithm DeeplabV3+ is used to identify dynamic objects in the image and then apply the motion consistency check to remove feature points belonging to dynamic objects. The remaining static feature points are then used for feature matching and pose estimation based on ORB-SLAM2 using the Technical University of Munich (TUM) dataset. Experimental results show that our method outperforms traditional visual odometry methods in accuracy and robustness, especially in dynamic environments. By eliminating the influence of moving objects, our method improves the accuracy and robustness of visual odometry in dynamic environments. Compared to the traditional ORB-SLAM2, the results show that the system significantly reduces the absolute trajectory error and the relative pose error in dynamic scenes. Our approach has significantly improved the accuracy and robustness of the SLAM system's pose estimation.

A 3D Lidar SLAM System Based on Semantic Segmentation for Rubber-Tapping Robot

Simultaneous localization and mapping (SLAM) in rubber plantations is a challenging task for rubber-tapping robots. Due to the long-term stability of tree trunks in rubber plantations, a SLAM system based on semantic segmentation, called Se-LOAM, is proposed in this work. The 3D lidar point cloud datasets of trunks collected in rubber plantations of Hainan University are used to train the semantic model, and the model is used to extract features of trunk point clouds. After clustering the trunk point clouds, each single rubber tree instance is segmented based on the Viterbi algorithm. The point clouds of tree instances are fitted to the cylindrical trunk models for semantic cluster association and positional estimation, which are used for lidar odometry and mapping. The experimental results show that the present SLAM system is accurate in establishing online mapping, and the location of the trunk in the map is clearer. Specifically, the average relative pose error is 0.02 m, which is better than the positioning performance of LOAM and LeGO-LOAM. The average error of estimating the diameter at breast height (DBH) is 0.57 cm, and it only takes 401.4 kB to store a map of the area of approximately 500 m2, which is about 10% less than other classic methods. Therefore, Se-LOAM can meet the requirements of online mapping, providing a robust SLAM method for rubber-tapping robots.

NDT‐6D for color registration in agri‐robotic applications

AbstractRegistration of point cloud data containing both depth and color information is critical for a variety of applications, including in‐field robotic plant manipulation, crop growth modeling, and autonomous navigation. However, current state‐of‐the‐art registration methods often fail in challenging agricultural field conditions due to factors such as occlusions, plant density, and variable illumination. To address these issues, we propose the NDT‐6D registration method, which is a color‐based variation of the Normal Distribution Transform (NDT) registration approach for point clouds. Our method computes correspondences between pointclouds using both geometric and color information and minimizes the distance between these correspondences using only the three‐dimensional (3D) geometric dimensions. We evaluate the method using the GRAPES3D data set collected with a commercial‐grade RGB‐D sensor mounted on a mobile platform in a vineyard. Results show that registration methods that only rely on depth information fail to provide quality registration for the tested data set. The proposed color‐based variation outperforms state‐of‐the‐art methods with a root mean square error (RMSE) of 1.1–1.6 cm for NDT‐6D compared with 1.1–2.3 cm for other color‐information‐based methods and 1.2–13.7 cm for noncolor‐information‐based methods. The proposed method is shown to be robust against noises using the TUM RGBD data set by artificially adding noise present in an outdoor scenario. The relative pose error (RPE) increased 14% for our method compared to an increase of 75% for the best‐performing registration method. The obtained average accuracy suggests that the NDT‐6D registration methods can be used for in‐field precision agriculture applications, for example, crop detection, size‐based maturity estimation, and growth modeling.

Semantic closed-loop based visual mapping algorithm for automated valet parking

Simultaneous localization and mapping (SLAM) is one of the core technologies to realize automated valet parking (AVP). Currently, advanced visual feature-based SLAM systems suffer from feature extraction failure and tracking loss due to the constraints of textureless scenes, unclear illumination, and dynamic conditions. To address these problems, this paper proposes a visual SLAM algorithm based on a semantic closed-loop detection algorithm using surround-view cameras and inertial measurement units (IMU) as sensors. The algorithm combines semantic features and the idea of inverse index to improve the traditional keyframes selection methods and the closed-loop detection algorithms, effectively avoiding the tedious and complicated feature point matching and improving the computational efficiency of the computer. Experiments show that the algorithm in this paper achieves better results in terms of precision and recall, absolute trajectory error (ATE), and relative pose error (RPE), and can meet the demand for SLAM and subsequent navigation in indoor parking lots.

Semantic stereo visual SLAM toward outdoor dynamic environments based on ORB-SLAM2

PurposeThe prerequisite for most traditional visual simultaneous localization and mapping (V-SLAM) algorithms is that most objects in the environment should be static or in low-speed locomotion. These algorithms rely on geometric information of the environment and restrict the application scenarios with dynamic objects. Semantic segmentation can be used to extract deep features from images to identify dynamic objects in the real world. Therefore, V-SLAM fused with semantic information can reduce the influence from dynamic objects and achieve higher accuracy. This paper aims to present a new semantic stereo V-SLAM method toward outdoor dynamic environments for more accurate pose estimation.Design/methodology/approachFirst, the Deeplabv3+ semantic segmentation model is adopted to recognize semantic information about dynamic objects in the outdoor scenes. Second, an approach that combines prior knowledge to determine the dynamic hierarchy of moveable objects is proposed, which depends on the pixel movement between frames. Finally, a semantic stereo V-SLAM based on ORB-SLAM2 to calculate accurate trajectory in dynamic environments is presented, which selects corresponding feature points on static regions and eliminates useless feature points on dynamic regions.FindingsThe proposed method is successfully verified on the public data set KITTI and ZED2 self-collected data set in the real world. The proposed V-SLAM system can extract the semantic information and track feature points steadily in dynamic environments. Absolute pose error and relative pose error are used to evaluate the feasibility of the proposed method. Experimental results show significant improvements in root mean square error and standard deviation error on both the KITTI data set and an unmanned aerial vehicle. That indicates this method can be effectively applied to outdoor environments.Originality/valueThe main contribution of this study is that a new semantic stereo V-SLAM method is proposed with greater robustness and stability, which reduces the impact of moving objects in dynamic scenes.

Dynamic Object-Aware Visual Odometry (VO) Estimation Based on Optical Flow Matching

In this work, we propose a new visual odometry (VO) system that exploits the dynamic parts of an image. The key idea of our method is to identify the dynamic parts by combining semantic segmentation and optical flow and to suppress the dynamic parts in the process of VO estimation. First, movable objects are detected using the semantic segmentation. If an object contains many pixels of inconsistent optical flow, the object is considered as dynamic and merged with other dynamic objects to create a dynamic mask. Next, the ego-motion of the camera is estimated by using only the remaining static parts of the image and suppressing its dynamic parts. Unlike the other popular deep learning-based VO approaches, our method uses geometric approach based on optimization to achieve high performance. That is, the ego-motion of the camera is obtained by optimizing the correspondences obtained using the dynamic mask and optical flow consistency. Finally, the proposed method is applied to the KITTI Odometry benchmark dataset, and its performance is compared with that of the previous VO methods. Our method achieves an average of 7.67(%) translation error and 2.186 rotation error(°/100m), which imply the performance improvement by 21% and 11% from the state-of-the-art baseline, DF-VO, respectively. In addition, our method yields the RPE (Relative Pose Error) of 0.186m, which is the performance improvement by 52% against ORB-SLAM2, a popular geometry-based method. The experimental results show that the proposed VO method reliably predicts excellent visual odometry in the presence of dynamic objects.

High Precision Calibration for Three-Dimensional Vision-Guided Robot System

Calibration is an important premise for vision-guided robot system, especially for precise industrial operations on curved surfaces. Most of existing methods focus on one or partial union of errors including assembly error of the robot, relative pose error between the vision unit and the robot, or installation error of end effector, which is still challenging in practice. To solve this problem, this article proposes a new calibration method based on error correction matrix by which different manifestations of each error is unified. As the nonlinear equations formed by these matrices cannot be solved analytically, a numerical optimization solution based on Lie algebra is presented. Specifically, the matrices acquisition is formulated as the problem of minimizing the sum of distance deviations between actual and ideal tool center points. This problem is then solved by differentiating these matrices with a form of left multiplicative perturbation. In this way, a high precision joint calibration with multisource errors is achieved. The proposed method is verified by simulations and experiments.

Development of RGB-D simultaneous localization and mapping in dynamic environments based on motion removal and dense map reconstruction

The state-of-art simultaneous localization and mapping (SLAM) methods have been demonstrated of satisfactory results in static environments. However, most of the existing methods cannot be directly applied in dynamic environments to account for moving objects. To solve the practical problem of SLAM in dynamic environments, an RGB-D SLAM method is developed based on moving objects removal and dense map reconstruction. Specifically, deep learning-based instance segmentation is performed to obtain the semantic information which is furthermore combined with multiview geometry theory to detect possible moving objects. Kalman filter and feature fusion algorithms are further used for moving objects tracking to improve the detection accuracy. Then the camera pose is estimated using the static feature points after motion removal to realize reliable localization in dynamic environments. Out of concern for the infeasibility of navigation and perception with sparse feature maps, each-single-view of dense map is constructed using the RGB image and the depth map collected by the RGB-D camera after motion removal. The dense environmental map is finally reconstructed through the registration of point cloud of selected key image frames. Experiments are conducted on public datasets and real scene data to evaluate the performance of the proposed method. Compared with the state-of-art method ORB-SLAM2, the absolute trajectory error and the relative pose error obtained by the proposed method are significantly reduced by at least 91%, and the localization accuracy reaches 0.001 m.

A Benchmark Comparison of Four Off-the-Shelf Proprietary Visual–Inertial Odometry Systems

Commercial visual-inertial odometry (VIO) systems have been gaining attention as cost-effective, off-the-shelf, six-degree-of-freedom (6-DoF) ego-motion-tracking sensors for estimating accurate and consistent camera pose data, in addition to their ability to operate without external localization from motion capture or global positioning systems. It is unclear from existing results, however, which commercial VIO platforms are the most stable, consistent, and accurate in terms of state estimation for indoor and outdoor robotic applications. We assessed four popular proprietary VIO systems (Apple ARKit, Google ARCore, Intel RealSense T265, and Stereolabs ZED 2) through a series of both indoor and outdoor experiments in which we showed their positioning stability, consistency, and accuracy. After evaluating four popular VIO sensors in challenging real-world indoor and outdoor scenarios, Apple ARKit showed the most stable and high accuracy/consistency, and the relative pose error was a drift error of about 0.02 m per second. We present our complete results as a benchmark comparison for the research community.

Finding the best hardware configuration for 2D SLAM in indoor environments via simulation based on Google Cartographer

One of the most challenging topics in robotics is simultaneous localization and mapping (SLAM) in the indoor environments. Due to the fact that Global Navigation Satellite Systems cannot be successfully used in such environments, different data sources are used for this purpose, among others light detection and ranging (LiDARs ), which have advanced from numerous other technologies. Other embedded sensors can be used along with LiDARs to improve SLAM accuracy, e.g. the ones available in the Inertial Measurement Units and wheel odometry sensors. Evaluation of different SLAM algorithms and possible hardware configurations in real environments is time consuming and expensive. In our study, we evaluate the accuracy of mapping and localization (based on Absolute Trajectory Error and Relative Pose Error). Our use case is a robot used for room decontamination. The results for a small room show that for our robot the best hardware configuration consists of three LiDARs 2D, IMU and wheel odometry sensors. On the other hand, for long hallways, a configuration with one LiDAR 3D sensor and IMU works better and more stable. We also described a general approach together with tools and procedures that can be used to find the best sensor setup in simulation.

Research on Underground Coal Mine Map Construction Method Based on LeGO-LOAM Improved Algorithm

The application of intelligent equipment and technologies such as robots and unmanned vehicles is an important part of the construction of intelligent mines, and has become China’s national coal energy development strategy and the consensus of the coal industry. Environment perception and instant positioning is one of the key technologies destined to realize unmanned and autonomous navigation in underground coal mines, and simultaneous location and mapping (SLAM) is an effective method of deploying this key technology. The underground space of a coal mine is long and narrow, the environment is complex and changeable, the structure is complex and irregular, and the lighting is poor. This is a typical unstructured environment, which poses a great challenge to SLAM. This paper summarizes the current research status of underground coal mine map construction based on visual SLAM and Lidar SLAM, and analyzes the defects of the LeGO-LOAM algorithm, such as loopback detection errors or omissions. We use SegMatch to improve the loopback detection module of LeGO-LOAM, use the iterative closest point (ICP) algorithm to optimize the global map, then propose an improved SLAM algorithm, namely LeGO-LOAM-SM, and describe its principle and implementation. The performance of the LeGO-LOAM-SM was also tested using the KITTI dataset 00 sequence and SLAM experimental data collected in two coal mine underground simulation scenarios, and the performance indexes such as the map construction effect, trajectory overlap and length deviation, absolute trajectory error (ATE), and relative pose error (RPE) were analyzed. The results show that the map constructed by LeGO-LOAM-SM is clearer, has a better loopback effect, the estimated trajectory is smoother and more accurate, and the translation and rotation accuracy is improved by approximately 5%. This can construct more accurate point cloud map and low drift position estimation, which verifies the effectiveness and accuracy of the improved algorithm. Finally, to satisfy the navigation requirements, the construction method of a two-dimensional occupancy grid map was studied, and the underground coal mine simulation environment test was carried out. The results show that the constructed raster map can effectively filter out outlier noise such as dynamic obstacles, has a mapping accuracy of 0.01 m, and the required storage space compared with the point cloud map is reduced by three orders of magnitude. The research results enrich the SLAM algorithm and implementation in unstructured environments such as underground coal mines, and help to solve the problems of environment perception, real-time positioning, and the navigation of coal mine robots and unmanned vehicles.

Research on docking ring pose estimation method based on point cloud grayscale image

The star-arrow docking ring is a ubiquitous structure on space vehicles and is often used as a feature component for pose measurement and capture in non-cooperative space capture missions. When the capture task is performed at a close distance of 0.5 m, due to the high radiation and high light intensity of the space and the coating of the satellite surface with thermal control multilayer materials, the lighting environment is complex, and the grayscale image processing is difficult. The point cloud grayscale image contour feature localization detection method converts the point cloud image into a grayscale image and then estimates the relative pose of the docking ring. Aiming at the problem of fast pose estimation in the case of relative motion, the algorithm adds a fast positioning module of frame image feature area based on the motion model on the original basis, which effectively reduces the feature retrieval area and improves the real-time performance of the algorithm. It is verified by physical simulation experiments. The results show that the method can effectively reduce the influence of stray light and eliminate noise contours. The pose estimation results of the docking ring are accurate and stable, the relative position calculation error is less than 1.5 mm, and the relative pose error is less than 0.45°. The frequency is better than 2 Hz, which meets the needs of real-time capture by space robotic arms.

Balancing the Budget: Feature Selection and Tracking for Multi-Camera Visual-Inertial Odometry

We present a multi-camera visual-inertial odometry system based on factor graph optimization which estimates motion by using all cameras simultaneously while retaining a fixed overall feature budget. We focus on motion tracking in challenging environments, such as narrow corridors, dark spaces with aggressive motions, and abrupt lighting changes. These scenarios cause traditional monocular or stereo odometry to fail. While tracking motion with extra cameras should theoretically prevent failures, it leads to additional complexity and computational burden. To overcome these challenges, we introduce two novel methods to improve multi-camera feature tracking. First, instead of tracking features separately in each camera, we track features continuously as they move from one camera to another. This increases accuracy and achieves a more compact factor graph representation. Second, we select a fixed budget of tracked features across the cameras to reduce back-end optimization time. We have found that using a smaller set of informative features can maintain the same tracking accuracy. Our proposed method was extensively tested using a hardware-synchronized device consisting of an IMU and four cameras (a front stereo pair and two lateral) in scenarios including: an underground mine, large open spaces, and building interiors with narrow stairs and corridors. Compared to stereo-only state-of-the-art visual-inertial odometry methods, our approach reduces the drift rate, relative pose error, by up to 80% in translation and 39% in rotation.

Weighted triplet loss based on deep neural networks for loop closure detection in VSLAM

Smart living is an emerging technology that has attracted a lot of attention all around the world. As a key technology of smart space, which is the principal part of smart living, the SLAM system has effectively expanded the ability of space intelligent robots to explore unknown environments. Loop closure detection is an important part of SLAM system and plays a very important role in eliminating cumulative errors. The SLAM system without loop closure detection is degraded to an odometer. The state estimation solely relying on an odometer will be seriously deviated in the long-term and large-scale navigation and positioning. This paper proposes a metric learning method that uses deep neural networks for loop closure detection based on triplet loss. The map points obtained by metric learning are fused with all map points in the current keyframe, and the map points that do not meet the filtering conditions are eliminated. Based on the Batch Hard Triplet loss, the weighted triplet loss function avoids suboptimal convergence in the learning process by applying weighted value constraints. At the same time, considering that fixed boundary parameters cannot be well adapted to the diversity of scales between different samples, we use the semantic similarity of anchor samples and negative samples to redefine boundary parameters. Finally, a SLAM system based on metric learning is constructed, and the SLAM dataset TUM and KITTI are used to evaluate the proposed model’s accuracy rate and recall rate. The scene features in this method are extracted automatically through neural networks instead of being artificially set. Finally, a high-precision closed-loop detection method based on weight adaptive triple loss is effectively realised through the closed-loop detection experiment. The minimum relative pose error is 0.00048 m, which is 15.8% less than that of the closed-loop detection algorithm based on the word bag model.