Simultaneous Localization And Mapping Research Articles

Development of a combined harvester navigation control system based on visual simultaneous localization and mapping-inertial guidance fusion

Recently, the existing unmanned systems of combine harvesters mostly adopts satellite navigation scheme, lacking real-time observation of harvesting adjustment. To improve the operational efficiency of combine harvester assisted navigation operation, this paper designs a combine harvester navigation control system based on vision simultaneous localization and mapping (SLAM)-inertial guidance fusion. The system acquires field image information and extracts the crop boundary line as the navigation datum by binocular camera. First, the system acquires field image information through binocular camera and extracts the crop boundary line as the navigation datum. Second, fusing camera and inertial guidance information to obtain the real-time relative position of a combine harvester. Third, constrained optimization of image and inertial guidance information is achieved through a sliding window optimization method based on tightly coupled nonlinear optimization. Finally, obtain the position of the combine harvester relative to the navigation datum line, and output a signal to the steering mechanism to realize the combine harvester in the field intelligent positioning navigation control. The system consists of binocular camera, inertial measurement unit, motorized steering wheel, monitor display, angle sensor and microcontroller. During field testing, the system underwent repetitive harvesting trials over a distance of 25 m.. The testing machine performs field operations at a speed of 0.9-1.5 m/s, with an average lateral deviation range of 2.21-8.62 cm, a standard deviation range of 0.13-4.21 cm and an average cutting rate range of 92.2%-96.0%, achieving the expected harvesting effect.

Comprehensive Performance Evaluation between Visual SLAM and LiDAR SLAM for Mobile Robots: Theories and Experiments

SLAM (Simultaneous Localization and Mapping), primarily relying on camera or LiDAR (Light Detection and Ranging) sensors, plays a crucial role in robotics for localization and environmental reconstruction. This paper assesses the performance of two leading methods, namely ORB-SLAM3 and SC-LeGO-LOAM, focusing on localization and mapping in both indoor and outdoor environments. The evaluation employs artificial and cost-effective datasets incorporating data from a 3D LiDAR and an RGB-D (color and depth) camera. A practical approach is introduced for calculating ground-truth trajectories and during benchmarking, reconstruction maps based on ground truth are established. To assess the performance, ATE and RPE are utilized to evaluate the accuracy of localization; standard deviation is employed to compare the stability during the localization process for different methods. While both algorithms exhibit satisfactory positioning accuracy, their performance is suboptimal in scenarios with inadequate textures. Furthermore, 3D reconstruction maps established by the two approaches are also provided for direct observation of their differences and the limitations encountered during map construction. Moreover, the research includes a comprehensive comparison of computational performance metrics, encompassing Central Processing Unit (CPU) utilization, memory usage, and an in-depth analysis. This evaluation revealed that Visual SLAM requires more CPU resources than LiDAR SLAM, primarily due to additional data storage requirements, emphasizing the impact of environmental factors on resource requirements. In conclusion, LiDAR SLAM is more suitable for the outdoors due to its comprehensive nature, while Visual SLAM excels indoors, compensating for sparse aspects in LiDAR SLAM. To facilitate further research, a technical guide was also provided for the researchers in related fields.

Tracking and co-location of global point clouds for large-area indoor environments

Extended reality (XR) experiences are on the verge of becoming widely adopted in diverse application domains. An essential part of the technology is accurate tracking and localization of the headset to create an immersive experience. A subset of the applications require perfect co-location between the real and the virtual world, where virtual objects are aligned with real-world counterparts. Current headsets support co-location for small areas, but suffer from drift when scaling up to larger ones such as buildings or factories. This paper proposes tools and solutions for this challenge by splitting up the simultaneous localization and mapping (SLAM) into separate mapping and localization stages. In the pre-processing stage, a feature map is built for the entire tracking area. A global optimizer is applied to correct the deformations caused by drift, guided by a sparse set of ground truth markers in the point cloud of a laser scan. Optionally, further refinement is applied by matching features between the ground truth keyframe images and their rendered-out SLAM estimates of the point cloud. In the second, real-time stage, the rectified feature map is used to perform localization and sensor fusion between the global tracking and the headset. The results show that the approach achieves robust co-location between the virtual and the real 3D environment for large and complex tracking environments.

A new covariance intersection based integrated SLAM framework for 3D outdoor agricultural applications

AbstractThis letter introduces a novel integrated framework for simultaneous localization and mapping (SLAM) tailored for general agricultural applications. The framework combines a cutting‐edge SLAM method, LIO‐SAM, with covariance intersection for sensor fusion. Agricultural robots often operate in unstructured environments with sparse feature points and encounter repeated similar information, such as trees. Therefore, a fusion framework based on 3D SLAM augmented with additional information, such as feature‐independent GPS data, becomes essential. This study proposes an integrated SLAM framework by introducing a convergence strategy based on covariance analysis, incorporating a state‐of‐the‐art 3D SLAM technique. The convergence methods, namely “dynamic weight assignment” and “winner takes all”, are presented alternatively, tailored to seamlessly integrate with the proposed framework. Evaluations using a public dataset and an experiment demonstrate the effectiveness of the approach through numerical analysis and visual representation. The results illustrate that this method surpasses conventional approaches in accurately estimating the robot's position. In the future, this research will focus on automating crop cultivation and harvesting by integrating the proposed system with robot arm control.

Viewpoint-Aware Visibility Scoring for Point Cloud Registration in Loop Closure

Viewpoint-Aware Visibility Scoring for Point Cloud Registration in Loop Closure

PC-IDN: Fast 3D loop closure detection using projection context descriptor and incremental dynamic nodes

3D Loop Closure Detection (LCD) is a crucial technique for robotics, which can correct accumulated errors after long time exploration. Existing 3D LCD methods have been designed for mechanical LiDAR, limiting their performance for various solid-state LiDARs. In this paper, we propose a new LiDAR LCD method, PC-IDN, based on a novel descriptor called Projection Context (PC), and an efficient data structure named Incremental Dynamic Nodes (IDN). The PC considers side-view and describes scenes by leveraging distance and intensity values of the point clouds. Subsequently, the PC descriptors are collected in corresponding nodes for candidate scans selection. To ensure real-time performance, the IDN module dynamically adjusts the visited PCs in the database. The results demonstrate that our method is able to achieve better precision–recall and efficiency performance than existing 3D LCD methods.

Fast loop closure detection using probabilistic integration of pose and appearance similarity

Loop closure detection is a key technique for robots to minimize the accumulated localization and mapping errors after long-time explorations of simultaneous localization and mapping. However, the requirement for efficiency and accuracy performance for mobile robot applications is not well satisfied. In this article, we propose a fast and accurate loop closure detection method by exploiting both pose-based and appearance-based information in a probabilistic manner, inspired by the complementarity between the pose-based and the appearance-based information. Our approach formulates a probability framework combing the pose-based loop closure detection probability and the appearance-based loop closure detection probability. In the proposed framework, the pose-based loop closure detection model is firstly derived from the nonlinear optimization model of odometry. Then the appearance-based loop similarity and the pose-based loop similarity are combined into a joint framework to improve the loop closure detection performance. We implemented our approach using C++ and ROS and thoroughly tested it on the publicly available datasets. The experiments presented in this article suggest that the proposed method can achieve high efficiency and accuracy performance on loop closure detection.

PSMD-SLAM: Panoptic Segmentation-Aided Multi-Sensor Fusion Simultaneous Localization and Mapping in Dynamic Scenes

Multi-sensor fusion is pivotal in augmenting the robustness and precision of simultaneous localization and mapping (SLAM) systems. The LiDAR–visual–inertial approach has been empirically shown to adeptly amalgamate the benefits of these sensors for SLAM across various scenarios. Furthermore, methods of panoptic segmentation have been introduced to deliver pixel-level semantic and instance segmentation data in a single instance. This paper delves deeper into these methodologies, introducing PSMD-SLAM, a novel panoptic segmentation assisted multi-sensor fusion SLAM approach tailored for dynamic environments. Our approach employs both probability propagation-based and PCA-based clustering techniques, supplemented by panoptic segmentation. This is utilized for dynamic object detection and the removal of visual and LiDAR data, respectively. Furthermore, we introduce a module designed for the robust real-time estimation of the 6D pose of dynamic objects. We test our approach on a publicly available dataset and show that PSMD-SLAM outperforms other SLAM algorithms in terms of accuracy and robustness, especially in dynamic environments.

Visual place recognition for autonomous mobile robot navigation using LoFTR and MAGSAC++

Autonomous mobile robots are defined as robotic entities capable of independent movement and intelligent decision-making, relying on their ability to perceive and analyze their surroundings, including objects in their environment. In Simultaneous Localization and Mapping (SLAM) systems, loop closure is often achieved through visual place recognition techniques, where the system compares the current visual input with previously observed scenes to identify matches. In computer vision applications, Speeded-Up Robust Features (SURF) and Scale-Invariant Feature Transform (SIFT) are popular feature extraction algorithms used for such as key point detection, matching, and image registration tasks. The choice of inlier threshold should be based on the specific characteristics of the application and the nature of the images being processed. It often requires experimentation and tuning to find the optimal balance between robustness and accuracy. It Utilizes the pre-trained Local Feature Transformer (LoFTR) and MAGSAC++ estimator to address these drawbacks by employing the number of inliers to determine the similarity between two images for visual place recognition. Our experiment demonstrates that the number of inliers can determine the similarity of locations between two images. Scale variations and translation in location significantly influence the resulting number of inliers. Comparing images from the same location and from different locations yields varying numbers of inliers. The number of inliers significantly influences the similarity of locations. At the same location, the number of inliers is above 150, while at different locations, the number is below 150.

农业环境高精度三维点云地图构建方法研究

In agricultural operation scenarios, the diversity of farmland terrain, crops and other forms, as well as uncertain factors such as weather changes and crop growth during agricultural operation, can have an impact on the construction of high-precision maps. To address these challenges and analyze operational scenarios based on the characteristics of agricultural scenarios, this paper proposes a point cloud map construction algorithm for plant point removal and locatability estimation. Based on the existing Simultaneous Localization and Mapping (SLAM) framework, plant point removal and locatability estimation are improved. Firstly, Red, Green, Blue (RGB) images and Near Infrared (NIR) images are fused to identify and remove plant point clouds, preserving effective inter frame matching information, reducing the impact of dynamic points on inter frame matching, and achieving high front-end motion estimation accuracy. Then, the localization estimation method based on learning is used to determine the motion estimation status and determine whether to execute the backend optimization algorithm. Finally, the back-end optimization algorithm based on Factor graph is designed, and the Factor graph, constraint relationship and optimization function are constructed to optimize the pose of all frames. The optimized map construction algorithm reduces the re projection errors between field roads, paths, and crop rows by 10.27%, 20.76%, and 14.36% compared to before optimization. To verify the actual operational effectiveness of the point cloud map construction algorithm, the hardware part of the multi-sensor information collection system was designed, and sensor internal and external parameter calibration were also carried out. A map information collection vehicle was built and field experiments were conducted. The results showed that the positioning error of the point cloud map construction method proposed in this paper is less than 0.5°, and the cumulative error of 30 m translation is less than 12 cm, which meets the actual operational requirements.

Lp-slam: language-perceptive RGB-D SLAM framework exploiting large language model

With the development of deep learning, a higher level of perception of the environment such as the semantic level can be achieved in the simultaneous localization and mapping (SLAM) domain. However, previous works did not achieve a natural-language level of perception. Therefore, LP-SLAM (Language-Perceptive RGB-D SLAM) is proposed that leverages large language models (LLMs). The texts in the scene can be detected by scene text recognition (STR) and mapped as landmarks with a task-driven selection. A text error correction chain (TECC) is designed with a similarity classification method, a two-stage memory strategy, and a text clustering method. The proposed architecture is designed to deal with the mis-detection and mis-recognition cases of STR and to provide accurate text information to the framework. The proposed framework takes input images and generates a 3D map with sparse point cloud and task-related texts. Finally, a natural user interface (NUI) is designed based on the constructed map and LLM, which gives position instructions based on users’ natural queries. The experimental results validated the proposed TECC design and the overall framework. We publish the virtual dataset with ground truth, as well as the source code for further research. https://github.com/GroupOfLPSLAM/LP_SLAM.

Action-Aware Vision Language Navigation (AAVLN): AI vision system based on cross-modal transformer for understanding and navigating dynamic environments

Visually impaired individuals face great challenges with independently navigating dynamic environments because of their inability to fully comprehend the environment and actions of surrounding people. Conventional navigation approaches like Simultaneous Localization And Mapping (SLAM) rely on complete scanned maps to navigate static, fixed environments. With Vision Language Navigation (VLN), agents can understand semantic information to expand navigation to similar environments. However, both cannot accurately navigate dynamic environments containing human actions. To address this challenge, we propose a novel cross-modal transformer-based Action-Aware VLN system (AAVLN). Our AAVLN Agent Algorithm is trained using Reinforcement Learning in our Environment Simulator. AAVLNs novel cross-modal transformer structure allows the Agent Algorithm to understand natural language instructions and semantic information for navigating dynamic environments and recognizing human actions. For training, we use Reinforcement Learning in our action-based environment simulator. We created it by combining an existing simulator with our novel 3D human action generator. Our experimental results demonstrate the effectiveness of our approach, outperforming current methods on various metrics across challenging benchmarks. Our ablation studies also highlight that we increase dynamic navigation accuracy with our Vision Transformer based human action recognition module and cross-modal encoding. We are currently constructing 3D models of real-world environments, including hospitals and schools, for further training AAVLN. Our project will be combined with Chat-GPT to improve natural language interactions. AAVLN will have numerous applications in robotics, AR, and other computer vision fields.

Bundle adjustment with motion constraints for uncalibrated multi-camera systems at the ground level

Multi-camera systems for structure from motion (SfM) are widely deployed in many mapping applications. Existing solutions assume known rig calibration, synchronized frames among cameras, as well as overlapping field of views (FoVs). In this paper, we derive novel geometric constraints assuming minimal knowns about the multi-camera systems, to benefit low-cost and non-expert use cases where uncalibrated multi-camera systems with non-typical geometry setups present, i.e., no rig calibration, no overlapping FoVs. Assuming that these cameras are co-located and share the same motion of the platform, the proposed constraints utilize the parallelism and length proportionality of motion vectors of these co-located cameras and formulate them as translation constraints into the bundle adjustment (BA). The proposed constraints (called motion constraints) impose a first-order penalty to co-located cameras whose motion speeds and directions between frames do not match. With soft constraints, it can handle loosely synchronized frames (with an error within one second). The proposed constraints are integrated into the BA framework and experimented with different camera setups, i.e., on a group of casually co-located GoPro cameras with no rig calibration, and some with no overlapping views. Our results show that the constraints are extremely effective in improving the reconstruction and pose accuracy for ground motion images: in our self-collected open trajectories without loop closure, the proposed constraints are effective in correcting topographical errors (i.e., trajectory drifts) of the resulting models, and the dense point clouds achieve up to 11.34 m (86.12 %) of mean absolute error (MAE) improvement as compared to reference LiDAR point clouds; our results on KITTI-odometry and KITTI-360 datasets also show an improvement of up to 28.82 m (81.05 %) in terms of the root mean square error (RMSE) of absolute pose error (APE). We expect that the proposed constraints are significant not only as additional geometric constraints for image-based mobile mapping, but also will benefit the broader use of photogrammetry, since it empowers the possibility to harness the traditionally so-called low-quality stereo/multi-camera data (e.g., by non-photogrammetry citizen scientists) into improved 3D products.

Monomeric Esterase: Insights into Cooperative Behavior, Hysteresis/Allokairy.

Herein, we present a novel esterase enzyme, Ade1, isolated from a metagenomic library of Amazonian dark earths soils, demonstrating its broad substrate promiscuity by hydrolyzing ester bonds linked to aliphatic groups. The three-dimensional structure of the enzyme was solved in the presence and absence of substrate (tributyrin), revealing its classification within the α/β-hydrolase superfamily. Despite being a monomeric enzyme, enzymatic assays reveal a cooperative behavior with a sigmoidal profile (initial velocities vs substrate concentrations). Our investigation brings to light the allokairy/hysteresis behavior of Ade1, as evidenced by a transient burst profile during the hydrolysis of substrates such as p-nitrophenyl butyrate and p-nitrophenyl octanoate. Crystal structures of Ade1, coupled with molecular dynamics simulations, unveil the existence of multiple conformational structures within a single molecular state (E̅1). Notably, substrate binding induces a loop closure that traps the substrate in the catalytic site. Upon product release, the cap domain opens simultaneously with structural changes, transitioning the enzyme to a new molecular state (E̅2). This study advances our understanding of hysteresis/allokairy mechanisms, a temporal regulation that appears more pervasive than previously acknowledged and extends its presence to metabolic enzymes. These findings also hold potential implications for addressing human diseases associated with metabolic dysregulation.

ACCURACY ASSESSMENT OF A SLAM-ACQUIRED POINT CLOUD DATA USING A VARIETY OF CLASSIFICATION APPLICATIONS

Abstract. Laser scanning techniques, such as Simultaneous Localization and Mapping (SLAM), produce three-dimensional data representing the real world, which may provide significant information for Building Information Models (BIM). These processes produce 3D point clouds, which require classification before being used in various applications such as structural assessments. However, most widely available software applications for classifying 3D point clouds are proprietary, giving an incomplete depiction of how the data is manipulated and processed. Thus, this research aims to assess the accuracy of the different classification applications in classifying the 3D point cloud data and perform a comparative analysis of the results. Precision, Recall, F1-score, and Accuracy are the evaluation metrics used to assess the classified 3D point cloud data. Results for Precision and Recall show that some of the applications can classify a particular class, the Ground and Building classes. However, the overall performance of the classification method, which is evaluated through the F1-score, produced low values. Results for the F1-score demonstrate that these low values indicate low overall reliability of the classification results despite high values for Accuracy. Based on the conducted experiments, further research is suggested to investigate the effect of increasing dataset size and equalizing class sizes used in classification.

2DLIW-SLAM:2D LiDAR-inertial-wheel odometry with real-time loop closure

Due to budgetary constraints, indoor navigation typically employs two-dimensional (2D) LiDAR rather than 3D LiDAR. However, the utilization of 2D LiDAR in simultaneous localization and mapping (SLAM) frequently encounters challenges related to motion degeneracy, particularly in geometrically similar environments. To address this problem, this paper proposes a robust, accurate, and multi-sensor-fused 2D LiDAR SLAM system specifically designed for indoor mobile robots. To commence, the original LiDAR data undergoes meticulous processing through point and line extraction. Leveraging the distinctive characteristics of indoor environments, line–line constraints are established to complement other sensor data effectively, thereby augmenting the overall robustness and precision of the system. Concurrently, a tightly-coupled front-end is created, integrating data from the 2D LiDAR, inertial measurement unit, and wheel odometry, thus enabling real-time state estimation. Building upon this solid foundation, a novel global feature point matching-based loop closure detection algorithm is proposed. This algorithm proves highly effective in mitigating front-end accumulated errors and ultimately constructs a globally consistent map. The experimental results indicate that our system fully meets real-time requirements. When compared to cartographer, our system not only exhibits lower trajectory errors but also demonstrates stronger robustness, particularly in degeneracy problem. We open source our methods here: https://github.com/LittleDang/2DLIW-SLAM.

3D RECONSTRUCTION OF THE FORT SANTIAGO DUNGEONS USING HANDHELD LASER SCANNING METHOD

Abstract. The preservation of Cultural Heritage sites is an inherent responsibility of the citizens of a country. To this end, developments in digitization serve as a permanent way of preserving a site at its optimum state, and the creation of accurate 3D models becomes a repository of measurements for future reconstruction. Thus, in this study, the Fort Santiago Dungeons, a popular Cultural Heritage site in the Philippines, was 3D-reconstructed using the GeoSLAM ZEB Revo RT, a handheld Light Detection and Ranging (LiDAR) scanner that uses a Simultaneous Localization and Mapping (SLAM) Algorithm. With it, dense point clouds were produced with acceptable results in terms of colorization using the ZEB-CAM accessory, completeness, and measurement noise. This trend extended to quantitative evaluations of the dense point clouds of different formats (LAS and PLY) and colorization settings, as the Root Mean Square Errors (RMSE) against corresponding ground measurements were 0.734cm and 0.739cm for the Non-colorized, and 1.308cm and 1.312cm for Colorized, respectively. Likewise, the derived mesh files gave similar RMSEs with the Non-colorized at 1.044cm (LAS) and 0.999cm (PLY); and with the Colorized at 1.339cm (LAS) and 1.326cm (PLY). Overall, results showed that the mesh file derived from PLY dense point cloud exceeded the other format - as the dungeon's interior was better represented. Nonetheless, even though there were differences in the RMSEs of the measurements, due to factors such as the number of points and the processing itself, all are within the 5cm threshold to be considered useful in the event of reconstruction.

LGD: A fast place recognition method based on the fusion of local and global descriptors

Place recognition, which is also known as loop detection, has been one of the main problems in the field of robot navigation, and it is crucial in realizing high-performance simultaneous localization and mapping (SLAM). Most of the existing methods directly construct local or global descriptors with point clouds, ignoring the offset of different visits in the same place. To overcome the current limitations, this study emphasizes the importance of point cloud features in improving the environment recognition precision and proposes an innovative framework that combines local features that are spatially invariant and prominent descriptive global features as a representation of a laser scanner frame. The feature center of the laser point cloud is obtained by computing local feature points with spatial invariance, which improves the offset between the original point cloud and the feature center. Furthermore, a coarse-to-fine hierarchical recognition strategy is developed to improve the efficiency of place recognition. An evaluation with different offsets and synthesis is conducted on the publicly available KITTI and self-developed CHDloop datasets. The experimental results show that the proposed method can improve time efficiency by more than 18% compared to the SC and ISC, and there are also significant improvements in the recall rate and accuracy in slight lane-change scenarios. Moreover, the proposed method can obtain rich lane-change loops in moderate and large lane-change scenarios where the SC and ISC fail to obtain effective loops. The proposed method can be used as a lightweight and interpretable approach for place recognition, which not only can be quickly deployed in any SLAM framework but can also efficiently handle more complex place recognition issues in urban environments.

Robust loop closure detection and relocalization with semantic-line graph matching constraints in indoor environments

Loop closure detection (LCD) plays an essential role in the Simultaneous Localization and Mapping (SLAM) process, effectively reducing cumulative trajectory errors. However, conventional LCD methods often encounter challenges when dealing with variations in illumination, changes in viewpoint, and environments with weak textures. This is due to their reliance on low-level geometric or image features. To address these issues, we propose a robust LCD method named SL-LCD, which integrates semantic information and line features to fully leverage the semantic content and line attributes within indoor scenes, thereby establishing a reliable feature correspondence between query images and loop closure images. For the retrieval of candidate closed-loop images, we construct a semantic-line-segment topological graph and introduce a graph matching algorithm to perform the LCD task. This approach fully exploits image features and spatial information to achieve closed-loop detection in complex indoor scenes. Furthermore, we present a semantic voxel-based generalized ICP (SVGICP) closed-loop relocalization algorithm tailored for challenging and complex indoor scenes, enhancing the accuracy of closed-loop relocalization in such scenarios. Experimental results demonstrate that the SL-LCD algorithm proposed in this paper surpasses state-of-the-art methods, accurately detecting closed loops, and effectively eliminating trajectory drift.

Slam loop closure detection algorithm based on MSA-SG

Slam loop closure detection algorithm based on MSA-SG