Monocular Simultaneous Localization Research Articles

Reducing Bandwidth Usage in CVSLAM: A Novel Approach to Map Point Selection and Efficient Data Compression

In the field of collaborative visual simultaneous localization and mapping (CVSLAM), efficient data communication poses a significant challenge, particularly in environments with limited bandwidth. To address this issue, we introduce a method aimed at reducing communication consumption. Our approach starts with a strategic culling of map points, aiming at maximizing pose-visibility and expanding spatial diversity to effectively eliminate redundant data in CVSLAM. We achieve this by formulating the problem of maximizing pose-visibility and spatial diversity as a minimum-cost maximum-flow graph optimization problem. Subsequently, we apply finite state entropy encoding for the compression of visual information, further alleviating bandwidth constraints. To verify the proposed method, we implement it within a centralized collaborative monocular simultaneous localization and mapping (SLAM) system. Our approach has been tested on publicly available datasets and in real-world scene. The results show a prominent reduction in bandwidth usage by 49% while maintaining mapping accuracy and without introducing additional latency, confirming its effectiveness in a multi-agent system setting.

Real-Time Estimation of Tree Position, Tree Height, and Tree Diameter at Breast Height Point, Using Smartphones Based on Monocular SLAM

Precisely estimating the position, diameter at breast height (DBH), and height of trees is essential in forest resource inventory. Augmented reality (AR)-based devices help overcome the issue of inconsistent global point cloud data under thick forest canopies with insufficient Global Navigation Satellite System (GNSS) coverage. Although monocular simultaneous localization and mapping (SLAM) is one of the current mainstream systems, there is still no monocular SLAM solution for forest resource inventories, particularly for the precise measurement of inclined trees. We developed a forest plot survey system based on monocular SLAM that utilizes array cameras and Inertial Measurement Unit (IMU) sensors provided by smartphones, combined with augmented reality technology, to achieve a real-time estimation of the position, DBH, and height of trees within forest plots. Our results from the tested plots showed that the tree position estimation is unbiased, with an RMSE of 0.12 m and 0.11 m in the x-axis and y-axis directions, respectively; the DBH estimation bias is −0.17 cm (−0.65%), with an RMSE of 0.83 cm (3.59%), while the height estimation bias is −0.1 m (−0.95%), with an RMSE of 0.99 m (5.38%). This study will be useful in designing an algorithm to estimate the DBH and position of inclined trees using point clouds constrained by sectional planes at the breast height of the trunk, developing an algorithm to estimate the height of inclined trees utilizing the relationship between rays and plane positions, and providing observers with visual measurement results using augmented reality technology, allowing them to judge the accuracy of the estimates intuitively. Clearly, this system has significant potential applications in forest resource management and ecological research.

Enhanced Unmanned Aerial Vehicle Localization in Dynamic Environments Using Monocular Simultaneous Localization and Mapping and Object Tracking

This work proposes an innovative approach to enhance the localization of unmanned aerial vehicles (UAVs) in dynamic environments. The methodology integrates a sophisticated object-tracking algorithm to augment the established simultaneous localization and mapping (ORB-SLAM) framework, utilizing only a monocular camera setup. Moving objects are detected by harnessing the power of YOLOv4, and a specialized Kalman filter is employed for tracking. The algorithm is integrated into the ORB-SLAM framework to improve UAV pose estimation by correcting the impact of moving elements and effectively removing features connected to dynamic elements from the ORB-SLAM process. Finally, the results obtained are recorded using the TUM RGB-D dataset. The results demonstrate that the proposed algorithm can effectively enhance the accuracy of pose estimation and exhibits high accuracy and robustness in real dynamic scenes.

PAL-SLAM2: Visual and visual–inertial monocular SLAM for panoramic annular lens

This paper presents PAL-SLAM2, a visual and visual–inertial monocular simultaneous localization and mapping (SLAM) system for a panoramic annular lens (PAL) with an ultra-hemispherical field of view (FoV), overcoming the limitations of traditional frameworks in handling fast turns, nighttime conditions and rapid lighting changes. The system incorporates modules for initialization, tracking, local mapping, loop and map merging. To fully exploit information from the negative space (z<0), keypoints are projected onto the unit sphere for visual initialization and tightly-coupled visual–inertial optimization. Leveraging a multimap strategy, the feature-based PAL-SLAM2 is capable of detecting unidirectional and bidirectional public areas using PAL images, thereby enhancing the performance of loop correction and map merging. Testing indicates that it achieves an average accuracy of 7.1 cm on the PALVIO indoor dataset, surpassing similar state-of-the-art frameworks. Furthermore, we have collected a large-scale dataset comprising over 120,000 images with corresponding inertial measurement unit (IMU) data, demonstrating PAL-SLAM2’s robustness under challenging outdoor conditions. Relying solely on visual and inertial inputs, our system is particularly suited for environments where Global Navigation Satellite System (GNSS) signals are frequently obstructed, such as indoor spaces or dense urban areas. The dataset can be accessed at: https://github.com/wwendy233/Plaza-Dataset.

Accurate Monocular SLAM Initialization via Structural Line Tracking.

In this paper, we present a novel monocular simultaneous localization and mapping (SLAM) initialization algorithm that relies on structural features by tracking structural lines. This approach addresses the limitations of the traditional method, which can fail to account for a lack of features or their uneven distribution. Our proposed method utilizes a sliding window approach to guarantee the quality and stability of the initial pose estimation. We incorporate multiple geometric constraints, orthogonal dominant directions, and coplanar structural lines to construct an efficient pose optimization strategy. Experimental evaluations conducted on both the collected chessboard datasets and real scene datasets show that our approach provides superior results in terms of accuracy and real-time performance compared to the well-tuned baseline methods. Notably, our algorithm achieves these improvements while being computationally lightweight, without the need for matrix decomposition.

Improved Real-Time Monocular SLAM Using Semantic Segmentation on Selective Frames

Monocular simultaneous localization and mapping (SLAM) is emerging in advanced driver assistance systems and autonomous driving, because a single camera is cheap and easy to install. Conventional monocular SLAM has two major challenges leading inaccurate localization and mapping. First, it is challenging to estimate scales in localization and mapping. Second, conventional monocular SLAM uses inappropriate mapping factors such as dynamic objects and low-parallax areas in mapping. This paper proposes an improved real-time monocular SLAM that resolves the aforementioned challenges by efficiently using deep learning-based semantic segmentation. To achieve the real-time execution of the proposed method, we apply semantic segmentation only to downsampled keyframes in parallel with mapping processes. In addition, the proposed method corrects scales of camera poses and three-dimensional (3D) points, using estimated ground plane from road-labeled 3D points and the real camera height. The proposed method also removes inappropriate corner features labeled as moving objects and low parallax areas. Experiments with eight video sequences demonstrate that the proposed monocular SLAM system achieves significantly improved and comparable trajectory tracking accuracy, compared to existing state-of-the-art monocular and stereo SLAM systems, respectively. The proposed system can achieve real-time tracking on a standard CPU potentially with a standard GPU support, whereas existing segmentation-aided monocular SLAM does not.

OG-SLAM: A real-time and high-accurate monocular visual SLAM framework

The challenge of improving the accuracy of monocular Simultaneous Localization and Mapping (SLAM) is considered, which widely appears in computer vision, autonomous robotics, and remote sensing. A new framework (ORB-GMS-SLAM (or OG-SLAM)) is proposed, which introduces the region-based motion smoothness into a typical Visual SLAM (V-SLAM) system. The region-based motion smoothness is implemented by integrating the Oriented Fast and Rotated Brief (ORB) features and the Grid-based Motion Statistics (GMS) algorithm into the feature matching process. The OG-SLAM significantly reduces the absolute trajectory error (ATE) on the key-frame trajectory estimation without compromising the real-time performance. This study compares the proposed OG-SLAM to an advanced V-SLAM system (ORB-SLAM2). The results indicate the highest accuracy improvement of almost 75% on a typical RGB-D SLAM benchmark. Compared with other ORB-SLAM2 settings (1800 key points), the OG-SLAM improves the accuracy by around 20% without losing performance in real-time. The OG-SLAM framework has a significant advantage over the ORB-SLAM2 system in that it is more robust for rotation, loop-free, and long ground-truth length scenarios. Furthermore, as far as the authors are aware, this framework is the first attempt to integrate the GMS algorithm into the V-SLAM.

Warehouse Inspection with an Autonomous Micro Air Vehicle

In this work, we present an approach to address the problem of warehouse inspection using a MicroAir Vehicle (MAV) that performs autonomous flight by following a set of waypoints in a GPS-denied environment. During the navigation, a second onboard camera is used to observe the inspection area where it is expected to observe QR codes attached to packages placed on shelves. To this end, we use a metric monocular simultaneous localization And Mapping (SLAM) system to estimate the MAVs position in metres, avoiding the need for an external positioning system. The onboard computer executes the detection and decoding of QR codes and the SLAM system. The MAV can also communicate with a Ground Control Station (GCS) to display telemetry, the MAVs position, images from the inspection camera with detected QR codes, and a list of the found packages. This approach was used to compete in the indoors competition of the International Micro Air Vehicle Competition (IMAV) 2019, where we received the special award: Best Flight Performance.

AFE-ORB-SLAM: Robust Monocular VSLAM Based on Adaptive FAST Threshold and Image Enhancement for Complex Lighting Environments

Monocular Visual Simultaneous Localisation and Mapping (VSLAM) systems are widely utilised for intelligent mobile robots to work in unknown environments. However, complex and varying illuminations challenge the accuracy and robustness of VSLAM systems significantly. Existing feature-based VSLAM methods often fail due to the insufficient feature points that can be extracted in those challenging illumination environments. Therefore, this paper proposes an improved ORB-SLAM algorithm based on adaptive FAST threshold and image enhancement (AFE-ORB-SLAM), which works in the environments with complex lighting conditions. An improved truncated Adaptive Gamma Correction (AGC) is combined with unsharp masking to reduce the effect caused by different illuminations. What is more, an improved ORB feature extraction method with the adaptive FAST threshold is proposed and adopted to obtain more reliable feature points. To verify the performance of the AFE-ORB-SLAM, three public datasets (the extended Imperial College London and National University of Ireland Maynooth (ICL-NUIM) dataset with different lighting conditions, Onboard Illumination Visual-Inertial Odometry (OIVIO) dataset and the European Robotics Challenge (EuRoC) dataset) are utilised. The results are compared with other state-of-the-art monocular VSLAM methods. The experimental results demonstrate that the AFE-ORB-SLAM could achieve the highest average localisation accuracy with robust performance in the environments with complex lighting conditions while keeping similar performance in the normal lighting scenarios.

Lie Group Modelling for an EKF-Based Monocular SLAM Algorithm

This paper addresses the problem of monocular Simultaneous Localization And Mapping on Lie groups using fiducial patterns. For that purpose, we propose a reformulation of the classical camera model as a model on matrix Lie groups. Thus, we define an original-state vector containing the camera pose and the set of transformations from the world frame to each pattern, which constitutes the map’s state. Each element of the map’s state, as well as the camera pose, are intrinsically constrained to evolve on the matrix Lie group SE(3). Filtering is then performed by an extended Kalman filter dedicated to matrix Lie groups to solve the visual SLAM process (LG-EKF-VSLAM). This algorithm has been evaluated in different scenarios based on simulated data as well as real data. The results show that the LG-EKF-VSLAM can improve the absolute position and orientation accuracy, compared to a classical EKF visual SLAM (EKF-VSLAM).

3D Texture Reconstruction of Abdominal Cavity Based on Monocular Vision SLAM for Minimally Invasive Surgery

The depth information of abdominal tissue surface and the position of laparoscope are very important for accurate surgical navigation in computer-aided surgery. It is difficult to determine the lesion location by empirically matching the laparoscopic visual field with the preoperative image, which is easy to cause intraoperative errors. Aiming at the complex abdominal environment, this paper constructs an improved monocular simultaneous localization and mapping (SLAM) system model, which can more accurately and truly reflect the abdominal cavity structure and spatial relationship. Firstly, in order to enhance the contrast between blood vessels and background, the contrast limited adaptive histogram equalization (CLAHE) algorithm is introduced to preprocess abdominal images. Secondly, combined with AKAZE algorithm, the Oriented FAST and Rotated BRIEF(ORB) algorithm is improved to extract the features of abdominal image, which improves the accuracy of extracted symmetry feature points pair and uses the RANSAC algorithm to quickly eliminate the majority of mis-matched pairs. The medical bag-of-words model is used to replace the traditional bag-of-words model to facilitate the comparison of similarity between abdominal images, which has stronger similarity calculation ability and reduces the matching time between the current abdominal image frame and the historical abdominal image frame. Finally, Poisson surface reconstruction is used to transform the point cloud into a triangular mesh surface, and the abdominal cavity texture image is superimposed on the 3D surface described by the mesh to generate the abdominal cavity inner wall texture. The surface of the abdominal cavity 3D model is smooth and has a strong sense of reality. The experimental results show that the improved SLAM system increases the registration accuracy of feature points and the densification, and the visual effect of dense point cloud reconstruction is more realistic for Hamlyn dataset. The 3D reconstruction technology creates a realistic model to identify the blood vessels, nerves and other tissues in the patient’s focal area, enabling three-dimensional visualization of the focal area, facilitating the surgeon’s observation and diagnosis, and digital simulation of the surgical operation to optimize the surgical plan.

3FO: The Three-Frame-Only Approach for Fast and Accurate Monocular SLAM Initialization

The monocular simultaneous localization and mapping (SLAM) system is one of the most important among all visual or visual-inertial SLAM (VSLAM or VISLAM) systems due to its low cost, easy calibration and identification. Initialization is always crucial to bootstrap the monocular SLAM system. With the rapid growth of some quick-start required SLAM applications, e.g., augmented reality (AR) and unmanned aerial vehicles (UAVs), devising faster and more accurate initialization has become a central problem. Traditional initialization uses the first two frames to create landmarks and then computes camera poses for the subsequential frames, using a 3D-to-2D perspective-n-points (PnP) mechanism. In this paper, we propose a novel three-frame-only (3FO) initialization approach for the monocular SLAM system, which consists of two steps. In the first step, we use the first two frames to preinitialize poses and landmarks, and in the second step, we use the second and third frames to improve the preinitialization by using the scale consistency of the landmarks generated in the first step to filter out outliers and using inliers to generate more robust landmarks. In both steps, we use the pretrained multilayer perceptron (MLP) combined with homotopy continuation to solve the essential matrices. Finally, we perform a global bundle adjustment (BA) to refine the three camera poses and all the created landmarks. The proposed 3FO initialization approach is evaluated experimentally on the EuRoC benchmark data set with the initialization time and trajectory metrics. The results show that, compared to the traditional ORB-SLAM2 initialization, the 3FO approach reduces the initialization time by 5 times and improves the accuracy by 36.7% on average.

Dense monocular Simultaneous Localization and Mapping by direct surfel optimization

This work presents a novel approach for monocular dense Simultaneous Localization and Mapping. The surface to be estimated is represented as a piecewise planar surface, defined as a group of surfels each having as parameters its position and normal. These parameters are then directly estimated from the raw camera pixels measurements, by a Gauss-Newton iterative process. The representation of the surface as a group of surfels has several advantages. It allows the recovery of robust and accurate pixel depths, without the need to use a computationally demanding depth regularization schema. This has the further advantage of avoiding the use of a physically unlikely surface smoothness prior. New surfels can be correctly initialized from the information present in nearby surfels, avoiding also the need to use an expensive initialization routine commonly needed in Gauss-Newton methods. The method was written in the GLSL shading language, allowing the usage of GPU thus achieving real-time. The method was tested against several datasets, showing both its depth and normal estimation correctness, and its scene reconstruction quality. The results presented here showcase the usefulness of the more physically grounded piecewise planar scene depth prior, instead of the more commonly pixel depth independence and smoothness prior.

Uncertainty Estimation of Dense Optical Flow for Robust Visual Navigation.

This paper presents a novel dense optical-flow algorithm to solve the monocular simultaneous localisation and mapping (SLAM) problem for ground or aerial robots. Dense optical flow can effectively provide the ego-motion of the vehicle while enabling collision avoidance with the potential obstacles. Existing research has not fully utilised the uncertainty of the optical flow—at most, an isotropic Gaussian density model has been used. We estimate the full uncertainty of the optical flow and propose a new eight-point algorithm based on the statistical Mahalanobis distance. Combined with the pose-graph optimisation, the proposed method demonstrates enhanced robustness and accuracy for the public autonomous car dataset (KITTI) and aerial monocular dataset.

Visual-Pressure Fusion for Underwater Robot Localization With Online Initialization

The motion of underwater robot is usually slow, which leads to large scale error easily introduced in the positioning method based on monocular Visual Inertial Odometry (VIO). To solve the above problem, we present a positioning method based on visual-pressure fusion in this letter. Firstly, it is proved by observability analysis that when the robot's motion is not on the same horizontal plane, the system scale can be uniquely determined by monocular image and depth information; secondly, a novel visual-pressure initialization method is proposed, which can calculate the system scale and initial attitude with respect to world frame online; finally, we integrate our method into current advanced monocular Simultaneous Localization And Mapping (SLAM) system, and the effectiveness and superiority of the method are verified by experiments on simulation datasets and real datasets.

Utilization of Semantic Planes: Improved Localization and Dense Semantic Map for Monocular SLAM in Urban Environment

In this letter, we propose a novel semantic direct monocular simultaneous localization and mapping (SLAM) system that fuses the semantic information obtained by an advanced deep neural network (DNN) into direct sparse odometry with loop closure(LDSO), with the purpose of improving the localization accuracy and building a dense semantic map of the urban environment. For localization, we apply a point reselection strategy based on coarse semantic plane (CSP) constraints to discard static points inconsistent with the nearby co-plane points of the same semantic class and dynamic points beyond the visible range. Moreover, a point group movement consistency (PGMC) check is utilized to decrease the impact of moving dynamic objects. For the dense semantic map, we model numerous small semantic planes from well-estimated points to measure the depth of each static pixel, rather than conduct stereo matching. Experimental results show that our method is more accurate than LDSO and comparable with ORB-SLAM in terms of localization. Moreover, it is capable of building a dense semantic map of the urban environment for better scene understanding.

Hybrid Monocular SLAM Using Double Window Optimization

This letter presents a hybrid framework, both in front-end and back-end, for monocular simultaneous localization and mapping (SLAM), capable of utilizing the robustness of feature matching and the accuracy of direct alignment. In the front-end, the feature-based method is first used for coarse pose estimation that is subsequently taken by the direct alignment module as initialization for further refinement. In the back-end, a double window structure is constructed based on the maintained semi-dense map and the sparse feature map, of which the states are optimized via a multi-layer optimization scheme based on the reprojection constraints and the relative pose constraints. Our evaluation on several public datasets demonstrates that this hybrid design retains the superior resilience to scene variations of salient features, and achieves better tracking accuracy due to the integration of the direct modules, leading to a comparable performance with the state-of-the-arts.

Visual navigation and docking for a planar type AUV docking and charging system

As being increasingly used for underwater explorations, Autonomous Underwater Vehicles (AUVs) are diversified in their structures and outlines to match up with various mission requirements, which brings challenges to the design of customized submerged docking stations. Aiming at providing a generalized solution, this paper proposes an omnidirectional and positioning-tolerant planar type AUV docking and charging platform, which has no constraints on AUV structures. In order to solve the planar-type docking issues, an integrated visual navigation and docking algorithm is developed for the miniaturized prototype AUV. The adopted monocular Simultaneous Localization and Mapping (SLAM) program takes charge of the wide-range underwater locationing, and handles practical problems such as scale drift and tracking failure. A combined control strategy of horizontal dynamic positioning and vertical visual servo docking is developed for the terminal docking process. The strategy is robust against constant water flow disturbance and performs stable docking using a roll motion adjustment. The visual navigation and docking performances and other system functions are successfully validated in simulation, water pool experiment, and sea trial.

Monocular Simultaneous Localization and Mapping Initialization Method Based on Point and Line Features

Monocular Simultaneous Localization and Mapping Initialization Method Based on Point and Line Features

JD-SLAM: Joint camera pose estimation and moving object segmentation for simultaneous localization and mapping in dynamic scenes

As a fundamental assumption in simultaneous localization and mapping, the static scenes hypothesis can be hardly fulfilled in applications of indoor/outdoor navigation or localization. Recent works about simultaneous localization and mapping in dynamic scenes commonly use heavy pixel-level segmentation net to distinguish dynamic objects, which brings enormous calculations and limits the real-time performance of the system. That restricts the application of simultaneous localization and mapping on the mobile terminal. In this article, we present a lightweight system for monocular simultaneous localization and mapping in dynamic scenes, which can run in real time on central processing unit (CPU) and generate a semantic probability map. The pixel-wise semantic segmentation net is replaced with a lightweight object detection net combined with three-dimensional segmentation based on motion clustering. And a framework integrated with an improved weighted-random sample consensus solver is proposed to jointly solve the camera pose and perform three-dimensional object segmentation, which enables high accuracy and efficiency. Besides, the prior information of the generated map and the object detection results is introduced for better estimation. The experiments on the public data set, and in the real-world demonstrate that our method obtains an outstanding improvement in both accuracy and speed compared to state-of-the-art methods.