Monocular Simultaneous Localization And Mapping System Research Articles

A brain-inspired compact cognitive mapping system.

In many simultaneous localization and mapping (SLAM) systems, the map of the environment grows over time as the robot explores the environment. The ever-growing map prevents long-term mapping, especially in large-scale environments. In this paper, we develop a compact cognitive mapping approach inspired by neurobiological experiments. Mimicking the firing activities of neighborhood cells, neighborhood fields determined by movement information, i.e. translation and rotation, are modeled to describe one of the distinct segments of the explored environment. The vertices with low neighborhood field activities are avoided to be added into the cognitive map. The optimization of the cognitive map is formulated as a robust non-linear least squares problem constrained by the transitions between vertices, and is numerically solved efficiently. According to the cognitive decision-making of place familiarity, loop closure edges are clustered depending on time intervals, and then batch global optimization of the cognitive map is performed to satisfy the combined constraint of the whole cluster. After the loop closure process, scene integration is performed, in which revisited vertices are removed subsequently to further reduce the size of the cognitive map. The compact cognitive mapping approach is tested on a monocular visual SLAM system in a naturalistic maze for a biomimetic animated robot. Our results demonstrate that the proposed method largely restricts the growth of the size of the cognitive map over time, and meanwhile, the compact cognitive map correctly represents the overall layout of the environment. The compact cognitive mapping method is well suitable for the representation of large-scale environments to achieve long-term robot navigation.

Monocular Visual SLAM Based on a Cooperative UAV-Target System.

To obtain autonomy in applications that involve Unmanned Aerial Vehicles (UAVs), the capacity of self-location and perception of the operational environment is a fundamental requirement. To this effect, GPS represents the typical solution for determining the position of a UAV operating in outdoor and open environments. On the other hand, GPS cannot be a reliable solution for a different kind of environments like cluttered and indoor ones. In this scenario, a good alternative is represented by the monocular SLAM (Simultaneous Localization and Mapping) methods. A monocular SLAM system allows a UAV to operate in a priori unknown environment using an onboard camera to simultaneously build a map of its surroundings while at the same time locates itself respect to this map. So, given the problem of an aerial robot that must follow a free-moving cooperative target in a GPS denied environment, this work presents a monocular-based SLAM approach for cooperative UAV–Target systems that addresses the state estimation problem of (i) the UAV position and velocity, (ii) the target position and velocity, (iii) the landmarks positions (map). The proposed monocular SLAM system incorporates altitude measurements obtained from an altimeter. In this case, an observability analysis is carried out to show that the observability properties of the system are improved by incorporating altitude measurements. Furthermore, a novel technique to estimate the approximate depth of the new visual landmarks is proposed, which takes advantage of the cooperative target. Additionally, a control system is proposed for maintaining a stable flight formation of the UAV with respect to the target. In this case, the stability of control laws is proved using the Lyapunov theory. The experimental results obtained from real data as well as the results obtained from computer simulations show that the proposed scheme can provide good performance.

DM-SLAM: Monocular SLAM in Dynamic Environments

Many classic visual monocular SLAM (simultaneous localization and mapping) systems have been developed over the past decades, yet most of them fail when dynamic scenarios dominate. DM-SLAM is proposed for handling dynamic objects in environments based on ORB-SLAM2. This article mainly concentrates on two aspects. Firstly, we proposed a distribution and local-based RANSAC (Random Sample Consensus) algorithm (DLRSAC) to extract static features from the dynamic scene based on awareness of the nature difference between motion and static, which is integrated into initialization of DM-SLAM. Secondly, we designed a candidate map points selection mechanism based on neighborhood mutual exclusion to balance the accuracy of tracking camera pose and system robustness in motion scenes. Finally, we conducted experiments in the public dataset and compared DM-SLAM with ORB-SLAM2. The experiments corroborated the superiority of the DM-SLAM.

Panoramic SLAM from a multiple fisheye camera rig

This paper presents a feature-based simultaneous localization and mapping (SLAM) system for panoramic image sequences obtained from a multiple fisheye camera rig in a wide baseline mobile mapping system (MMS). First, the developed fisheye camera calibration method combines an equidistance projection model and trigonometric polynomial to achieve high-accuracy calibration from the fisheye camera to an equivalent ideal frame camera, which warrants an accurate transform from the fisheye images to a corresponding panoramic image. Second, we developed a panoramic camera model, corresponding bundle adjustment with a specific back propagation error function, and linear pose initialization algorithm. Third, the implemented feature-based SLAM pipeline consists of several specific strategies and algorithms in initialization, feature matching, frame tracking, and loop closing to overcome the difficulties in tracking wide baseline panoramic image sequences. We conducted experiments on large-scale MMS datasets of more than 15 km trajectories and 14,000 panoramic images as well as small-scale public video datasets. Our results show that the developed panoramic SLAM system PAN-SLAM can achieve fully-automatic camera localization and sparse map reconstruction in both small-scale indoor and large-scale outdoor environments including challenging scenes (e.g., dark tunnel) without the aid of any other sensors. The localization accuracy, which was measured by the absolute trajectory error (ATE), resembled the high-accuracy GNSS/INS reference of 0.1 m. PAN-SLAM also outperformed several feature-based fisheye and monocular SLAM systems with incomparable robustness in various environments. The system could be considered as a robust complementary solution and an alternative to expensive commercial navigation systems, especially in urban environments where signal obstruction and multipath interference are common. Source code and demo are available at http://study.rsgis.whu.edu.cn/pages/download/.

Accurate and Robust Monocular SLAM with Omnidirectional Cameras.

Simultaneous localization and mapping (SLAM) are fundamental elements for many emerging technologies, such as autonomous driving and augmented reality. For this paper, to get more information, we developed an improved monocular visual SLAM system by using omnidirectional cameras. Our method extends the ORB-SLAM framework with the enhanced unified camera model as a projection function, which can be applied to catadioptric systems and wide-angle fisheye cameras with 195 degrees field-of-view. The proposed system can use the full area of the images even with strong distortion. For omnidirectional cameras, a map initialization method is proposed. We analytically derive the Jacobian matrices of the reprojection errors with respect to the camera pose and 3D position of points. The proposed SLAM has been extensively tested in real-world datasets. The results show positioning error is less than 0.1% in a small indoor environment and is less than 1.5% in a large environment. The results demonstrate that our method is real-time, and increases its accuracy and robustness over the normal systems based on the pinhole model.

Real-time human segmentation by BowtieNet and a SLAM-based human AR system

BackgroundGenerally, it is difficult to obtain accurate pose and depth for a non-rigid moving object from a single RGB camera to create augmented reality (AR). In this study, we build an augmented reality system from a single RGB camera for a non-rigid moving human by accurately computing pose and depth, for which two key tasks are segmentation and monocular Simultaneous Localization and Mapping (SLAM). Most existing monocular SLAM systems are designed for static scenes, while in this AR system, the human body is always moving and non-rigid. MethodsIn order to make the SLAM system suitable for a moving human, we first segment the rigid part of the human in each frame. A segmented moving body part can be regarded as a static object, and the relative motions between each moving body part and the camera can be considered the motion of the camera. Typical SLAM systems designed for static scenes can then be applied. In the segmentation step of this AR system, we first employ the proposed BowtieNet, which adds the atrous spatial pyramid pooling (ASPP) of DeepLab between the encoder and decoder of SegNet to segment the human in the original frame, and then we use color information to extract the face from the segmented human area. ResultsBased on the human segmentation results and a monocular SLAM, this system can change the video background and add a virtual object to humans. ConclusionsThe experiments on the human image segmentation datasets show that BowtieNet obtains state-of-the-art human image segmentation performance and enough speed for real-time segmentation. The experiments on videos show that the proposed AR system can robustly add a virtual object to humans and can accurately change the video background.

Loosely-Coupled Semi-Direct Monocular SLAM

We propose a novel semi-direct approach for monocular simultaneous localization and mapping (SLAM) that combines the complementary strengths of direct and feature-based methods. The proposed pipeline loosely couples direct odometry and feature-based SLAM to perform three levels of parallel optimizations: 1) photometric bundle adjustment (BA) that jointly optimizes the local structure and motion, 2) geometric BA that refines keyframe poses and associated feature map points, and 3) pose graph optimization to achieve global map consistency in the presence of loop closures. This is achieved in real-time by limiting the feature-based operations to marginalized keyframes from the direct odometry module. Exhaustive evaluation on two benchmark datasets demonstrates that our system outperforms the state-of-the-art monocular odometry and SLAM systems in terms of overall accuracy and robustness.

Real-Time Dense Monocular SLAM With Online Adapted Depth Prediction Network

Considerable advances have been achieved in estimating the depth map from a single image via convolutional neural networks (CNNs) during the past few years. Combining depth prediction from CNNs with conventional monocular simultaneous localization and mapping (SLAM) is promising for accurate and dense monocular reconstruction, in particular addressing the two long-standing challenges in conventional monocular SLAM: low map completeness and scale ambiguity. However, depth estimated by pretrained CNNs usually fails to achieve sufficient accuracy for environments of different types from the training data, which are common for certain applications such as obstacle avoidance of drones in unknown scenes. Additionally, inaccurate depth prediction of CNN could yield large tracking errors in monocular SLAM. In this paper, we present a real-time dense monocular SLAM system, which effectively fuses direct monocular SLAM with an online-adapted depth prediction network for achieving accurate depth prediction of scenes of different types from the training data and providing absolute scale information for tracking and mapping. Specifically, on one hand, tracking pose (i.e., translation and rotation) from direct SLAM is used for selecting a small set of highly effective and reliable training images, which acts as ground truth for tuning the depth prediction network on-the-fly toward better generalization ability for scenes of different types. A stage-wise Stochastic Gradient Descent algorithm with a selective update strategy is introduced for efficient convergence of the tuning process. On the other hand, the dense map produced by the adapted network is applied to address scale ambiguity of direct monocular SLAM which in turn improves the accuracy of both tracking and overall reconstruction. The system with assistance of both CPUs and GPUs, can achieve real-time performance with progressively improved reconstruction accuracy. Experimental results on public datasets and live application to obstacle avoidance of drones demonstrate that our method outperforms the state-of-the-art methods with greater map completeness and accuracy, and a smaller tracking error.

Semi-direct monocular visual and visual-inertial SLAM with loop closure detection

A novel semi-direct monocular visual simultaneous localization and mapping (SLAM) system is proposed to maintain the fast performance of a direct method and the high precision and loop closure capability of a feature-based method. This system extracts and matches Oriented FAST and Rotated BRIEF features in a keyframe and tracks a non-keyframe via a direct method without the requirement of extracting and matching features. A keyframe is used for global or local optimization and loop closure, whereas a non-keyframe is used for fast tracking and localization, thereby combining the advantages of direct and feature-based methods. A monocular visual-inertial SLAM system that fuses inertial measurement data with visual SLAM is also proposed. This system successfully recovers the metric scale successfully. The evaluation on datasets shows that the proposed approach accomplishes loop closure detection successfully and requires less time to achieve accuracy comparable with that of feature-based method. The physical experiment demonstrates the feasibility and robustness of the proposed SLAM. The approach achieves good balance between speed and accuracy and provides valuable references for design and improvement of other SLAM methods.

Monocular SLAM System for MAVs Aided with Altitude and Range Measurements: a GPS-free Approach

A typical navigation system for a Micro Aerial Vehicle (MAV) relies basically on GPS for position estimation. However, for several kinds of applications, the precision of the GPS is inappropriate or even its signal can be unavailable. In this context, and due to its flexibility, Monocular Simultaneous Localization and Mapping (SLAM) methods have become a good alternative for implementing visual-based navigation systems for MAVs that must operate in GPS-denied environments. On the other hand, one of the most important challenges that arises with the use of the monocular vision is the difficulty to recover the metric scale of the world. In this work, a monocular SLAM system for MAVs is presented. In order to overcome the problem of the metric scale, a novel technique for inferring the approximate depth of visual features from an ultrasonic range-finder is developed. Additionally, the altitude of the vehicle is updated using the pressure measurements of a barometer. The proposed approach is supported by the theoretical results obtained from a nonlinear observability test. Experiments performed with both computer simulations and real data are presented in order to validate the performance of the proposal. The results confirm the theoretical findings and show that the method is able to work with low-cost sensors.

Minimalistic approach for monocular SLAM system applied to micro aerial vehicles in GPS-denied environments

In this work, a novel monocular simultaneous localization and mapping (SLAM) system with application to micro aerial vehicles is proposed. The main difference with respect to previous approaches is that a barometer is used as a unique sensory aid for incorporating altitude information into the system in order to recover an absolute metric scale. First, an observability analysis of a simplified model of a monocular SLAM system is developed. From this analysis, several theoretical results are derived. Among others, one important result is related to the fact that the metric scale can become observable when measurements of altitude are included in the system. In this case, sufficient conditions for observability are presented. The design of the proposed method is based on these theoretical results. Simulations and experiments with real data are presented to validate the proposed approach. The results confirm that the metric scale can be retrieved by including altitude measurements in the system. It is also shown that the proposed method can be practically implemented, using low-cost sensors, to perform visual-based navigation in GPS-denied environments.

Vision-based SLAM system for MAVs in GPS-denied environments

Using a camera, a micro aerial vehicle (MAV) can perform visual-based navigation in periods or circumstances when GPS is not available, or when it is partially available. In this context, the monocular simultaneous localization and mapping (SLAM) methods represent an excellent alternative, due to several limitations regarding to the design of the platform, mobility and payload capacity that impose considerable restrictions on the available computational and sensing resources of the MAV. However, the use of monocular vision introduces some technical difficulties as the impossibility of directly recovering the metric scale of the world. In this work, a novel monocular SLAM system with application to MAVs is proposed. The sensory input is taken from a monocular downward facing camera, an ultrasonic range finder and a barometer. The proposed method is based on the theoretical findings obtained from an observability analysis. Experimental results with real data confirm those theoretical findings and show that the proposed method is capable of providing good results with low-cost hardware.

Multi-camera visual SLAM for autonomous navigation of micro aerial vehicles

In this paper, we present a visual simultaneous localization and mapping (SLAM) system which integrates measurements from multiple cameras to achieve robust pose tracking for autonomous navigation of micro aerial vehicles (MAVs) in unknown complex environments. We analyze the iterative optimizations for pose tracking and map refinement of visual SLAM in multi-camera cases. The analysis ensures the soundness and accuracy of each optimization update. A well-known monocular visual SLAM system is extended to utilize two cameras with non-overlapping fields of view (FOVs) in the final implementation. The resulting visual SLAM system enables autonomous navigation of an MAV in complex scenarios. The theory behind this system can easily be extended to multi-camera configurations, when the onboard computational capability allows this. For operations in large-scale environments, we modify the resulting visual SLAM system to be a constant-time robust visual odometry. To form a full visual SLAM system, we further implement an efficient back-end for loop closing. The back-end maintains a keyframe-based global map, which is also used for loop-closure detection. An adaptive-window pose-graph optimization method is proposed to refine keyframe poses of the global map and thus correct pose drift that is inherent in the visual odometry. We demonstrate the efficiency of the proposed visual SLAM system for applications onboard of MAVs in experiments with both autonomous and manual flights. The pose tracking results are compared with ground truth data provided by an external tracking system.

Monocular SLAM for large scale scenes

Monocular simultaneous localization and mapping (SLAM) methods easily accumulate error along with the growing map. Especially for a large scale scene, accumulating error may be too large to prevent the close of loops. Bundle adjustment can be employed to close loops and eliminate accumulation error. However, for a large scale scene, traditional bundle adjustment may have the bottleneck in terms of memory and efficiency, which limits its applications in practice. Based on our prior off-line SfM work, in this paper, we present a novel monocular SLAM system for large scale scenes. We modify and improve the key techniques to make them appropriate for real-time SLAM. Particularly, a novel online loop closure detection method based on non-consecutive track matching is proposed, which can automatically detect and efficiently match the common features among different subsequences. In addition, a segment-based bundle adjustment is employed to close the loop efficiently, which can perform an efficient global optimization in a limited memory space to effectively eliminate accumulation error. In the experiments, we made qualitative and quantitative comparison with two state-of-the-art SLAM systems using both our own dataset and KITTI dataset, which demonstrates the effectiveness of the proposed approach.

COP-SLAM: Closed-Form Online Pose-Chain Optimization for Visual SLAM

In this paper, we analyze and extend the recently proposed closed-form online pose-chain simultaneous localization and mapping (SLAM) algorithm. Pose-chains are a specific type of extremely sparse pose-graphs and a product of contemporary SLAM front-ends, which perform accurate visual odometry and reliable appearance-based loop detection. They are relevant for challenging robotic applications in large-scale 3-D environments for which frequent loop detection is not desired or not possible. Closed-form online pose-chain SLAM efficiently and accurately optimizes pose-chains by exploiting their Lie group structure. The convergence and optimality properties of this solution are discussed in detail and are compared against state-of-the-art iterative methods. We also provide a novel solution space, that of similarity transforms, which has not been considered earlier for the proposed algorithm. This allows for closed-form optimization of pose-chains that exhibit scale drift, which is important to monocular SLAM systems. On the basis of extensive experiments, specifically targeting 3-D pose-chains and using a total of 60 km of challenging binocular and monocular data, it is shown that the accuracy obtained by closed-form online pose-chain SLAM is comparable with that of state-of-the-art iterative methods, while the time it needs to compute its solution is orders of magnitudes lower. This novel SLAM technique thereby is relevant to a broad range of robotic applications and computational platforms.

NARROW FIELD-OF-VIEW VISUAL ODOMETRY BASED ON A FOCUSED PLENOPTIC CAMERA

Abstract. In this article we present a new method for visual odometry based on a focused plenoptic camera. This method fuses the depth data gained by a monocular Simultaneous Localization and Mapping (SLAM) algorithm and the one received from a focused plenoptic camera. Our algorithm uses the depth data and the totally focused images supplied by the plenoptic camera to run a real-time semi-dense direct SLAM algorithm. Based on this combined approach, the scale ambiguity of a monocular SLAM system can be overcome. Furthermore, the additional light-field information highly improves the tracking capabilities of the algorithm. Thus, visual odometry even for narrow field of view (FOV) cameras is possible. We show that not only tracking profits from the additional light-field information. By accumulating the depth information over multiple tracked images, also the depth accuracy of the focused plenoptic camera can be highly improved. This novel approach improves the depth error by one order of magnitude compared to the one received from a single light-field image.

A robust approach for a filter-based monocular simultaneous localization and mapping (SLAM) system.

Simultaneous localization and mapping (SLAM) is an important problem to solve in robotics theory in order to build truly autonomous mobile robots. This work presents a novel method for implementing a SLAM system based on a single camera sensor. The SLAM with a single camera, or monocular SLAM, is probably one of the most complex SLAM variants. In this case, a single camera, which is freely moving through its environment, represents the sole sensor input to the system. The sensors have a large impact on the algorithm used for SLAM. Cameras are used more frequently, because they provide a lot of information and are well adapted for embedded systems: they are light, cheap and power-saving. Nevertheless, and unlike range sensors, which provide range and angular information, a camera is a projective sensor providing only angular measurements of image features. Therefore, depth information (range) cannot be obtained in a single step. In this case, special techniques for feature system-initialization are needed in order to enable the use of angular sensors (as cameras) in SLAM systems. The main contribution of this work is to present a novel and robust scheme for incorporating and measuring visual features in filtering-based monocular SLAM systems. The proposed method is based in a two-step technique, which is intended to exploit all the information available in angular measurements. Unlike previous schemes, the values of parameters used by the initialization technique are derived directly from the sensor characteristics, thus simplifying the tuning of the system. The experimental results show that the proposed method surpasses the performance of previous schemes.

A vertical and floor line-based monocular SLAM system for corridor environments

In this paper, we propose a vertical and floor line-based monocular simultaneous localization and mapping (SLAM) system which utilizes vertical lines, floor lines, and vanishing points as sensory input to perform robust SLAM in corridor environments. By combining three map feature types, our design can help a robot to perform accurate pose estimation, repeatable loop closure, and to construct a more expressive environmental map. As a primitive element of a geometric structure, a line segment has one additional dimension compared to a point feature, thereby allowing the use of line segments to easily represent a geometric structure using a smaller number of features. This system presents map features on a 2D ground space: the vertical line as a projection point, the floor line as the original line, and the vanishing point as a directional vector. Although the vertical line, floor line, and vanishing point use different parameterization and initialization methods, their measurement models are integrated into a unified extended Kalman filter (EKF) framework. Experimental results show that our system can be deployed in a structured indoor environment as a suitable SLAM solution.

On Optimal Dynamic Sequential Search for Matching in Real-Time Machine Vision

In the matching tasks of tracking and geometrical vision, there are usually priors available on the absolute and/or relative image locations of features of interest. In this paper, we use these priors dynamically to guide a feature by feature matching search that can achieve global matching with much fewer image processing operations and lower overall computational cost. First, the concept of dynamic sequential search (DSS) is presented. Then, the problem of determining an optimal search order for DSS is investigated, when the probabilistic distribution of the features can be described by a multivariate Gaussian model. Based on the general formulas for sequentially updating the predicted positions of the features as well as their innovation covariance, the theoretic lower bound for the sum of the areas of the features search-regions is derived, and the necessary and sufficient condition for the optimal search order to approach this lower bound is presented. After that, an algorithm for dynamically determining a suboptimal search order is presented, with a computational complexity of O(n3), which is two magnitudes lower than those of the state-of-the-art algorithms. The effectiveness of the proposed method is validated by both statistical simulation and real-world experiments with a monocular visual SLAM (simultaneous localization and mapping) system. The results verify that the performance of the proposed method is better than the state-of-the-art algorithms, with both fewer image processing operations and lower overall computational cost.