Camera Ego-motion Estimation Research Articles

ARD-SLAM: Accurate and robust dynamic SLAM using dynamic object identification and improved multi-view geometrical approaches

In the evolving landscape of autonomous navigation, traditional Visual Simultaneous Localization and Mapping (SLAM) systems often encounter challenges in dynamic environments, primarily due to their reliance on assumptions of static surroundings. In response to these limitations, we introduce ARD-SLAM, a groundbreaking approach to dynamic SLAM that innovatively combines global dense optical tracking with sophisticated geometric methodologies. The core innovation of ARD-SLAM lies in its dynamic object identification technique, which harmoniously integrates geometric motion information with prospective motion data. This integration facilitates effective segmentation of moving objects, thereby substantially diminishing their impact on camera ego-motion estimation. ARD-SLAM is further enhanced by an advanced multi-view geometry method that emphasizes the selection of well-matched feature points. This approach is instrumental in efficiently managing dynamic scenarios while also reducing computational load. Rigorous testing on the TUM RGB-D and Bonn RGB-D benchmark datasets has established ARD-SLAM's superiority over established techniques like ORB-SLAM2/3, DynaSLAM, SD-SLAM, DGS-SLAM, and OVD-SLAM. Notably, ARD-SLAM achieves a substantial average reduction in Absolute Trajectory Error (ATE) by 86.1% and in Relative Pose Error (RPE) by 88.0% compared to ORB-SLAM3. The results from the Bonn RGB-D Dataset further underscore ARD-SLAM's effectiveness. Compared to other SLAM methods, ARD-SLAM shows remarkable improvements: 37.8% and 66.4% over DynaSLAM, 41.2% and 73.1% over DGS-SLAM, and 48.9% and 79.7% over OVD-SLAM in ATE and RPE metrics, respectively. This robust performance in dynamically changing environments solidifies ARD-SLAM as a significant advancement in SLAM technology, offering a more precise and adaptable solution for the complex challenges of real-world autonomous navigation.

Accurate and robust visual SLAM with a novel ray-to-ray line measurement model

Line feature is regarded as a more intuitive and accurate landmark than point feature in visual SLAM for its multiple-pixel comprehensiveness. However, uncertain factors, such as partial occlusion and noise, frequently hinder the mapping accuracy and destabilize the line-assisted SLAM system. Structural regulations and prior hypotheses are often used to tackle the issues, whereas only few people explore the impact of line feature optimization. In this paper, we attempt to improve the accuracy and robustness of visual SLAM system through line feature optimization process. First, a concise ray-to-ray residual model is proposed to replace the prevalent point-to-line model to integrally use line features. Second, the information matrix related to observation uncertainties is calculated to normalize the residual model, which aims to better balance the weights of different lines. Third, we add the line model to ORB-SLAM3 system and design the method of point-and-line based tracking and optimization. Finally, quantitative criteria are proposed to objectively evaluate the line feature map. Both synthetical and real datasets experiments are carried out to demonstrate the advantages of our algorithm in terms of camera ego-motion estimation and mapping. For camera ego-motion estimation experiments, the proposed ray-to-ray residual model produces more accurate results compared to state-of-the-art line-assisted SLAM/VIO algorithms. Furthermore, the model runs faster and obtains more robust results than the prevalent point-to-line reprojection residual model. For mapping experiments, quantitative criteria are proposed, which also open a new perspective to evaluate line-assisted SLAM systems, and give clues to evidence that the proposed method builds a more accurate line feature map.

DecoupledPoseNet: Cascade Decoupled Pose Learning for Unsupervised Camera Ego-Motion Estimation

Although many impressive works on learning-based camera ego-motion estimation methods have been proposed recently, most of them promote the accuracy of camera pose estimation by various sequential learning with loop closure optimization, while neglecting the improvement of PoseNet itself. In this paper, we focus on the coupling of rotation and translation in ego-motion estimation, and design a cascade decoupling structure to separately learn the rotation and translation of camera relative motion between adjacent frames. Meanwhile, a rigid-aware unsupervised learning framework with iterative pose refinement scheme is proposed for camera ego-motion estimation. It can disambiguate rigid motion and deformations in dynamic scenarios by jointly learning of optical flow, stereo disparity and camera pose. Validated with evaluation experiments on the public available datasets, our method is superior to the state-of-the-art unsupervised methods, and can achieve comparable results with the supervised ones.

Rigid-aware self-supervised GAN for camera ego-motion estimation

Learning-based camera ego-motion estimation has attracted increasing attention and has made impressive improvements. However, the accuracy of unsupervised paradigm is still limited, especially in complex dynamic environment. In this paper, we propose a rigid-aware self-supervised generative adversarial network (GAN) for camera ego-motion estimation, which can effectively learn the rigidity of the scene and improve the accuracy of ego-motion estimation by combining the pixel- and the structure-level perception. Specifically, a rigid-aware generator is first designed for joint unsupervised learning of optical flow, stereo depth and camera pose from two consecutive frames. Then, an iterative pose refinement strategy with rigidity learning is presented to reduce the impact of moving objects in scenes. To overcome the limitation of the purely pixel-wised photometric methods, a rigidity mask embedded discriminator is attached to perceive structural distortion artifacts in synthesized fake images, which encourages the generator to learn additional structure-level information to improve the accuracy of pose estimation. Experiments on the benchmark datasets show that our model achieves the state-of-the-art performance in terms of both RPE and ATE compared to recent GAN-based methods.

Unsupervised Learning of Monocular Depth and Ego-Motion with Optical Flow Features and Multiple Constraints.

This paper proposes a novel unsupervised learning framework for depth recovery and camera ego-motion estimation from monocular video. The framework exploits the optical flow (OF) property to jointly train the depth and the ego-motion models. Unlike the existing unsupervised methods, our method extracts the features from the optical flow rather than from the raw RGB images, thereby enhancing unsupervised learning. In addition, we exploit the forward-backward consistency check of the optical flow to generate a mask of the invalid region in the image, and accordingly, eliminate the outlier regions such as occlusion regions and moving objects for the learning. Furthermore, in addition to using view synthesis as a supervised signal, we impose additional loss functions, including optical flow consistency loss and depth consistency loss, as additional supervision signals on the valid image region to further enhance the training of the models. Substantial experiments on multiple benchmark datasets demonstrate that our method outperforms other unsupervised methods.

Ego-Motion Estimation Using Recurrent Convolutional Neural Networks through Optical Flow Learning

Visual odometry (VO) refers to incremental estimation of the motion state of an agent (e.g., vehicle and robot) by using image information, and is a key component of modern localization and navigation systems. Addressing the monocular VO problem, this paper presents a novel end-to-end network for estimation of camera ego-motion. The network learns the latent subspace of optical flow (OF) and models sequential dynamics so that the motion estimation is constrained by the relations between sequential images. We compute the OF field of consecutive images and extract the latent OF representation in a self-encoding manner. A Recurrent Neural Network is then followed to examine the OF changes, i.e., to conduct sequential learning. The extracted sequential OF subspace is used to compute the regression of the 6-dimensional pose vector. We derive three models with different network structures and different training schemes: LS-CNN-VO, LS-AE-VO, and LS-RCNN-VO. Particularly, we separately train the encoder in an unsupervised manner. By this means, we avoid non-convergence during the training of the whole network and allow more generalized and effective feature representation. Substantial experiments have been conducted on KITTI and Malaga datasets, and the results demonstrate that our LS-RCNN-VO outperforms the existing learning-based VO approaches.

RDMO-SLAM: Real-Time Visual SLAM for Dynamic Environments Using Semantic Label Prediction With Optical Flow

Visual simultaneous localization and mapping (vSLAM) are considered a fundamental technology for augmented reality and intelligent mobile robots. However, rigid scene assumption is common in vSLAM, which limits the wide usage in populated real-world environments. Recently, with the widespread use of artificial neural networks, many solutions have tried to eliminate the influence of dynamic objects using semantic information provided by object detection or semantic segmentation. Mask R-CNN is popular in many applications, but is usually slow and limits the speed of vSLAM because it waits for the semantic results before camera ego-motion estimation. We had previously introduced a real-time vSLAM, RDS-SLAM, which isolates tracking and semantic segmentation by adding a semantic thread and moving probability estimation. However, Mask R-CNN only supplies a small amount of semantic information because only a few keyframes can be segmented within a short time. Therefore, in this study, we propose a novel vSLAM, RDMO-SLAM, which can leverage more semantic information while ensuring the real-time nature by adding semantic label prediction using dense optical flow. Besides, we also estimate the velocity of each landmark and use them as constraints to reduce the influence of dynamic objects in tracking. Demonstrations are presented, which compare the proposed method to comparable state-of-the-art approaches using dynamic sequences. We improved the real-time performance from 15 Hz (RDS-SLAM) to 30 Hz while keeping robust tracking in dynamic scenes.

Independent Moving Object Detection Based on a Vehicle Mounted Binocular Camera

Accurate detection of independent moving objects is the key to ensure the safety of driverless vehicles during travelling. In this study, a moving object detection method based on a vehicle-mounted binocular camera is proposed. Firstly, a feature matching points selection strategy is designed for high accuracy camera ego-motion estimation. Then, the residual optical flow filed generated only by moving objects is estimated by taking into account the camera ego-motion and global mixed optical flow. A dynamic threshold segmentation method based on disparity is proposed to separate the moving regions from the residual optical flow field. Finally, 3D and 2D information are combined for extracting each single moving object from moving regions. The innovation of the proposed method is that it can detect any type and any size of moving object theoretically by using the dense motion information of the images. By taking the data in KITTI database as the test samples, the detection precision of the moving objects using the proposed algorithm can reach up to 91% without considering tracking strategy.

Unsupervised Learning of Depth, Optical Flow and Pose With Occlusion From 3D Geometry

In autonomous driving, monocular sequences contain lots of information. Monocular depth estimation, camera ego-motion estimation and optical flow estimation in consecutive frames are high-profile concerns recently. By analyzing tasks above, pixels in the middle frame are modeled into three parts: the rigid region, the non-rigid region, and the occluded region. In joint unsupervised training of depth and pose, we can segment the occluded region explicitly. The occlusion information is used in unsupervised learning of depth, pose and optical flow, as the image reconstructed by depth-pose and optical flow will be invalid in occluded regions. A less-than-mean mask is designed to further exclude the mismatched pixels interfered with by motion or illumination change in the training of depth and pose networks. This method is also used to exclude some trivial mismatched pixels in the training of the optical flow network. Maximum normalization is proposed for depth smoothness term to restrain depth degradation in textureless regions. In the occluded region, as depth and camera motion can provide more reliable motion estimation, they can be used to instruct unsupervised learning of optical flow. Our experiments in KITTI dataset demonstrate that the model based on three regions, full and explicit segmentation of the occlusion region, the rigid region, and the non-rigid region with corresponding unsupervised losses can improve performance on three tasks significantly. The source code is available at: https://github.com/guangmingw/DOPlearning.

LS-VO: Learning Dense Optical Subspace for Robust Visual Odometry Estimation

This work proposes a novel deep network architecture to solve the camera Ego-Motion estimation problem. A motion estimation network generally learns features similar to Optical Flow (OF) fields starting from sequences of images. This OF can be described by a lower dimensional latent space. Previous research has shown how to find linear approximations of this space. We propose to use an Auto-Encoder network to find a non-linear representation of the OF manifold. In addition, we propose to learn the latent space jointly with the estimation task, so that the learned OF features become a more robust description of the OF input. We call this novel architecture LS-VO. The experiments show that LS-VO achieves a considerable increase in performances in respect to baselines, while the number of parameters of the estimation network only slightly increases.

Evaluation of non-geometric methods for visual odometry

Visual Odometry (VO) is one of the fundamental building blocks of modern autonomous robot navigation and mapping. While most state-of-the-art techniques use geometrical methods for camera ego-motion estimation from optical flow vectors, in the last few years learning approaches have been proposed to solve this problem. These approaches are emerging and there is still much to explore. This work follows this track applying Kernel Machines to monocular visual ego-motion estimation. Unlike geometrical methods, learning-based approaches to monocular visual odometry allow issues like scale estimation and camera calibration to be overcome, assuming the availability of training data. While some previous works have proposed learning paradigms to VO, to our knowledge no extensive evaluation of applying kernel-based methods to Visual Odometry has been conducted. To fill this gap, in this work we consider publicly available datasets and perform several experiments in order to set a comparison baseline with traditional techniques. Experimental results show good performances of learning algorithms and set them as a solid alternative to the computationally intensive and complex to implement geometrical techniques.

A review on egomotion by means of differential epipolar geometry applied to the movement of a mobile robot

The estimation of camera egomotion is an old problem in computer vision. Since the 1980s, many approaches based on both the discrete and the differential epipolar constraint have been proposed. The discrete case is used mainly in self-calibrated stereoscopic systems, whereas the differential case deals with a single moving camera. This article surveys several methods for 3D motion estimation unifying the mathematics convention which are then adapted to the common case of a mobile robot moving on a plane. Experimental results are given on synthetic data covering more than 0.5 million estimations. These surveyed algorithms have been programmed and are available on the Internet.