Textureless Scenes Research Articles

Robust depth-verified RGB-D visual odometry with structural regularities for indoor environments

This paper proposes an RGB-D visual odometry method that leverages point, line, plane features and Manhattan structures to achieve robust frame tracking and precise pose estimation, especially in textureless scenes. A validation method is introduced that ensures accurate frame-to-frame rotation estimation by comparing rotation angles computed from multiple Manhattan structures. Depth verification methods involving parameter fitting and outlier removal for point, line, and plane features are implemented by investigating the covariance of sensor depth measurements. We also employ local bundle adjustment in the local mapping thread to refine keyframe poses and landmarks. Comprehensive ablation studies confirm the effectiveness of our contributions. Experimental results on public datasets demonstrate that our method achieves obvious advantages in accuracy and robustness while maintaining real-time performance.

TopicFM: Robust and Interpretable Topic-Assisted Feature Matching

This study addresses an image-matching problem in challenging cases, such as large scene variations or textureless scenes. To gain robustness to such situations, most previous studies have attempted to encode the global contexts of a scene via graph neural networks or transformers. However, these contexts do not explicitly represent high-level contextual information, such as structural shapes or semantic instances; therefore, the encoded features are still not sufficiently discriminative in challenging scenes. We propose a novel image-matching method that applies a topic-modeling strategy to encode high-level contexts in images. The proposed method trains latent semantic instances called topics. It explicitly models an image as a multinomial distribution of topics, and then performs probabilistic feature matching. This approach improves the robustness of matching by focusing on the same semantic areas between the images. In addition, the inferred topics provide interpretability for matching the results, making our method explainable. Extensive experiments on outdoor and indoor datasets show that our method outperforms other state-of-the-art methods, particularly in challenging cases.

Semantic closed-loop based visual mapping algorithm for automated valet parking

Simultaneous localization and mapping (SLAM) is one of the core technologies to realize automated valet parking (AVP). Currently, advanced visual feature-based SLAM systems suffer from feature extraction failure and tracking loss due to the constraints of textureless scenes, unclear illumination, and dynamic conditions. To address these problems, this paper proposes a visual SLAM algorithm based on a semantic closed-loop detection algorithm using surround-view cameras and inertial measurement units (IMU) as sensors. The algorithm combines semantic features and the idea of inverse index to improve the traditional keyframes selection methods and the closed-loop detection algorithms, effectively avoiding the tedious and complicated feature point matching and improving the computational efficiency of the computer. Experiments show that the algorithm in this paper achieves better results in terms of precision and recall, absolute trajectory error (ATE), and relative pose error (RPE), and can meet the demand for SLAM and subsequent navigation in indoor parking lots.

Line-based visual odometry using local gradient fitting

Visual odometry aims to estimate the relative pose between frames, which is a fundamental task for visual SLAM. In this paper, we present a novel line-based visual odometry (VO) algorithm that fully utilizes the characteristic of line to estimate the projected line of adjacent frame by minimizing the local gradient fitness evaluation. In contrast to the current feature-based or line-based visual odometry, we don′t need to explicitly match points or lines of two frames, which is non-trivial and inaccurate in challenging scenarios such as texture-less scenes. In our method, the projected line is calculated simultaneously with the local gradient fitting function of pose estimation based on the constraint that the orientation of the projected line should be perpendicular to the gradient orientation of pixels of its local regions. The proposed method is more robust and reliable than other line-based VO since it fully uses the pixel orientations in the local regions to estimate the projected line and relative pose. We evaluate our method on the real-world RGB-D dataset and synthetic benchmark dataset. Experimental results show that our method achieves the state-of-the-art algorithms in indoors scenes, especially in texture-less scenes.

3D Curve Creation on and Around Physical Objects With Mobile AR.

The recent advance in motion tracking (e.g., Visual Inertial Odometry) allows the use of a mobile phone as a 3D pen, thus significantly benefiting various mobile Augmented Reality (AR) applications based on 3D curve creation. However, when creating 3D curves on and around physical objects with mobile AR, tracking might be less robust or even lost due to camera occlusion or textureless scenes. This motivates us to study how to achieve natural interaction with minimum tracking errors during close interaction between a mobile phone and physical objects. To this end, we contribute an elicitation study on input point and phone grip, and a quantitative study on tracking errors. Based on the results, we present a system for direct 3D drawing with an AR-enabled mobile phone as a 3D pen, and interactive correction of 3D curves with tracking errors in mobile AR. We demonstrate the usefulness and effectiveness of our system for two applications: in-situ 3D drawing, and direct 3D measurement.

Enhanced Spinning Parallelogram Operator Combining Color Constraint and Histogram Integration for Robust Light Field Depth Estimation

Accurate depth estimation is an essential part of most light field applications. Previous approach Spinning Parallelogram Operator (SPO) achieves robust depth estimation results in noisy and occluded scenes by fitting lines in Epipolar Plane Images (EPIs). However, only using histogram distances, SPO cannot handle complex occlusions and is sensitive to different parameters. In this letter, we propose enhanced Spinning Parallelogram Operator utilizing color constraint and histogram integration (SPO-CH) to address the above problems. The color consistency between different views is first introduced to exclude occlusions when measuring the histogram distances of occluded areas. We then design a novel Gaussian integration histogram that combines information from other adjacent bins, which improves the performances of noisy and texture-less scenes. Experimental results show that the proposed method effectively improves the depth estimation results. Comparisons with other state-of-the-art approaches show that the proposed method achieves better results for both synthetic and real-world images.

A Robust RGB-D SLAM System With Points and Lines for Low Texture Indoor Environments

High-performance 6D pose estimation and dense 3D mapping with an RGB-D camera has recently attracted substantial research attention since this type of camera can simultaneously capture RGB and depth information. However, there is an unresolved problem in estimating pose and generating highly accurate 3D maps from challenging indoor scenes. This paper presents a real-time simultaneous localization and mapping (SLAM) system based on the RGB-D camera for indoor mobile robots. Our contributions are fourfold. First, we propose a complete high-accuracy SLAM system based on a combination of information from points and lines, which differs from most solutions that rely on only point features. Second, we propose a novel pose solver to handle point and line correspondences, in which a line-based inliers refinement (LBIR) algorithm is proposed to remove outliers. Third, we construct a unified optimization model to concurrently minimize point and line reprojection errors, and extend it to the bundle adjustment (BA) method. Fourth, extensive experiments demonstrate the robustness, accuracy, and real-time performance of the proposed system on public TUM datasets and real world scenes. The empirical results show that the proposed system achieves a comparable or better performance than state-of-the-art methods. Notably, our system can operate in nearly texture-less scenes, while other methods are prone to failure.

Visual Semantic Landmark-Based Robust Mapping and Localization for Autonomous Indoor Parking.

Autonomous parking in an indoor parking lot without human intervention is one of the most demanded and challenging tasks of autonomous driving systems. The key to this task is precise real-time indoor localization. However, state-of-the-art low-level visual feature-based simultaneous localization and mapping systems (VSLAM) suffer in monotonous or texture-less scenes and under poor illumination or dynamic conditions. Additionally, low-level feature-based mapping results are hard for human beings to use directly. In this paper, we propose a semantic landmark-based robust VSLAM for real-time localization of autonomous vehicles in indoor parking lots. The parking slots are extracted as meaningful landmarks and enriched with confidence levels. We then propose a robust optimization framework to solve the aliasing problem of semantic landmarks by dynamically eliminating suboptimal constraints in the pose graph and correcting erroneous parking slots associations. As a result, a semantic map of the parking lot, which can be used by both autonomous driving systems and human beings, is established automatically and robustly. We evaluated the real-time localization performance using multiple autonomous vehicles, and an repeatability of 0.3 m track tracing was achieved at a 10 kph of autonomous driving.

Robust and Efficient CPU-Based RGB-D Scene Reconstruction.

3D scene reconstruction is an important topic in computer vision. A complete scene is reconstructed from views acquired along the camera trajectory, each view containing a small part of the scene. Tracking in textureless scenes is well known to be a Gordian knot of camera tracking, and how to obtain accurate 3D models quickly is a major challenge for existing systems. For the application of robotics, we propose a robust CPU-based approach to reconstruct indoor scenes efficiently with a consumer RGB-D camera. The proposed approach bridges feature-based camera tracking and volumetric-based data integration together and has a good reconstruction performance in terms of both robustness and efficiency. The key points in our approach include: (i) a robust and fast camera tracking method combining points and edges, which improves tracking stability in textureless scenes; (ii) an efficient data fusion strategy to select camera views and integrate RGB-D images on multiple scales, which enhances the efficiency of volumetric integration; (iii) a novel RGB-D scene reconstruction system, which can be quickly implemented on a standard CPU. Experimental results demonstrate that our approach reconstructs scenes with higher robustness and efficiency compared to state-of-the-art reconstruction systems.

Visual-inertial SLAM method based on optical flow in a GPS-denied environment

PurposeThis study aims to present a visual-inertial simultaneous localization and mapping (SLAM) method for accurate positioning and navigation of mobile robots in the event of global positioning system (GPS) signal failure in buildings, trees and other obstacles.Design/methodology/approachIn this framework, a feature extraction method distributes features on the image under texture-less scenes. The assumption of constant luminosity is improved, and the features are tracked by the optical flow to enhance the stability of the system. The camera data and inertial measurement unit data are tightly coupled to estimate the pose by nonlinear optimization.FindingsThe method is successfully performed on the mobile robot and steadily extracts the features on low texture environments and tracks features. The end-to-end error is 1.375 m with respect to the total length of 762 m. The authors achieve better relative pose error, scale and CPU load than ORB-SLAM2 on EuRoC data sets.Originality/valueThe main contribution of this study is the theoretical derivation and experimental application of a new visual-inertial SLAM method that has excellent accuracy and stability on weak texture scenes.

3D SLAM in texture-less environments using rank order statistics

SUMMARYWe present a real time 3D SLAM system for texture-less scenes using only depth information provided by a low cost RGB-D sensor. The proposed method is based on a novel informative sampling scheme that extracts points carrying the most useful 3D information for registration. The aim of the proposed sampling technique is to informatively sample a point cloud into a subset of points based on their 3D information. The flatness of a point is measured by applying a rank order statistics based robust segmentation method to surface normals in its local vicinity. The extracted keypoints from sequential frames are then matched and a rank order statistics based robust estimator is employed to refine the matches and estimate a rigid-body transformation between the frames. Experimental evaluations show that the proposed keypoint extraction method is highly repeatable and outperforms the state of the art methods in terms of accuracy and repeatability. We show that the performance of the registration algorithm is also comparable to other well-known methods in texture-less environments.