Contour-Guided Feature Selection for Visual Relocalization

In complex urban environments, the GNSS/INS integration diverges over time when severe GNSS degradation occurs due to the harsh environment. Based on visual point features, the visual-inertial odometry system has been integrated with the GNSS to solve this problem. However, the accuracy of this system decreases when the GNSS and visual point features degrade simultaneously in the GNSS harsh environments and low-texture scenes. The artificial structures in complex urban environments render line features more reliable than point features. To further improve the positioning accuracy of the point-based fusion system, we propose a GNSS/INS/Vision integration method based on point and line features, where visual measurements in a sliding window and the GNSS position are used to reduce or eliminate the cumulative errors of the IMU state in the back end. In addition, visual features are merged at the front end via an adaptive threshold based on the line feature length. Two urban infield experiments are used to evaluate the effectiveness of the proposed method through improvements in the time consumption of line feature matching and position accuracy. The analysis demonstrates that the average time consumption is reduced by 50.03% and 82.19%, and the maximum position error is reduced by 42.56% and 55.48%, respectively in the two scenarios. Additional evaluations based on KAIST Urban 38 public datasets indicate that our proposed system achieves a better positioning accuracy than that of VINS-Fusion, which is a state-of-the-art GNSS/INS/Vision integration system that uses only point features.

In the study of RGB-D SLAM (Simultaneous Localization and Mapping), two types of primary visual features, point and line features, have been widely utilized to calculate the camera pose. As an RGB-D camera can capture RGB and depth information simultaneously, most RGB-D SLAM methods only utilize the 2D information within the point and line features. To obtain a higher accuracy camera pose and utilize the 2D and 3D information within points and lines better, a novel geometric constraint model of points and lines (PL-GM) using an RGB-D camera is proposed in this paper. Our contributions are threefold. Firstly, the 3D points and lines generated by an RGB-D camera combining with 2D point and line features are utilized to establish the PL-GM, which is different from most models of point-line SLAM (PL-SLAM). Secondly, in addition to the 2D re-projection error of point and line features, the constraint errors of 3D points and lines are constructed and minimized likewise, and then a unified optimization model based on PL-GM is extended to the bundle adjustment model (BA). Finally, extensive experiments have been performed on two public benchmark RGB-D datasets and a real scenario sequence. These experimental results demonstrate that our method achieves a comparable or better performance than the state-of-the-art SLAM methods based on point and line features, and point features.

Compared with point features, line features in the environment have more structural information. When indoor texture is not rich, making full use of the structural information of line features can improve the robustness and accuracy of simultaneous location and mapping algorithm. In this paper, we propose an improved monocular inertial indoor location algorithm considering point and line features. Firstly, the point features and line features in the environment are extracted, matched and parameterized, and then the inertial sensor is used to estimate the initial pose, and the tightly coupled method is adopted to optimize the observation error of the point and line features and the measurement error of the inertial sensor simultaneously in the back optimization to achieve accurate estimation of the pose of unmanned aerial vehicle. Finally, loop closure detection and pose graph optimization are used to optimize the pose in real time. The test results on public datasets show that the location accuracy of the proposed method is superior to 10 cm under sufficient light and texture conditions. The angle measurement accuracy is better than 0.05 rad, and the output frequency of positioning results is 10Hz, which effectively improves the accuracy of traditional visual inertial location method and meets the requirements of real-time.

RGB-D SLAM (Simultaneous Localization and Mapping) generally performs smoothly in a static environment. However, in dynamic scenes, dynamic features often cause wrong data associations, which degrade accuracy and robustness. To address this problem, in this paper, a new RGB-D dynamic SLAM method, PLD-SLAM, which is based on point and line features for dynamic scenes, is proposed. First, to avoid under-over segmentation caused by deep learning, PLD-SLAM combines deep learning for semantic information segmentation with the K-Means clustering algorithm considering depth information to detect the underlying dynamic features. Next, two consistency check strategies are utilized to check and filter out the dynamic features more reasonably. Then, to obtain a better practical performance, point and line features are utilized to calculate camera pose in the dynamic SLAM, which is also different from most published dynamic SLAM algorithms based merely on point features. The optimization model with point and line features is constructed and utilized to calculate the camera pose with higher accuracy. Third, enough experiments on the public TUM RGB-D dataset and the real-world scenes are conducted to verify the location accuracy and performance of PLD-SLAM. We compare our experimental results with several state-of-the-art dynamic SLAM methods in terms of average localization errors and the visual difference between the estimation trajectories and the ground-truth trajectories. Through the comprehensive comparisons with these dynamic SLAM schemes, it can be fully demonstrated that PLD-SLAM can achieve comparable or better performances in dynamic scenes. Moreover, the feasibility of camera pose estimation based on both point features and line features has been proven by the corresponding experiments from a comparison with our proposed PLD-SLAM only based on point features.

After using a terrestrial laser scanner to acquire building facade point clouds, the extraction of feature lines can simplify the expression of building objects, thereby contributing to the accurate construction of building facade geometric models. To address the problems of missing extraction and low accuracy in existing methods, this study proposes a feature line extraction method for building facade point clouds by exploring the different spatial topological relationship between feature and non‐feature points. The method comprises three steps: feature point extraction, feature line generation, and feature line merging and optimization. First, feature points are extracted using the convex hull distance value and relative angle, as defined in this study. Thereafter, feature points are clustered by combining the random sample consensus and region growing algorithms, and the feature lines are obtained by utilizing the iterative weighted least squares method based on the IGGIII weight function. Finally, the feature lines are merged and optimized using an endpoint search method to improve the discontinuity and missing common endpoints of the original feature lines. The experimental results obtained using simulated and measured point cloud data show that this method can accurately extract feature points, and the extracted feature lines obtained from building facade point clouds have better accuracy and completeness, which is more practical than the existing methods and can be used in many applications such as building facade measurement and urban three‐dimensional modeling.

In this paper, we propose the PLS-FUSION, where P stands for point features, L for line features, and S for stereo vision. Together, these components form a tightly-coupled stereo visual-inertial Simultaneous Localization and Mapping (SLAM) system that leverages both point and line features to enhance the robustness and accuracy of tracking. Compared with the classic SLAM methods which only use the point features, we extract both the point features and the line features of the stereo camera in the front end and use a modified Line Segment Detector (LSD) algorithm of PL-VINS to improve the speed of extracting the line features. We use Plücker coordinates and orthonormal representation to represent the line features and add re-projection errors of the line features between the stereo camera in the back end. Our proposal has been tested with several popular datasets (EuRoC and TUMVI) and in the real world with our flight platform based on PX4 and QGroundControl (QGC). The experiments validate that PLS-FUSION has a more robust performance than state-of-the-art methods such as PL-VINS and VINS-FUSION.

Traditional feature-based SLAM systems rely on point features in the environment to recover the camera pose and build an environmental map. With the in-depth research of scholars, in order to make up for the disadvantages of point features in a low texture environment, stereo visual SLAM systems that combine both points and line segments are proposed. Although the stereo visual SLAM systems that combine both point features and line features improve the accuracy of tracking and they also increase the computational burden of the computer and reduce the tracking efficiency. However, in the actual environment, the environment will not always be in a state of low texture, so our work considers that the line feature can be selectively used. We selectively use line features during the tracking process and use line features as a supplement to point features in a low-texture environment. The main content of our work is proposing an analysis method that analyzes the change of the environmental characteristics that have been tracked during the tracking process to make the SLAM system possible to make good use of point features and line features in the tracking process. We test our system on public datasets and compare our results with state-of-the-art methods. The test results show that our stereo visual SLAM method can obtain more accurate results than the stereo visual SLAM system that uses points and even the stereo visual SLAM system that combines both point features and line features.KeywordsStereo visual SLAMPoint featuresLine featuresTracking feature analytical method

PurposeThis paper aims to enable the unmanned aerial vehicles to inspect the surface condition of wind turbine in close range when the global positioning system signal is not reliable, and further improve its intelligence. So a visual-inertial odometry with point and line features is developed.Design/methodology/approachVisual front-end combining point and line features, as well as its purification strategies, are first presented to improve the robustness of feature tracking in low-textured scene and rapidity of segment detector. Additionally, the inertial measurement is integrated between keyframes as constrain to reduce tracking error existed in visual-only system. Second, the graph-based visual-inertial back-end is constructed. To parameterize line features effectively, the infinite line representation not sensitive to outdoor light is employed, in which Plücker and Cayley are selected for line re-projection and nonlinear optimization. Furthermore, Jacobians of the line re-projection errors are analytically derived for better accuracy.FindingsExperiments are performed in various scenes of the wind farm. The results demonstrate that the tight-coupled visual-inertial odometry with point and line features is more precise on all the samples than conventional algorithms in complex wind farm environments. Additionally, the constructed line feature map can be used in the following research for autonomous navigation.Originality/valueThe proposed visual-inertial odometry works robustly in strong electromagnetic interference, low-textured and illumination-change wind farm.

Visual Simultaneous Localization and Mapping(SLAM) technology is widely used in the autonomous navigation of mobile robots. However, in the face of increasingly complex application scenarios, there are still many problems that need to be solved, among which poor localization in weak texture environments is one of the most important problems. Therefore, in this study, to address the problem of large positioning errors caused by insufficient point features in weak texture environments, based on Oriented FAST and Rotated BRIEF Simultaneous Localization and Mapping 2(ORB-SLAM2), we optimize the estimation of the camera’s position information by extracting the line features in the environment and integrating the point and line features and realize a visual odometry that can simultaneously extract the matching point features and line features, which enhances the positioning accuracy and stability of the SLAM system in weak texture environments. Experiments are conducted on the The Technical University of Munich (TUM) dataset and real scenes, and the experimental results show that the constructed algorithm can effectively utilize the point and line feature information in the environment, and the localization accuracy is improved by about 30% in the indoor structured texture-free scene, about 10% in the indoor structured textured scene, and about 15% in the other indoor scenes. The experimental results demonstrate the effectiveness of the visual SLAM algorithm based on point and line features proposed in this paper.

Many RGB-D simultaneous localization and mapping (SLAM) systems solely rely on point features, suffering from performance degradation in weak-texture scenes where line features can be complementary. However, existing RGB-D SLAM methods fusing point and line features commonly fail to explicitly estimate the feature uncertainties, hindering an accurate evaluation of both reliabilities and contributions of different features. In this paper, an RGB-D SLAM approach termed PLUE-SLAM integrating the closed-form uncertainty estimation of point and line features is proposed. Specifically, PLUE-SLAM derives the uncertainties of point and line features by jointly considering image observations, depth measurements and triangulation. In the front-end tracking, feature uncertainties provide an adaptive weighting mechanism for feature fusion, and are further propagated to the uncertainties of the camera poses. The uncertainties of both features and poses are used to weight the cost function in local bundle adjustment (BA) optimization, and are subsequently updated. Finally, a loop closure detection method using the feature uncertainties is designed to improve the robustness of PLUE-SLAM. Extensive experiments on public datasets and in real-world environments demonstrate that PLUE-SLAM outperforms state-of-the-art visual SLAM systems. PLUE-SLAM is open-sourced to benefit the community<sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup>.

In this paper, a degeneracy avoidance method for a point and line based visual SLAM algorithm is proposed. Visual SLAM predominantly uses point features. However, point features lack robustness in low texture and illuminance variant environments. Therefore, line features are used to compensate the weaknesses of point features. In addition, point features are poor in representing discernable features for the naked eye, meaning mapped point features cannot be recognized. To overcome the limitations above, line features were actively employed in previous studies. However, since degeneracy arises in the process of using line features, this paper attempts to solve this problem. First, a simple method to identify degenerate lines is presented. In addition, a novel structural constraint is proposed to avoid the degeneracy problem. At last, a point and line based monocular SLAM system using a robust optical-flow based lien tracking method is implemented. The results are verified using experiments with the EuRoC dataset and compared with other state-of-the-art algorithms. It is proven that our method yields more accurate localization as well as mapping results.

The shape of the object is mainly described by feature points and lines. Since a feature point can be described by the intersection of two feature lines, feature lines are the key to determine the contour of the object. In this article, a novel method for the generation and regularization of point cloud feature line is presented, which consists of two main steps: extraction of the outline points according to the property of vectors distribution and cluster, feature points are sorted according to the vector deflection angle and distance and they are fitted using the improved cubic b-spline curve fitting algorithm. The performance of the proposed method is evaluated with both large and small point clouds acquired by terrestrial laser scanning devices in real-world scenes. The results show that the proposed method and the analysis of geometrical properties of neighborhoods (AGPN) method achieve very similar performance in the case of planar objects, accurately extracting the outline points of objects. However, in the presence of a curved surface, the proposed method significantly outperforms the existing methods in detecting outline points. The outlines are regularized by the improved cubic b-spline and it is superior to the traditional cubic b-spline curve fitting algorithm.

In order to recognize an arbitrary 3D object, it is often required to extract feature points and feature lines from its surface model. The feature points and feature lines include peaks, pits, ridge lines, and valley lines. In this paper, we present an efficient technique for finding the features from the triangular surface model of an arbitrary 3D object. Given a set of surface data points, we find, using the local adjustment technique, the triangular patches that best fit the surface of the object. For the resulting triangle-based surface model, unit normal vectors and side lengths of the triangular patches are used systematically to locate the feature points and lines of the surface. We present experimental results on simple objects with feature points and feature lines.

Autonomous navigation of mobile robots in complex environments is challenging. Solving the problems of inaccuracy localization and frequent tracking losses of mobile robots in challenging scenes is beyond the power of point-based visual simultaneous localization and mapping (vSLAM). This paper proposes a real-time and robust point-line based monocular visual inertial SLAM (VINS) system for mobile robots of smart cities towards 6G. To extract robust line features for tracking in challenging scenes, EDLines with adaptive gamma correction is adopted to fast extract a larger ratio of long line features among all extracted line features. A real-time line feature matching approach is proposed to track the extracted line features between adjacent frames without the need of computing descriptors. Compared with LSD and KNN matching method based on LBD descriptors, the proposed method runs three times faster. Furthermore, a tightly coupled sensor fusion optimization framework is constructed for accurate state estimation, which contains point-line feature reprojection errors and IMU residuals. By evaluating on public benchmark datasets, our VINS system has high localization accuracy, real-time performance and robustness compared with other advanced SLAM systems. Our VINS system enables mobile robots to locate accurately in smart cities with complex environments.

Visual-inertial odometry that uses point features only has poor localization performance in the low-texture environments, or even fails. The introduction of line features can improve this situation, because line features have richer scene structure information and can appropriately improve the positioning accuracy. We propose a real-time optimized tightly coupled visual-inertial algorithm, which based on point-line features. It can achieve a good balance in the positioning accuracy and the real-time performance. The front-end part proposes an algorithm for adjusting image brightness based on image preprocessing to increase the number of image feature extractions in low-light environments. In order to reduce the wrong segmentation of long line features, the least squares method is used to combine short line segments with similar directions and distances, thereby reducing the difficulty of matching line features. Experiments on the EuRoc dataset show that, under the same experimental environment, our algorithm has higher stability and positioning accuracy compared with the mainstream VIO algorithm based only on point features. At the same time, compared with PL-VINS which uses point-line features, our algorithm improves the positioning accuracy by 6%-7%.