Line Feature Matching Research Articles

Correlative Scan Matching Position Estimation Method by Fusing Visual and Radar Line Features

Millimeter-wave radar and optical cameras are one of the primary sensing combinations for autonomous platforms such as self-driving vehicles and disaster monitoring robots. The millimeter-wave radar odometry can perform self-pose estimation and environmental mapping. However, cumulative errors can arise during extended measurement periods. In particular scenes where loop closure conditions are absent and visual geometric features are discontinuous, existing loop detection methods based on back-end optimization face challenges. To address this issue, this study introduces a correlative scan matching (CSM) pose estimation method that integrates visual and radar line features (VRL-SLAM). By making use of the pose output and the occupied grid map generated by the front end of the millimeter-wave radar’s simultaneous localization and mapping (SLAM), it compensates for accumulated errors by matching discontinuous visual line features and radar line features. Firstly, a pose estimation framework that integrates visual and radar line features was proposed to reduce the accumulated errors generated by the odometer. Secondly, an adaptive Hough transform line detection method (A-Hough) based on the projection of the prior radar grid map was introduced, eliminating interference from non-matching lines, enhancing the accuracy of line feature matching, and establishing a collection of visual line features. Furthermore, a Gaussian mixture model clustering method based on radar cross-section (RCS) was proposed, reducing the impact of radar clutter points online feature matching. Lastly, actual data from two scenes were collected to compare the algorithm proposed in this study with the CSM algorithm and RI-SLAM.. The results demonstrated a reduction in long-term accumulated errors, verifying the effectiveness of the method.

EPLF-VINS: Real-Time Monocular Visual-Inertial SLAM With Efficient Point-Line Flow Features

This letter introduces an efficient visual-inertial simultaneous localization and mapping (SLAM) method using point and line features. Currently, point-based SLAM methods do not perform well in scenarios such as weak textures and motion blur. Many researchers have noticed the excellent properties of line features in space and have attempted to develop line-based SLAM systems. However, the vast computational effort of the line extraction and description matching process makes it challenging to guarantee the real-time performance of the whole SLAM system, and the incorrect line detection and matching limit the performance improvement of the SLAM system. In this letter, we improve the traditional line detection model by means of short-line fusion, line feature uniform distribution, and adaptive threshold extraction to obtain high-quality line features for constructing SLAM constraints. Based on the gray level invariance assumption and colinear constraint, we propose a line optical flow tracking method, which significantly improves the speed of line feature matching. In addition, a measurement model that is independent of line endpoints is presented for estimating line residuals. The experimental results show that our algorithm improves the efficiency of line feature detection and matching and localization accuracy.

Improved Simulation Preassembly Method Based on Line Feature Matching

Improved Simulation Preassembly Method Based on Line Feature Matching

Fast visual inertial odometry with point–line features using adaptive EDLines algorithm

In mainstream visual inertial odometry (VIO) systems, the method of positional solution by feature point extraction and matching in the image is widely used. However, the tracking accuracy of point features is dependent on the texture richness in the environment. Although many existing algorithms introduce line features in the front end to improve the system’s environmental adaptability, most of them sacrifice system real-time in exchange for higher positioning accuracy. The extraction and matching of line features often require more time, thus failing to meet the real-time requirements of the system for localization. In this paper, we therefore propose a fast VIO fused with point and line features, which enables the system to maintain a high level of positioning robustness in dim and changing light environments with low time cost. The point–line features VIO algorithm is based on adaptive thresholding of EDLines. By adding an adaptive thresholding component to the EDLines algorithm, the robustness of line feature extraction is enhanced to better adapt to changes in ambient lighting. The time needed for line feature extraction is also significantly reduced. A line feature matching algorithm based on geometric information and structural similarity is proposed, which enables fast and accurate line feature matching. The algorithm is compared with point-line visual-inertial odometry and monocular visual-inertial state estimator algorithms on the European robotics challenge dataset and real-world scenes. Many experiments prove that the algorithm has improved in both real time and accuracy.

PLI-VINS: Visual-Inertial SLAM Based on Point-Line Feature Fusion in Indoor Environment.

In indoor low-texture environments, the point feature-based visual SLAM system has poor robustness and low trajectory accuracy. Therefore, we propose a visual inertial SLAM algorithm based on point-line feature fusion. Firstly, in order to improve the quality of the extracted line segment, a line segment extraction algorithm with adaptive threshold value is proposed. By constructing the adjacent matrix of the line segment and judging the direction of the line segment, it can decide whether to merge or eliminate other line segments. At the same time, geometric constraint line feature matching is considered to improve the efficiency of processing line features. Compared with the traditional algorithm, the processing efficiency of our proposed method is greatly improved. Then, point, line, and inertial data are effectively fused in a sliding window to achieve high-accuracy pose estimation. Finally, experiments on the EuRoC dataset show that the proposed PLI-VINS performs better than the traditional visual inertial SLAM system using point features and point line features.

A New Visual Inertial Simultaneous Localization and Mapping (SLAM) Algorithm Based on Point and Line Features

In view of traditional point-line feature visual inertial simultaneous localization and mapping (SLAM) system, which has weak performance in accuracy so that it cannot be processed in real time under the condition of weak indoor texture and light and shade change, this paper proposes an inertial SLAM method based on point-line vision for indoor weak texture and illumination. Firstly, based on Bilateral Filtering, we apply the Speeded Up Robust Features (SURF) point feature extraction and Fast Nearest neighbor (FLANN) algorithms to improve the robustness of point feature extraction result. Secondly, we establish a minimum density threshold and length suppression parameter selection strategy of line feature, and take the geometric constraint line feature matching into consideration to improve the efficiency of processing line feature. And the parameters and biases of visual inertia are initialized based on maximum posterior estimation method. Finally, the simulation experiments are compared with the traditional tightly-coupled monocular visual–inertial odometry using point and line features (PL-VIO) algorithm. The simulation results demonstrate that the proposed an inertial SLAM method based on point-line vision for indoor weak texture and illumination can be effectively operated in real time, and its positioning accuracy is 22% higher on average and 40% higher in the scenario that illumination changes and blurred image.

Simultaneous Localization and Mapping Based on Point and Line Feature Matching

Simultaneous Localization and Mapping Based on Point and Line Feature Matching

Robust Pose Estimation via Hybrid Point and Twin Line Reprojection for RGB-D Vision Navigation

Pose estimation is a crucial technique to achieve robot localization using natural features for an RGB-D vision navigation system. However, it still suffers from the impoverishment of robustness and accuracy in complex workspaces. A robust real-time pose estimation approach using hybrid point and twin line reprojection is proposed to allow accurate localization for a vision-based robot under various challenging scenarios including low-texture, low-structure, occlusion, and viewpoint and illumination changes. Firstly, an improved adaptive threshold (IAT)-based oriented fast and rotated Brief (ORB) feature detection method is developed to extract sufficient IAT-ORB point features, which are matched by a multi-stage KNN-RTC-CS-RANSAC (KRCR) refinement feature matching strategy to generate reliable point matches. Then a robust EDLines line feature matching method is developed by integrating dynamic matching (DM) and median absolute deviation (MAD) to obtain fine line matches. Furthermore, the hybrid point reprojection errors are built from the fusion of 3D-2D and 3D-3D point pairs, and the twin line reprojection errors are constructed by integrating virtual right-eye lines generated by depth measurement, which both are incorporated into a novel unified errors optimization model to achieve optimal motion estimation by minimizing the reprojection errors of points and lines concurrently. Finally, the extensive experiments verify the robustness, accuracy, and real-time performance of the proposed approach on the public datasets and our real indoor scenarios. The quantitative results demonstrate that our proposed approach gains more accurate and stable localization than other state-of-the-art methods under challenging environments.

An automatic method for matching salient structures in optical remote sensing images

ABSTRACT Feature matching in optical remote sensing images usually suffers from significant intensity differences and background variations. In this paper, an effective image matching approach is proposed to match salient structures in optical remote sensing images. The proposed method consists of three main steps. In the first step, EDLines algorithm is implemented to extract straight lines from two corresponding images, and these lines are further processed and divided into line segments for feature matching. In the second step, a robust descriptor namely line segment context (LSC) descriptor is computed for each line segment. An iterative procedure that combines line segment description and feature matching is presented for accurate matching. Finally, the consistency check is implemented to further refine the matching results. We conduct the experiments on pairs of optical remote sensing images, and experimental results show that the proposed method can automatically and robustly generate line segment features and achieve favourable performance compared to existing image matching approaches.

ROBUST ESTIMATION IN ROBOT VISION AND PHOTOGRAMMETRY: A NEW MODEL AND ITS APPLICATIONS

Abstract. Robust estimation (RE) is a fundamental issue in robot vision and photogrammetry, which is the theoretical basis of geometric model estimation with outliers. However, M-estimations solved by iteratively reweighted least squares (IRLS) are only suitable for cases with low outlier rates (< 50%); random sample consensus (RANSAC) can only obtain approximate solutions. In this paper, we propose an accurate and general RE model that unifies various robust costs into a common objective function by introducing a “robustness-control” parameter. It is a superset of typical least-squares, l1-l2, Cauchy, and Geman-McClure estimates. We introduce a parameter-decreasing strategy into the IRLS to optimize our model, called adaptive IRLS. The adaptive IRLS begins with a least-squares estimate for initialization. Then, the “robustness-control” parameter is decreased along with iterations so that the proposed model acts as different robust loss functions and has different degrees of robustness. We also apply the proposed model in several important tasks of robot vision and photogrammetry, such as line fitting, feature matching, image orientation, and point cloud registration (scan matching). Extensive simulated and real experiments show that the proposed model is robust to more than 80% outliers and preserves the advantages of M-estimations (fast and optimal). Our source code will be made publicly available in https://ljy-rs.github.io/web.

A Fast Vision-inertial Odometer Based on Line Midpoint Descriptor

Visual Simultaneous Localization and Mapping are essential technologies to realize the autonomous movement of vehicles. Visual-Inertial Odometry (VIO) is often used as the front-end of VSLAM because of its rich information, lightweight, and robustness. This article proposes the FPL-VIO, an optimization-based Fast Vision-Inertial Odometer with Points and Lines. Traditional VIO mostly uses points as landmarks; meanwhile, most of the geometrical structure information is ignored. Therefore, the accuracy will be jeopardized under motion blur and texture-less area. Some researchers improve accuracy by adding lines as landmarks in the system. However, almost all of them use Line Segment Detector (LSD) and Line Band Descriptor (LBD) in line processing, which is very time-consuming. This article first proposes a fast line feature description and matching method based on the midpoint and compares the three line detection algorithms of LSD, FLD (Fast Line Detector), and EDLines (Edge Drawing Line). Then, the measurement model of the line is introduced in detail. Finally, FPL-VIO is proposed by adding the above method to VINS-Mono, an optimization-based Fast Vision-Inertial Odometer with lines described by midpoint and points. Compared with PL-VIO, the line processing efficiency of FPL-VIO is increased by 3-4 times while ensuring the same accuracy.

Mobile Robot Self-Localization Using Omnidirectional Vision with Feature Matching from Real and Virtual Spaces

This paper presents a novel self-localization technique for mobile robots based on image feature matching from omnidirectional vision. The proposed method first constructs a virtual space with synthetic omnidirectional imaging to simulate a mobile robot equipped with an omnidirectional vision system in the real world. In the virtual space, a number of vertical and horizontal lines are generated according to the structure of the environment. They are imaged by the virtual omnidirectional camera using the catadioptric projection model. The omnidirectional images derived from the virtual and real environments are then used to match the synthetic lines and real scene edges. Finally, the pose and trajectory of the mobile robot in the real world are estimated by the efficient perspective-n-point (EPnP) algorithm based on the line feature matching. In our experiments, the effectiveness of the proposed self-localization technique was validated by the navigation of a mobile robot in a real world environment.

A Multi-Feature Fusion Slam System Attaching Semantic In-Variant to Points and Lines.

The traditional simultaneous localization and mapping (SLAM) system uses static points of the environment as features for real-time localization and mapping. When there are few available point features, the system is difficult to implement. A feasible solution is to introduce line features. In complex scenarios containing rich line segments, the description of line segments is not strongly differentiated, which can lead to incorrect association of line segment data, thus introducing errors into the system and aggravating the cumulative error of the system. To address this problem, a point-line stereo visual SLAM system incorporating semantic invariants is proposed in this paper. This system improves the accuracy of line feature matching by fusing line features with image semantic invariant information. When defining the error function, the semantic invariant is fused with the reprojection error function, and the semantic constraint is applied to reduce the cumulative error of the poses in the long-term tracking process. Experiments on the Office sequence of the TartanAir dataset and the KITTI dataset show that this system improves the matching accuracy of line features and suppresses the cumulative error of the SLAM system to some extent, and the mean relative pose error (RPE) is 1.38 and 0.0593 m, respectively.

Robust line feature matching based on pair-wise geometric constraints and matching redundancy

This paper presents a novel method for matching line segments in images based on pair-wise geometric constraints and matching redundancy. In this study, pairs of line segments satisfying angle and distance constraints are used as matching primitives. To ensure that each extracted line segment is paired with another line segment, the search region of each line segment is gradually grown until it is paired. Initial pair-to-pair correspondences between two images are established using four pair-wise constraints; next, line-to-line correspondences are obtained. To effectively solve the matching conflict in the results, a method of recording the result of line pair matching based on a double-layer matrix is proposed. Based on the double-layer matrix, an effective checking method for the line matching results based on the collinearity constraint and matching redundancy is presented. It fully utilizes redundancy information and considers the collinearity of fragmented line segments. Further, it can effectively separate correct and incorrect matches from the one-to-many, many-to-one, and many-to-many matching results. The proposed method was tested on 12 image pairs from a benchmark of matched-lines, and compared with other state-of-the-art methods. The results demonstrate the superiority of the proposed method due to its higher accuracy and greater recall in challenging cases.

Adaptive Covariance Estimation Method for LiDAR-Aided Multi-Sensor Integrated Navigation Systems

The accurate estimation of measurements covariance is a fundamental problem in sensors fusion algorithms and is crucial for the proper operation of filtering algorithms. This paper provides an innovative solution for this problem and realizes the proposed solution on a 2D indoor navigation system for unmanned ground vehicles (UGVs) that fuses measurements from a MEMS-grade gyroscope, speed measurements and a light detection and ranging (LiDAR) sensor. A computationally efficient weighted line extraction method is introduced, where the LiDAR intensity measurements are used, such that the random range errors and systematic errors due to surface reflectivity in LiDAR measurements are considered. The vehicle pose change is obtained from LiDAR line feature matching, and the corresponding pose change covariance is also estimated by a weighted least squares-based technique. The estimated LiDAR-based pose changes are applied as periodic updates to the Inertial Navigation System (INS) in an innovative extended Kalman filter (EKF) design. Besides, the influences of the environment geometry layout and line estimation error are discussed. Real experiments in indoor environment are performed to evaluate the proposed algorithm. The results showed the great consistency between the LiDAR-estimated pose change covariance and the true accuracy. Therefore, this leads to a significant improvement in the vehicle’s integrated navigation accuracy.

Automatic Georeferencing of a Heritage of old analog aerial Photographs

Abstract. Historical photographs become widely used in geographical and environmental applications. Their enhancement involves converting them into georeferenced data, such as orthoimages or digital models. However no ground control points are available unlike in current image processing, and many problems such as image noise, landscape modifications, perspective distortion and unknown sensor calibration prevent automatic tie-point retrieval with current orthoimages. That is why photograph georeferencing remains a manual and time-consuming task. A novel method is presented in this paper to register photographs with current topographic database using line feature matching. Indeed, geometrical considerations only let avoid high radiometric difference issues when dealing with current orthoimages. Besides topographic database use lets selecting stable through time features, such as road network and historical buildings. A multi-scale approach allows very coarse georeferencing initialization, which can be set manually by a minimum number of ground control points per image set. At each scale an iterative processing improves the line matching and the registration model estimation at the same time. Finally, building integration makes registration more reliable for off-ground objects. Results are promising as georeferencing is much improved and its estimation converges in all test cases.

尺度、旋转以及亮度稳健的高分辨率影像直线特征匹配

A line feature matching method for high-resolution images was proposed to improve the low significant level of a point feature and to overcome the matching shortage between weak texture images.Firstly,the edges of images were extracted and tracked to fit straight-lines.All straight-lines were classified into two groups: long-lines and short-lines.The long-lines were matched based on the direction relationship primarily.Then,the relationship-descriptors of short-lines were constructed using the angle and the Euclidean distance between the short-line and long correspondences.Finally,short-lines were matched according to the similarity of their relationship-descriptors.The experimental results demonstrate that the proposed line matching algorithm is robust for the scale,rotation and illumination.As all the lines have corresponding linear equations in the same image,the image feature has higher significant level and can avoid the mismatching.The probability of correct matches of the algorithm exceeds 90% and its root mean square error has achieved sub-pixel level.The performance of the proposed algorithm is better than that of the point-based method,especially in weak texture areas.

Change detection based on DSM and image features in urban areas

On the basis of stereo image analysis, the change detection of man-made objects in urban areas is introduced. Information of the height of man-made objects can be applied to reinforce their change detection. By comparison between the new and old DSMs, the changed regions are extracted. However, our aim is to detect changes of man-made objects in urban area and further in the potential areas by the means of line-feature matching and gradient direction histogram. The experiments based on the aerial images from Japan have proven that the algorithm is correct and efficient.

A Line Feature Matching Technique Based on an Eigenvector Approach

In this paper, we propose a new eigenvector-based line feature matching algorithm, which is invariant to the in-plane rotation, translation, and scale. First, in order to reduce the number of possible matches, we use a preliminary correspondence test that generates a set of finite candidate models, by restricting combinations of line features in the input image. This approach resolves an inherent problem relating to ordering and correspondence in an eigenvector/modal approach. Second, we employ the modal analysis, in which the Gaussian weighted proximity matrices for reference and candidate models are constructed to record the relative distance and angle information between line features for each model. Then, the modes of the proximity matrices of the two models are compared to yield the dissimilarity measure, which describes the quantitative degree of the difference between the two models. Experimental results for synthetic and real images show that the proposed algorithm performs matching of the line features with affine variation fast and efficiently and provides the degree of dissimilarity in a quantitative way.

Two novel characteristics in palmprint verification: datum point invariance and line feature matching

As the first attempt of automatic personal identification by palmprint, in this paper, two novel characteristics, datum point invariance and line feature matching, are presented in palmprint verification. The datum points of palmprint, which have the remarkable advantage of invariable location, are defined and their determination using the directional projection algorithm is developed. Then, line feature extraction and line matching are proposed to detect whether a couple of palmprints are from the same palm. Various palmprint images have been tested to illustrate the effectiveness of the palmprint verification with both characteristics.