Feature Point Matching Research Articles

Intelligent vehicle visual pose estimation algorithm based on deep learning and parallel computing for dynamic scenes

In view of the disadvantages of the existing pose estimation algorithm, which has low real-time performance and the positioning accuracy will be greatly reduced in dynamic scene, a compound deep learning and parallel computing algorithm (DP-PE) is proposed. The detection algorithm based on deep learning is used to detect dynamic objects in the environment, and the dynamic feature points are removed before the matching of feature points to reduce the impact of dynamic objects on the positioning accuracy; A method for distinguishing “pseudo-dynamic objects” is proposed to solve the problem that the stationary vehicles and pedestrians in the environment are regarded as dynamic objects. The parallel computing framework for feature point extraction and matching is established on CPU-GPU heterogeneous platform to speed up DP-PE; In the localization part of DP-PE, we propose a 3D interior point detection strategy to achieve parallel search of map points, and the saturated linear kernel function is used to act on reprojection error to realize the parallelization of pose optimization. We verify the algorithm on KITTI dataset, the experimental results show that average speedup ratio of feature point extraction and matching is 6.5 times, and the overall computational efficiency of DP-PE is about 7 times higher than that before acceleration, which can realize high precision and efficient pose estimation in dynamic scene.

Self-Supervised Robust Feature Matching Pipeline for Teach and Repeat Navigation.

The performance of deep neural networks and the low costs of computational hardware has made computer vision a popular choice in many robotic systems. An attractive feature of deep-learned methods is their ability to cope with appearance changes caused by day–night cycles and seasonal variations. However, deep learning of neural networks typically relies on large numbers of hand-annotated images, which requires significant effort for data collection and annotation. We present a method that allows autonomous, self-supervised training of a neural network in visual teach-and-repeat (VT&R) tasks, where a mobile robot has to traverse a previously taught path repeatedly. Our method is based on a fusion of two image registration schemes: one based on a Siamese neural network and another on point-feature matching. As the robot traverses the taught paths, it uses the results of feature-based matching to train the neural network, which, in turn, provides coarse registration estimates to the feature matcher. We show that as the neural network gets trained, the accuracy and robustness of the navigation increases, making the robot capable of dealing with significant changes in the environment. This method can significantly reduce the data annotation efforts when designing new robotic systems or introducing robots into new environments. Moreover, the method provides annotated datasets that can be deployed in other navigation systems. To promote the reproducibility of the research presented herein, we provide our datasets, codes and trained models online.

FEATURE FUSION-BASED FEATURE POINT EXTRACTION AND MATCHING OF THE THREE-DIMENSIONAL HUMAN VERTEBRAE MODEL

In the positioning and registration of a three-dimensional human vertebrae model, the main difficulty encountered is the extraction and matching of the model feature points. To accurately locate the vertebrae, it is necessary to optimize the existing feature point extraction and matching method. In this context, the present study was envisaged to achieve the purpose of accurate extraction and matching by integrating the features of curvature and Euclidean distance. It was found that this method removed the redundant point cloud data in the 3D model, thereby effectively improving the matching rate. Initially, the feature points were manually picked as the circle center, and the mean value ([Formula: see text]) of average curvature ([Formula: see text]) of all points in the specified infinitesimal radius ([Formula: see text]) sphere neighborhood were calculated. When [Formula: see text], it implied the point [Formula: see text] tended to be flat relative to the whole point cloud surface, and was not regarded as a feature point. When [Formula: see text], the point [Formula: see text] exhibited a greater curvature relative to the whole point cloud surface, that is, a more prominent point. In this case, point [Formula: see text] met the basic conditions for becoming a feature point and was included in the candidate feature point set. Subsequently, the Euclidean distance between the points in the candidate feature point set and the manually picked circle center points was calculated individually. The smaller the Euclidean distance between the two points, the higher the degree of feature similarity between the two points, and the stronger the matching of feature points. Therefore, the manually picked points were replaced with the candidate feature point with the closest distance that accurately reflected the change of the local geometric characteristics of the point. It was found that the accuracy of the improved method of the feature point extraction and matching was about 18% higher than that before the improvement, which verifies the effectiveness and robustness of the proposed method.

Research on Postproduction of Film and Television Based on Computer Multimedia Technology

In the postproduction of film and television, the fast movement of objects or camera shake will lead to an excessive selection of key frames, resulting in information redundancy and affecting the stability of postproduction videos and video-editing effects. Aiming at this problem, this paper proposes a method of film and television postproduction based on digital media technology. Firstly, the film and video shots are segmented to find out the changing scenes; secondly, the bottom feature vector of each image is formed according to the color space entropy, motion vector, and motion region, and the video key frame is extracted based on digital media technology; finally, the pixel label information is established in the key frame, and the feature points located in the foreground are removed, while the time feature matching points and spatial feature matching points in the background are left, so as to achieve dynamic video mosaic. The experimental results show that the film and television postproduction method based on digital media technology has strong advantages compared with other methods, which can improve the stability score of video and reduce the average stitching error.

High-Precision Registration of Lunar Global Mapping Products Based on Spherical Triangular Mesh

Lunar global mapping products provide a solid data foundation for lunar scientific research and exploration. As the widespread geometric inconsistencies among multi-source mapping products seriously affect the synergistic use of the products, high-precision registration of multiple lunar global products is critical, and it is highly challenging due to large coverage and complex local geometric inconsistencies. In this research, we propose a spherical triangular-mesh-based method for high-precision registration of lunar global mapping products, which involves four steps: data preprocessing, feature point extraction and matching, spherical Delaunay triangulation, and geometric correction with spherical barycentric coordinates. This global registration method avoids map projection distortions by using spherical coordinates directly, and achieves high precision by confining the geometric models to spherical triangular facets. Experiments are conducted using two groups of different types of mapping products to verify the proposed method quantitatively and qualitatively. The results show that the geometric inconsistencies are reduced from hundreds of pixels to sub-pixel level globally after the registration, demonstrating the effectiveness of the proposed method.

An improved synthetic aperture radar‐scale invariant feature transform algorithm for interferometric imaging radar altimeter image registration

Interferometric imaging radar altimeter (InIRA) integrating synthetic aperture radar (SAR) and synthetic aperture radar interferometry (InSAR) is a new type of radar altimeter. InIRA will be widely adopted in marine scientific research and surface water topography measurements. Owing to the geometric distortion of InIRA images, there are difficulties in feature point extraction and matching, making InIRA registration challenging. Here, an improved SAR-scale invariant feature transform (SIFT, SAR-SIFT) algorithm is proposed to address these difficulties. First, a modified Harris non-linear scale space is constructed using the Harris function and non-linear scale space, and extracted the feature points according to the SAR-SIFT algorithm. Second, the orbital parameters of InIRA are used to calculate the geographic coordinates of each feature point by Hermit interpolation. Subsequently, a geographic-coordinate-based method is proposed to match the feature points. The results show that the proposed algorithm gained more correct matches and improved the registration accuracy by 13% and 16.7% compared with the SIFT and SAR-SIFT algorithms, respectively, and was 18% more time efficient than the SAR-SIFT algorithm. The experiments demonstrate that the proposed algorithm outperforms the SAR-SIFT algorithm for InIRA image registration in terms of accuracy and efficiency.

Approach to breech face impression comparison based on the robust estimation of a correspondence function

Forensic firearm analysis concerns an attempt to determine if ammunition is associated with a specific firearm based on tool-marks produced by it. A feature-based method using the Scale Invariant Feature Transform (SIFT) and RANdom SAmple Consensus (RANSAC) integration algorithm had been suggested to allow the automated comparison of breech face impressions. In this paper, an estimation method is proposed to establish a correspondence function among the features of comparison impression pairs, aiming to further improve the robustness and repeatability of automated feature matching. During the application of the iterative establishment algorithm, the Support Vector Regression (SVR) method is repeated to estimate the correspondence function based on current feature correspondences, and a robust weighting method excludes egregious outliers among putative correspondences by updating additional weightings. Moreover, the consistency detection method is adopted to overcome the over-fitting problem in SVR. Validation tests of the proposed method are conducted on three sets of cartridge case’s breech face impressions; namely the Fadul set consisting of 40 cartridge cases ejected from 10 Ruger P95PR15 pistols, the Weller sets containing 95 cartridge cases obtained from 11 Ruger P95DC firearms and the Lightstone set containing 30 cartridge cases from 10 SW40VE S&W Sigma pistol slides. Test results show that most known matching (KM) pairs possess no less than 20 matching feature points while the non-matching (KNM) pairs all maintain 3–8 correspondences. It also indicates that the feature-based method has apparent advantages in dealing with granular impressions with local peaks and valleys features, and poor performance on the striation marks. The clear distinction between KM and KNM impression pairs demonstrates the feasibility of the proposed method in ballistic feature comparison. Compared to the random hypothesize-and-verify modeling of RANSAC, this method can retain more reliable matching feature points of the impression pair to ensure the repeatability of feature correspondence selection.

Method for online matching of multiple identical targets using local homography matrix based on moving DLT

In the field of geometric measurement of stereo vision, problems such as interference sources and similarity of target characteristics increase the difficulty of the matching of multiple identical targets. An online matching method for multiple identical targets using local homography matrix based on moving direct linear transform is presented in this paper. Firstly, the matching feature point pairs between left and right background images are obtained in advance. Secondly, the left background image is evenly divided into multiple rectangular meshes, and the local homography matrix of each rectangular mesh is calculated offline. Thirdly, the mapping point of each target is estimated online by the local homography matrix corresponding to the rectangular mesh where the target is located, and the target point with the closest Euclidean distance to the mapping point is considered as the matching point. Computer simulation and quality experimental data proves effectiveness of the proposed method. Experimental results reflect that our method is better than existing matching methods based on Delaunay triangulation, and the root mean square error of image coordinates between the mapping points from our method and the actual matching points is less than 13 pixels when entire image size is 1920 pixels × 1200 pixels.

Mismatching Removal for Feature-Point Matching Based on Triangular Topology Probability Sampling Consensus

Feature-point matching between two images is a fundamental process in remote-sensing applications, such as image registration. However, mismatching is inevitable, and it needs to be removed. It is difficult for existing methods to remove a high ratio of mismatches. To address this issue, a robust method, called triangular topology probability sampling consensus (TSAC), is proposed, which combines the topology network and resampling methods. The proposed method constructs the triangular topology of the feature points of two images, quantifies the mismatching probability for each point pair, and then weights the probability into the random process of RANSAC by calculating the optimal homography matrix between the two images so that the mismatches can be detected and removed. Compared with the state-of-the-art methods, TSAC has superior performances in accuracy and robustness.

Fast Target Recognition Based on Improved ORB Feature

A novel fast target recognition algorithm is proposed under the dynamic scene moving target recognition. Aiming at the poor matching effect of the traditional Oriented Fast and Rotated Brief (ORB) algorithm on underexposed or overexposed images caused by illumination, the idea of combining adaptive histogram equalization with the ORB algorithm is proposed to get better feature point quality and matching efficiency. First, the template image and each frame of the video stream are processed by grayscale. Second, the template image and the image to be input in the video stream are processed by adaptive histogram equalization. Third, the feature point descriptors of the ORB feature are quantized by the Hamming distance. Finally, the K-nearest-neighbor (KNN) matching algorithm is used to match and screen feature points. According to the matching good feature point logarithm, a reasonable threshold is established and the target is classified. The comparison and verification are carried out by experiments. Experimental results show that the algorithm not only maintains the superiority of ORB itself but also significantly improves the performance of ORB under the conditions of underexposure or overexposure. The matching effect of the image is robust to illumination, and the target to be detected can be accurately identified in real time. The target can be accurately classified in the small sample scene, which can meet the actual production requirements.

Optimisation of computer virtual image reconstruction method based on feature point matching

Traditional computer virtual image reconstruction methods have the problems of slow reconstruction speed and insufficient image clarity. To solve this problem, an optimisation method of computer virtual image reconstruction based on feature point matching is proposed. SURF operator is used to extract image feature points. After obtaining feature points, image feature points are matched according to TZNCC constraints. The virtual image is reconstructed by sparse method, and the high resolution depth image in virtual vision is represented by dictionary sparse linear combination. The sparse coefficient of the image is solved by alternating direction multiplier algorithm, and the problem of virtual image reconstruction is transformed into a problem of solving sparse signal, so that a better reconstruction effect can be obtained. The experimental results show that the proposed method has high speed and clarity of image reconstruction.

Matching of small packages of traditional Chinese medicine based on improved RANSAC algorithm

The vision system model of small package sorting of traditional Chinese medicine was constructed, and the chessboard calibration method was used to calibrate the binocular camera. Surf algorithm and orb algorithm are analyzed by experiments, and orb algorithm with high real-time performance is used to extract feature points. The fast nearest neighbor FLANN algorithm is used to quickly match the feature points, and the matched feature points are screened according to Lowe’s algorithm and v-axis coordinate consistency principle, to solve the problem of false matching points. The method of selecting feature points based on matching quality ranking to calculate the homography matrix is designed, which solves the random sampling problem of the RANSAC algorithm and effectively shortens the time of feature matching.

A Novel Rock-Mass Point Cloud Registration Method Based on Feature Line Extraction and Feature Point Matching

Registration will directly affect the quality of overall rock-mass point cloud, which is the basis of 3-D reconstruction for rock mass. Advanced methods establish correspondence by extracting various features that remain unchanged. Although these methods have made great progress, they analyze the local characteristics of each sample point, which leads to be inefficient. In this article, we select registration interesting points from feature lines that were extracted based on supervoxel and innovatively introduce the “clustering, primary matching, and coarse registration” strategy, which effectively reduces the complexity of calculating the corresponding relationship during point cloud registration. Finally, the iterative closest point (ICP) algorithm is used to optimize the result of coarse registration. By selecting registration interesting points from the extracted feature lines, the proposed method inherits the robustness of feature lines to noise, initial position, and so on. The experimental results prove that the coarse registration and refined registration results of the proposed method both have high accuracy and efficiency.

Optimisation of computer virtual image reconstruction method based on feature point matching

Optimisation of computer virtual image reconstruction method based on feature point matching

Parallel VINS-Mono algorithm based on GPUs in embedded devices

Traditional visual-inertial simultaneous localization and mapping algorithms are usually designed based on CPUs, and they cannot effectively utilize the parallel computing function of GPUs if they are directly transplanted to an embedded board with a GPU module. However, the computing power of embedded devices is limited. It is unreasonable for the visual-inertial simultaneous localization and mapping algorithm to occupy most CPU computing resources while the GPU is idle. In this article, a parallelization scheme for the VINS-Mono algorithm based on GPU parallel computing technology is proposed. Based on the compute unified device architecture, the construction and solution of the incremental equation are parallelized in the nonlinear optimization process of the algorithm, and the parallelization methods provided by cuSOLVER and cuBLAS are used to carry out the marginalization of the algorithm. In addition, the program for the detection and matching of image feature points in the process of optical flow tracking is rewritten in the algorithm to realize the parallelization of optical flow tracking. After parallelization, the algorithm is found to run well on a heterogeneous computing model composed of a CPU and GPU and can fully exploit the parallel computing power of the GPU. The proposed method was tested on an NVIDIA’s Jetson TX2 module and compared with the VINS-Mono algorithm; the speeds of the construction and solution of the incremental equation were found to be the same, but the optical flow tracking and marginalization speed of the proposed scheme exhibited improvements of about 1.5–1.7 times and 1.9 times, respectively.

Targetless Extrinsic Calibration of Multiple Small FoV LiDARs and Cameras Using Adaptive Voxelization

Determining the extrinsic parameter between multiple LiDARs and cameras is essential for autonomous robots, especially for solid-state LiDARs, where each LiDAR unit has a very small Field-of-View (FoV), and multiple units are often used collectively. The majority of extrinsic calibration methods are proposed for 360° mechanical spinning LiDARs where the FoV overlap with other LiDAR or camera sensors is assumed. Few research works have been focused on the calibration of small FoV LiDARs and cameras nor on the improvement of the calibration speed. In this work, we consider the problem of extrinsic calibration among small FoV LiDARs and cameras, with the aim to shorten the total calibration time and further improve the calibration precision. We first implement an adaptive voxelization technique in the extraction and matching of LiDAR feature points. Such a process could avoid the redundant creation of k-d trees in LiDAR extrinsic calibration and extract LiDAR feature points in a more reliable and fast manner than existing methods. We then formulate the multiple LiDAR extrinsic calibration into a LiDAR Bundle Adjustment (BA) problem. By deriving the cost function up to second-order, the solving time and precision of the non-linear least square problem are further boosted. Our proposed method has been verified on data collected in four targetless scenes and under two types of solid-state LiDARs with a completely different scanning pattern, density, and FoV. The robustness of our work has also been validated under eight initial setups, with each setup containing 100 independent trials. Compared with the state-of-the-art methods, our work has increased the calibration speed 15 times for LiDAR-LiDAR extrinsic calibration (averaged result from 100 independent trials) and 1.5 times for LiDAR-Camera extrinsic calibration (averaged result from 50 independent trials) while remaining accurate. To benefit the robotics community, we have also open-sourced our implementation code on GitHub.

Local neighborhood feature point extraction and matching for point cloud alignment

Local neighborhood feature point extraction and matching for point cloud alignment

Simultaneous Localization and Mapping Based on Point and Line Feature Matching

Simultaneous Localization and Mapping Based on Point and Line Feature Matching

Research on Visual Odometer of Wheeled Robot with Motion Constraints

In this paper, aiming at the planar motion characteristics of wheeled robots, the Red Green Blue Depth Camera (RGBD camera) was used as an image acquisition device to reduce the motion of wheeled robots to two-dimensional plane processing, which simplified the calculation method of visual mileage of planar wheeled robots. In addition, according to the two-dimensional motion characteristics of the robot camera, the contour constraint condition of feature point matching between frames was proposed, and the linear constraint condition was proposed by linearizing the motion equation of the camera between frames. The research showed that the contour constraint condition and linear constraint condition could be used to screen the mismatched feature points, this condition could not only screen the mismatched point pairs of color images, but also screen the matching point pairs with correct color image matching but large depth error, which provided high quality matching point pairs for inter-frame motion estimation of visual odometer. Finally, the filtered matching point pairs were reduced in dimension and combined with the two-dimensional Iterative Closest Point (ICP) algorithm to estimate the trajectory of the robot camera. The experimentation results showed that compared with the original three-dimensional ICP algorithm, the combination of contour constraint condition, linear constraint condition and two-dimensional ICP algorithm could significantly improve the computational speed of visual odometry.

A Demodulation Algorithm for Periodically In-Plane Vibrating MEMS Based on a Stroboscopic Micro-Visual System.

A high-frequency short-pulsed stroboscopic micro-visual system was employed to capture the transient image sequences of a periodically in-plane working micro-electro-mechanical system (MEMS) devices. To demodulate the motion parameters of the devices from the images, we developed the feature point matching (FPM) algorithm based on Speeded-Up Robust Features (SURF). A MEMS gyroscope, vibrating at a frequency of 8.189 kHz, was used as a testing sample to evaluate the performance of the proposed algorithm. Within the same processing time, the SURF-based FPM method demodulated the velocity of the in-plane motion with a precision of 10−5 pixels of the image, which was two orders of magnitude higher than the template-matching and frame-difference algorithms.