Point Set Matching Research Articles

LightSGM: Local feature matching with lightweight seeded

Addressing the quintessential challenge of local feature matching in computer vision, this study introduces a novel fast sparse seed graph structure named LightSGM. This structure aims to refine the characterization of graph features while minimizing superfluous connections. Initially, a subset of high-quality seed feature points is curated using a confidence filter. Subsequently, keypoint features are assimilated into this seed set via graph pooling, and the composite features are further processed through a memory and computation-efficient seed transformer to capture rich contextual information about the keypoints. The seed feature points are then relayed back to the original keypoints using an inverse process known as graph unpooling. The paper also introduce an adaptive mechanism to determine the optimal number of model layers based on the intricacy of matching image pairs. A Matching Point Prediction Header is employed to extract the final set of matching points. Through extensive experimentation on image matching and position estimation, LightSGM has demonstrated its prowess in delivering competitive matching accuracy while maintaining a balance with real-time processing capabilities.

Identification and Positioning of Abnormal Maritime Targets Based on AIS and Remote-Sensing Image Fusion.

The identification of maritime targets plays a critical role in ensuring maritime safety and safeguarding against potential threats. While satellite remote-sensing imagery serves as the primary data source for monitoring maritime targets, it only provides positional and morphological characteristics without detailed identity information, presenting limitations as a sole data source. To address this issue, this paper proposes a method for enhancing maritime target identification and positioning accuracy through the fusion of Automatic Identification System (AIS) data and satellite remote-sensing imagery. The AIS utilizes radio communication to acquire multidimensional feature information describing targets, serving as an auxiliary data source to complement the limitations of image data and achieve maritime target identification. Additionally, the positional information provided by the AIS can serve as maritime control points to correct positioning errors and enhance accuracy. By utilizing data from the Jilin-1 Spectral-01 satellite imagery with a resolution of 5 m and AIS data, the feasibility of the proposed method is validated through experiments. Following preprocessing, maritime target fusion is achieved using a point-set matching algorithm based on positional features and a fuzzy comprehensive decision method incorporating attribute features. Subsequently, the successful fusion of target points is utilized for positioning error correction. Experimental results demonstrate a significant improvement in maritime target positioning accuracy compared to raw data, with over a 70% reduction in root mean square error and positioning errors controlled within 4 pixels, providing relatively accurate target positions that essentially meet practical requirements.

An errors-unidentified hand-eye calibration method via robot relocalization and iterative combinatorial refinement

The accuracy of hand-eye calibration is easily affected by robot kinematic error, measurement error, random error, etc., and can be to a certain extent enhanced by the complicated error identification. To address this problem, we propose a novel errors-unidentified hand-eye calibration method. A relocalization-based hand-eye calibration is implemented to overcome the challenge from robot kinematics at first owing to the high robot relocalization accuracy. The tool center point (TCP) coordinates are then obtained based on an iterative reweighted least squares (IRLS) spherical fitting algorithm. An iterative combinatorial refinement algorithm is finally presented to search the solutions in the form of permutation and combination when solving the rigid transformation matrix. Experiments on criterion sphere and blade demonstrate that by converting the hand-eye matrix solving problem into a point set matching problem, the accuracy of the proposed method is enhanced by 60% on average compared with three state-of-the-art hand-eye calibration methods.

Feature matching based on Gaussian kernel convolution and minimum relative motion

Feature matching is a necessary and important step for remote sensing image registration, intended to establish reliable point correspondences between two sets of features. In this paper, we propose a feature registration model based on local relative motion, which combines Gaussian kernel convolution with relative motion (GRM) vector to obtain better results by removing wrong matches and improving the inlier point accuracy. We first establish putative matching based on the similarity between local descriptors. Then, the preliminary hypothetical matching point set is filtered using consistency with nearest neighbors among the inlier points to obtain a more accurate motion vector, and to fit the real motion vector through the Gaussian convolution kernel. Finally, we find the displacement between the fitted motion vector and the matching generated motion vector. And combine the displacement with the optimization model to find the inlier point set. Experimental results show that our GRM method outperforms related work, achieving better matching results.

Efficient and precise docking trajectory optimization for the ship block assembly

The assembly of the ship block is an extremely important stage of the shipbuilding process. Nevertheless, currently, the manual assembly efficiency is low, the accuracy is poor, and collision is very easy to occur. Therefore, there is an urgent need to conduct technical research on the automatic docking of ship blocks. The core of the automated docking technology is the attitude estimation and the trajectory planning of the posturing equipment. However, current data measurement and point set matching methods lead to large attitude-estimation errors, and it is difficult to meet the accuracy requirements of the assembly. Moreover, the current ship block trajectory planning methods pay more attention to single metrics, for example, time or energy consumption, while omitting the shock degree. In response to the above, this study first proposes a high-precision matching method for measuring point sets, in order to estimate the exact attitude of the ship block. Subsequently, trajectory translation for the block is performed using the seventh-degree polynomial. On this basis, a nonlinear weighted improved particle swarm optimization (IPSO) method is proposed to optimize the time, energy consumption and shock degree in the ship block trajectory planning process. Finally, the accuracy of the matching optimization is validated by simulation analysis and it is concluded that the seventh-degree polynomial leads to less shock than other polynomials. Furthermore, the shock force does not change abruptly even when the ship block is poised in steps. Through IPSO, the energy consumption and shock degree performance indices are optimized by 37.07% and 50.06%, respectively, in the ship block translation process.

A two-step solution for robot-world calibration made intelligible by implementing Chasles’ motion decomposition in Ad(SE(3))

As a typical kinematic problem, an important criterion for the accuracy of robot-world calibration is whether the rotational and translational parts are calculated separately or not. Solving the kinematic equation separately increases the interest into the decomposition mode of the equation. This paper provides a decomposition mode for Ad(SE(3)) with embedded Chasles’ motion, providing a new decomposition mode for the robot-world equation. Firstly, the Lie algebra ad(se(3)) for Chasles’ rotational and translational motions are constructed and mapped exponentially to the Lie group Ad(SE(3)) separately, thus implementing the Chasles’ motion decomposition of Ad(SE(3)). Subsequently, the robot-world equation in Ad(SE(3)) is derived based on joint kinematics. A new decomposition mode for robot-world equation is proposed through the Chasles’ motion decomposition, and a two-step method based on point set matching is proposed. Finally, the effect of coordinate invariants on the calibration accuracy is discussed, and the superiority of the proposed method is verified with simulation and experimental tests.

An adaptive method of measuring the rake face wear of end mills based on image feature point set registration

An acceptable tool wear evaluation and acquisition approach is critical for developing an accurate, rapid, and practical online tool wear prediction system. The one-dimensional crater wear depth is difficult to quantify, and relying solely on flank wear width is very simplistic. External conditions such as vibration and illumination have a large impact on two-dimensional wear area and three-dimensional volume. In practice, these are difficult to successfully characterize tool wear. This work proposes a two-dimensional rake face edge loss area as an additional indicator to assess milling cutter wear. An adaptive measurement approach based on 2D image feature point set matching is presented to suppress the area calculation error caused by image position offset and chips or built-up edges. 201 images were collected in two groups of cutting experiments for comparison with the proposed method and the other two methods, manual registration and non-registration. The results show that the proposed method's measurement values are closer to the actual values and have a higher measurement efficiency. Furthermore, the proposed indicator is sensitive to micro-scale changes to the blade edge, especially at the tip, and can be used as a supplement one to monitor tool wear more comprehensively.

An Accurate Point Set Matching Method for Hull Blocks With Engineering Constraints

Abstract To effectively improve the precision matching technique of measurement point set and design point set for hull blocks, an accurate point set matching method for hull blocks with engineering constraints (verticality, levelness, symmetry, flatness, etc.) and elimination error points is presented. It is divided into initial matching and refine matching. In initial matching, the improved Random Sample Consensus algorithm is proposed to eliminate error points rapidly and accurately; the improved Coherent Point Drift algorithm is proposed to obtain more accurate initial matching. In refine matching, the Analytic Hierarchy Process is used to obtain each weight of engineering constraints automatically, and the weight vector is introduced into the multi-optimization objective function to achieve the more reasonable matching results. The results proved that this method can rapidly and automatically eliminate error points, and get more accurate and reasonable results meeting engineering constraints. It can provide the basis for the subsequent assembly of hull blocks.

Iterative Fine Point-Set Matching of a Free-Form Surface Based on the Point-to-Sphere Distance

Three-dimensional point-set matching that aligns the moving point set with the fixed design model is an increasingly popular technique in manufacturing automation, which also increases the demand for matching performance. In this article, an iterative fine matching algorithm based on point-to-sphere distance is proposed to balance convergence accuracy, stability, and speed. By considering the neighborhood of a fixed point as a spherical surface, a point-to-sphere distance from a moving point to the design model is derived to improve the distance calculation accuracy with respect to unknown motion parameters. A point-to-sphere iterative closest point (ICP) matching method is presented, which translates the matching into a nonlinear optimization problem. Using the Newton method, a second-order Taylor expansion is applied to iteratively solve the motion parameters. Theoretical analysis verifies the advantages in convergence performance. It is proven that the proposed method can show faster convergence speed than point-to-point ICP and better convergence stability than point-to-plane ICP in large initial poses. Experiments implementing five models verify the feasibility of the point-to-sphere distance and the corresponding matching method.

An Exact Hypergraph Matching algorithm for posture identification in embryonic C. elegans.

The nematode Caenorhabditis elegans (C. elegans) is a model organism used frequently in developmental biology and neurobiology [White, (1986), Sulston, (1983), Chisholm, (2016) and Rapti, (2020)]. The C. elegans embryo can be used for cell tracking studies to understand how cell movement drives the development of specific embryonic tissues. Analyses in late-stage development are complicated by bouts of rapid twitching motions which invalidate traditional cell tracking approaches. However, the embryo possesses a small set of cells which may be identified, thereby defining the coiled embryo's posture [Christensen, 2015]. The posture serves as a frame of reference, facilitating cell tracking even in the presence of twitching. Posture identification is nevertheless challenging due to the complete repositioning of the embryo between sampled images. Current approaches to posture identification rely on time-consuming manual efforts by trained users which limits the efficiency of subsequent cell tracking. Here, we cast posture identification as a point-set matching task in which coordinates of seam cell nuclei are identified to jointly recover the posture. Most point-set matching methods comprise coherent point transformations that use low order objective functions [Zhou, (2016) and Zhang, (2019)]. Hypergraphs, an extension of traditional graphs, allow more intricate modeling of relationships between objects, yet existing hypergraphical point-set matching methods are limited to heuristic algorithms which do not easily scale to handle higher degree hypergraphs [Duchenne, (2010), Chertok, (2010) and Lee, (2011)]. Our algorithm, Exact Hypergraph Matching (EHGM), adapts the classical branch-and-bound paradigm to dynamically identify a globally optimal correspondence between point-sets under an arbitrarily intricate hypergraphical model. EHGM with hypergraphical models inspired by C. elegans embryo shape identified posture more accurately (56%) than established point-set matching methods (27%), correctly identifying twice as many sampled postures as a leading graphical approach. Posterior region seeding empowered EHGM to correctly identify 78% of postures while reducing runtime, demonstrating the efficacy of the method on a cutting-edge problem in developmental biology.

Image Point Set Registration: Phased Gaussian Mixture Model

The introduction of probability model into point set matching has achieved great success. Because the image point set has more outliers and the image has complex transformation, the traditional Gaussian Mixture Model (GMM) does not perform well in image point set matching. In this paper, we propose a Phased Gaussian Mixture Model (PGMM) for point set registration between images. Different from the existing registration algorithms based on GMM, PGMM uses Locality Sensitive Hashing to distinguish the feature reliability of feature points. The initial transformation relation is obtained in the segment with high feature reliability and vector constraint is applied to other segments. The average Hamming distance is used to give different weights to each Gaussian component, which is related to the characteristics between data points and model points. The proposed method has good performance in the case of large image transformation scale.

Scale-Invariant Localization of Electric Vehicle Charging Port via Semi-Global Matching of Binocular Images

Automatic charging for electric vehicles has broad development prospects for meeting the personalized service experience of users while overcoming the inherent safety hazards. An identification and positioning approach suitable for engineering applications is the key to promoting automatic charging. In this paper, a low-cost, high-precision method to identify and position charging ports based on SIFT and SGBM is proposed. The feature extraction approach based on SIFT is adopted to produce the difference of Gaussian (DOG) for scale space construction, and the feature matching algorithm with nearest-neighbor search, which is a kind of machine learning, is utilized to yield the map set of matching points. In addition, the disparity calculation is conducted with a semi-global matching algorithm to obtain high-precision positioning results for the charging port. In order to verify the feasibility of the method, a complete identification and positioning experiment of charging port was carried out based on OpenCV and MATLAB.

SCRnet: A Spatial Consistency Guided Network Using Contrastive Learning for Point Cloud Registration

Point cloud registration is used to find a rigid transformation from the source point cloud to the target point cloud. The main challenge in the point cloud registration is in finding correct correspondences in complex scenes that may contain many noise and repetitive structures. At present, many existing methods use outlier rejections to help the network obtain more accurate correspondences, but they often ignore the spatial consistency between keypoints. Therefore, to address this issue, we propose a spatial consistency guided network using contrastive learning for point cloud registration (SCRnet), in which its overall stage is symmetrical. SCRnet consists of four blocks, namely feature extraction block, confidence estimation block, contrastive learning block and registration block. Firstly, we use mini-PointNet to extract coarse local and global features. Secondly, we propose confidence estimation block, which formulate outlier rejection as confidence estimation problem of keypoint correspondences. In addition, the local spatial features are encoded into the confidence estimation block, which makes the correspondence possess local spatial consistency. Moreover, we propose contrastive learning block by constructing positive point pairs and hard negative point pairs and using Point-Pair-INfoNCE contrastive loss, which can further remove hard outliers through global spatial consistency. Finally, the proposed registration block selects a set of matching points with high spatial consistency and uses these matching sets to calculate multiple transformations, then the best transformation can be identified by initial alignment and Iterative Closest Point (ICP) algorithm. Extensive experiments are conducted on KITTI and nuScenes dataset, which demonstrate the high accuracy and strong robustness of SCRnet on point cloud registration task.

Slam algorithm for multi-robot communication in unknown environment based on particle filter

To solve the SLAM (simultaneous localization and map building) of mobile robots, a multi-robot SLAM algorithm based on particle filter for communication in unknown environment is proposed. In the standard particle filter, the incremental map construction method based on point-line consistency is introduced to preserve the hypothesis of the line segment feature map in each particle, and the importance function is introduced to the observation information. On this basis, sampling map fusion algorithm extracts ORB (oriented FAST and rotated BRIEF) features of local map constructed by a single robot and finds the optimal matching points, calculates the homography matrix of local map optimal matching point set, determines the optimal affine transformation parameters among point sets, so as to complete the local map fusion, and realize the multi-robot communication based on SLAM in unknown environment. The results show that the algorithm achieves high-precision map fusion and it is an effective SLAM construction algorithm for multiple machines.

A Remote Sensing Image Registration Algorithm Based on Multiple Constraints and a Variational Bayesian Framework

ABSTRACT Aiming at complicated remote sensing image registration problem, we proposed a multi-constraint registration algorithm based on variational inference. Gaussian mixture model is used to construct the entire Bayesian network, which facilitates the estimation of heteroscedasticity noise in mapping function regression, and distribution of outliers in scene point density estimation. We use a variational parameter proxy to approximate complex posterior distribution. In addition, while maintaining the stability of global structure, similarity of local structure is restricted. In order to limit the cost of image conversion, we added constraints on spatial curvature for image. Accuracy and smoothness of the spatial transformation can be maintained by minimizing the curvature in horizontal and vertical directions. Experimental results show that compared with other point set matching algorithms, this method can achieve better results in robustness and matching accuracy.

A Simple Pipeline for Coherent Grid Maps.

Grid maps are spatial arrangements of simple tiles (often squares or hexagons), each of which represents a spatial element. They are an established, effective way to show complex data per spatial element, using visual encodings within each tile ranging from simple coloring to nested small-multiples visualizations. An effective grid map is coherent with the underlying geographic space: the tiles maintain the contiguity, neighborhoods and identifiability of the corresponding spatial elements, while the grid map as a whole maintains the global shape of the input. Of particular importance are salient local features of the global shape which need to be represented by tiles assigned to the appropriate spatial elements. State-of-the-art techniques can adequately deal only with simple cases, such as close-to-uniform spatial distributions or global shapes that have few characteristic features. We introduce a simple fully-automated 3-step pipeline for computing coherent grid maps. Each step is a well-studied problem: shape decomposition based on salient features, tile-based Mosaic Cartograms, and point-set matching. Our pipeline is a seamless composition of existing techniques for these problems and results in high-quality grid maps. We provide an implementation, demonstrate the efficacy of our approach on various complex datasets, and compare it to the state-of-the-art.

A fast and accurate new algorithm for hand–eye calibration on SO(3)[formula omitted

In this paper, we propose a new solution for an old problem in robotics related to the calibration of the hand, i.e., the robot’s end-effector, and the eye, i.e., a stereo vision camera. The problem is formulated as a point set matching problem and a nonlinear estimator on manifold SO(3)×R3 used for obtaining the solution. The main advantage of the proposed approach is that it allows to decouple the error associated with the rotational estimation on Lie group SO(3) from the error of the translational estimation on R3, which subsequently allows to tune the learning rates for the rotation and translation estimation, separately. This will result in significant increase in the convergence speed of the proposed approach. To show the advantages of our approach, we compare the results with those obtained from other conventional hand–eye calibration solutions as well as those based on point set matching. The experimental results will demonstrate that our proposed hand–eye calibration approach outperforms other approaches in terms of accuracy and computational speed.

3D Point Cloud-Based Indoor Mobile Robot in 6-DoF Pose Localization Using a Wi-Fi-Aided Localization System

Six-DoF (Six-degree-of-freedom) pose localization based on 3D point clouds is a challenging task for LBSs (localization-based services). This paper proposes a robust and efficient method that uses multimodal information (vision and Wi-Fi signal information) to estimate the 6-DoF pose of an RGBD camera on a robot with respect to complex 3D textured models of the indoor environment that can contain more than 650,000,000 points. Our developed method narrows the search scope, which delimits boundaries initially using the Wi-Fi location system and applies an environment-adaptive approach to determine the radius of the search sphere based on the signal stability of the Wi-Fi location system. In addition, we propose an algorithm for estimating a novel correspondence between local points with a 3D submap by combining 3D points and surface normals to acquire absolute poses from noisy and outlier-contaminated matching point sets for RGBD sensors in dynamic indoor scenes. Then, a novel two-level spatial verification strategy is used to estimate an accurate pose, which includes the use of a RANSAC (Random Sample Consensus) algorithm for identification and a direct least-square method to acquire the pose from the inliers. The proposed method has been implemented and tested extensively in various indoor scenes. The experimental results demonstrate that the Wi-Fi-aided localization system can efficiently localize a mobile robot in a variety of large-scale 3D point cloud datasets to realize stable time consumption and similar performance to state-of-the-art methods.

Automatic 3D point cloud registration algorithm based on triangle similarity ratio consistency

Three-dimensional (3D) point cloud registration is a fundamental key issue in 3D reconstruction, 3D object recognition and augmented reality. In this study, the authors propose a novel local feature descriptor called local angle statistics histogram (LASH) for efficient 3D point cloud registration. LASH forms a description of local shape geometries by encoding their properties on angles between the normal vector of the point and the vector formed by the point and other points in its local neighbourhood. In addition, they propose a 3D point cloud registration algorithm based on LASH. The registration algorithm firstly detects triangle matching points with consistent similarity ratios, and then aggregates each pair of triangular matching points into a set of matching points. They can use these matching sets to calculate multiple transformations between two point clouds. Finally, they use the error function to identify the best transformation and to achieve coarse alignment of the two point clouds. Experiments and comparisons with other global algorithms demonstrate that the proposed approach can be applied to register point clouds with considerable or limited overlaps and is robust to noise.

Shape point set matching based on oriented shape context in turbulence-cluttered scene

The main challenges of shape point set matching in a long-distance imaging scene stem from the optical turbulence effects, which lead to shape object deformation, rotation, shifted object positions, and cluttered outliers. To address this problem, we propose an effective energy cost model with figural continuity constraint. We first construct an Oriented Shape Context (OSC) descriptor using attributes of shape edges’ length and direction, which represent rotation invariance, by adding the oriented model (prototype) edges point set. Then, inspired by the figural continuity prior between the model and target point set, we transform the continuity constraint into a matching energy cost model. Lastly, we develop a simple 2-tree graph to minimize the matching cost function using the Dynamic Program (DP) optimization algorithm. The extensive experiments on both synthetic and real data validate that the proposed method can effectively detect the desired shapes in the complex and highly turbulence-cluttered scenes.