Outliers In Correspondences Research Articles

DMS: Low-overlap Registration of 3D Point Clouds with Double-layer Multi-scale Star-graph.

Registering 3D point clouds with low overlap is challenging in 3D computer vision, primarily due to difficulties in identifying small overlap regions and removing correspondence outliers. We observe that the neighborhood similarity can be utilized to detect point correspondence, and the consistent neighborhood correspondence can be used as a criterion to detect robust overlapping regions. So that a Double-layer Multi-scale Star-graph (DMS) structure is proposed to detect robust correspondences using two different types of multi-scale star-graphs. The first-layer Multi-scale Neighbor Feature Star-graphs (MNFS) takes each point as the center and its multi-scale nearest neighbors as the leaves. The MNFS enables to establish the initial correspondence candidate set between the two point clouds based on multi-scale neighborhood topology and feature similarity. Subsequently, each pair of corresponding points find their nearest neighbors within the correspondence sets to construct a Multi-scale Matching Star-graphs (MMS) on each side, so the mutual correspondence relationships between the MMS vertices are identified. These identified mutual correspondences are treated as vertices to construct the Multi-scale Correspondence Star-graphs (MCS), that indicate the relationships among the correspondences. We design edge weight and vertex weight criterion in MCS to detect only the robust correspondence set that has strong neighborhood consistency, so as to reject the outliers. Finally, the point cloud registration is conducted based on the detected robust correspondence. The experimental results demonstrate clearly that the proposed DMS method exhibits superior robustness when compared to existing state-of-the-art registration algorithms. The code of this study will be available at https://github.com/HualongCao/DMS.

GMF: General Multimodal Fusion Framework for Correspondence Outlier Rejection

Rejecting correspondence outliers enables to boost the correspondence quality, which is a critical step in achieving high point cloud registration accuracy. The current state-of-the-art correspondence outlier rejection methods only utilize the structure features of the correspondences. However, texture information is critical to reject the correspondence outliers in our human vision system. In this letter, we propose <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">G</b> eneral <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">M</b> ultimodal <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">F</b> usion (GMF) to learn to reject the correspondence outliers by leveraging both the structure and texture information. Specifically, two cross-attention-based fusion layers are proposed to fuse the texture information from paired images and structure information from point correspondences. Moreover, we propose a convolutional position encoding layer to enhance the difference between <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$Tokens$</tex-math></inline-formula> and enable the encoding feature pay attention to neighbor information. Our position encoding layer will make the cross-attention operation integrate both local and global information. Experiments on multiple datasets(3DMatch, 3DLoMatch, KITTI) and recent state-of-the-art models (3DRegNet, DGR, PointDSC) prove that our GMF achieves wide generalization ability and consistently improves the point cloud registration accuracy. Furthermore, several ablation studies demonstrate the robustness of the proposed GMF on different loss functions, lighting conditions and noises. The code is available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/XiaoshuiHuang/GMF</uri> .

An automatic and robust point cloud registration framework based on view-invariant local feature descriptors and transformation consistency verification

This paper presents an automatic and robust framework for simultaneously registering pairwise point clouds and identifying the correctness of registration results. Given two partially overlapping point clouds with arbitrary initial positions, a view-invariant local feature descriptor is utilized to build sparse correspondence. A geometry constraint sample consensus (GC-SAC) algorithm is proposed to prune correspondence outliers and obtain an optimal 3D transformation hypothesis. Furthermore, by measuring the similarity between the estimated local and global transformations, a transformation consistency verification method is presented to efficiently detect potential registration failures. Our method provides reliable registration correctness verification even when two point clouds are only roughly registered. Experimental results demonstrate that our framework exhibits high levels of effectiveness and robustness for automatic registration.

Fast Aerial Video Stitching

The highly efficient and robust stitching of aerial video captured by unmanned aerial vehicles (UAVs) is a challenging problem in the field of robot vision. Existing commercial image stitching systems have seen success with offline stitching tasks, but they cannot guarantee high-speed performance when dealing with online aerial video sequences. In this paper, we present a novel system which has an unique ability to stitch high-frame rate aerial video at a speed of 150 frames per second (FPS). In addition, rather than using a high-speed vision platform such as FPGA or CUDA, our system is running on a normal personal computer. To achieve this, after the careful comparison of the existing invariant features, we choose the FAST corner and binary descriptor for efficient feature extraction and representation, and present a spatial and temporal coherent filter to fuse the UAV motion information into the feature matching. The proposed filter can remove the majority of feature correspondence outliers and significantly increase the speed of robust feature matching by up to 20 times. To achieve a balance between robustness and efficiency, a dynamic key frame-based stitching framework is used to reduce the accumulation errors. Extensive experiments on challenging UAV datasets demonstrate that our approach can break through the speed limitation and generate an accurate stitching image for aerial video stitching tasks.

Registration of partially overlapping surfaces by rejection of false point correspondences

We present a new algorithm for the registration of three-dimensional partially overlapping surfaces. It is based on an efficient scheme for the rejection of false point correspondences (correspondence outliers) and does not require initial pose estimation or feature extraction. An initial list of corresponding points is first derived using the regional properties of vertices on both surfaces. From these point correspondences, pairs of corresponding rigid triplets are formed. The normal vectors at the vertices of each corresponding triplet are used to compute the candidate rotations. By clustering the candidate rotation axes and candidate rotation angles separately, a large number of false correspondences are eliminated and an approximate rotation is decided, from which an approximate translation is also obtained. Finally, the optimal transformation parameters are determined by further refining the estimated parameters in an iterative manner. Mathematical analysis and experimental results show that the registration process is fast and accurate even when the objects are regularly shaped and contain many regionally similar surface patches.

Efficient partial-surface registration for 3D objects

We present a new method for three-dimensional partial-surface registration that utilizes both regional surface properties and shape rigidity constraint to align a partial object surface and its corresponding complete surface. The statistical properties of the vertices on the object surface are first computed and compared with each other to find the initial candidate correspondences. We use the overall object-shape rigidity constraint and a clustering method to obtain an approximation of the transformation parameters while, at the same time, rejecting correspondence outliers. The transformation parameters can be further refined with an iterative approach. The algorithm does not require any feature extraction or initial pose estimation, and is especially applicable when the object surfaces are formed by a large number of vertices, smooth with few salient features, and contain many regionally similar surface patches. Experiments confirm that the proposed scheme can achieve accurate registration results in an efficient manner.