Fundamental Matrix Estimation Research Articles

A correspondence selection method based on same object and same position constraints

Establishing robust correspondences between two images is important for computer version tasks. However, in the real scene incorrect correspondences are inevitable no matter what kind of correspondence matching algorithms are adopted due to some complex factors, such as illumination, occlusion, and so on. To reduce the number of incorrect correspondences, an algorithm with the same object and same position constraints (SOSPC), is proposed to remove wrong correspondences from the given putative correspondences in this paper. The algorithm is based on the fact that in the given image pairs correct correspondences locate at the same position on the same objects. To select the correspondences on the same objects, an object matching method based on the correspondences selected by GMS is proposed. To select the correspondences on the correct positions, an iterative fundamental matrix estimation method based on clustering is presented. The experimental results have validated the effectiveness of the same object and the same position constraints, and the method achieves the state-of-art performance on five datasets.

Improving sparse graph attention for feature matching by informative keypoints exploration

Feature matching refers to building correct point correspondences from two-view images. Typical methods create coarse matches merely based on the similarity of local descriptors, then additionally use geometrical constraints or coherence to get a clean match set. However, this indirect paradigm would be strictly limited by the pre-matching results. This can be alleviated by a GNN-based matching network that constructs correct correspondences directly from feature positions and descriptions. Attentional pooling is utilized to reduce the high computational complexity and memory usage by sparsing the graph attention. In this paper, we propose an enhanced sparse GNN, namely KeyGNN, for fast feature matching, by designing Guided Attentional Pooling to emphasize the informative cues in GNN layers. Specifically, we propose Informative Keypoints Exploration Module and Guided Sparse Module to respectively extract information-rich points and use them to guide the attentional pooling for better information preservation. This process is constrained by the proposed Structural Importance Ranking Loss and Descriptor Distribution Loss, that ensure the structural information can be correctly extracted and the generated feature vectors be well distributed in the deep space. Unlike existing techniques, our method can focus and preserve more on the main information in the sparse GNN stage, thus avoiding information loss and enhancing the accuracy. Extensive experiments on large and public datasets reveal that our method can obtain better performance than the state-of-the-arts in the tasks of camera pose estimation, fundamental matrix estimation, and visual localization. The runtime and memory test shows that our network takes low computational and memory usage, which is friendly to real-time vision tasks.

Robust feature matching via progressive smoothness consensus

Feature matching is a long-standing fundamental and critical problem in computer vision and photogrammetry. The indirect matching strategy has become a popular choice because of its high precision and generality, but it finds only a limited number of correct matches, and the mismatch removal phase does not utilize the critical feature descriptors. To this end, this paper proposes a novel and effective feature matching method, named Progressive Smoothness Consensus (PSC). Our PSC designs an objective function to directly construct correct matches from two feature point sets. To optimize the objective, we introduce a stepwise strategy, where a small but reliable match set with the smooth function is used as initialization, and then the correct match set is iteratively enlarged and optimized by match expansion and smooth function estimation, respectively. In addition, the local geometric constraint is added to the compact representation with a Fourier basis, thus improving the estimation precision. We perform the match expansion as a Bayesian formulation to exploit both the spatial distribution and feature description information, thus finding feasible matches to expand the match set. Extensive experiments on feature matching, homography & fundamental matrix estimation, and image registration are conducted, which demonstrate the advantages of our PSC against state-of-the-art methods in terms of generality and effectiveness. Our code is publicly available at https://github.com/XiaYifan1999/Robust-feature-matching-via-Progressive-Smoothness-Consensus.

Efficient sampling using feature matching and variable minimal structure size

Greedy search-based guided sampling is a promising research field in model fitting to data with multiple structures in the presence of a large number of outliers. However, these greedy search-based guided sampling algorithms are sensitive to the fixed minimal (acceptable) structure size and the initial model hypothesis: when the fixed minimal structure size is too small, data subsets sampled by these algorithms are not representative. In contrast, when it is too large, data subsets might be contaminated by outliers. Furthermore, these algorithms may fail to obtain an accurate model hypothesis, if the initial model hypothesis is far from the true model. In this paper, we address the above-mentioned two issues by proposing two greedy search-based strategies: one aims to adaptively estimate minimal structure sizes and the other aims to generate effective initial model hypotheses. Specifically, on one hand, to avoid using the fixed minimal structure size, a strategy is proposed to adaptively estimate minimal structure sizes by using previously obtained ones. On the other hand, to reduce the impact of outliers, a strategy is proposed to filter out outliers to obtain a reduced data subset by using a feature matching algorithm. Then, this strategy generates promising initial model hypotheses by using a proximity sampling on the reduced data subset. Finally, an efficient sampling algorithm based on the two proposed greedy search-based strategies is applied to three vision tasks, i.e., fundamental matrix estimation, homography plane detection and 3D motion segmentation. Extensive experimental results demonstrate the effectiveness of the proposed sampling algorithm.

Study on Optimization Method of Visual Odometry Based on Feature Matching

The mismatching of image features affects the calculation of the fundamental matrix and then leads to poor estimation accuracy of SLAM visual odometry. Aiming at the above problems, a visual odometry optimization method based on feature matching is proposed. Firstly, the initial matching set is roughly filtered by the minimum distance threshold method, and then, the relative transformation relationship between images is calculated by the RANSAC algorithm. If it conforms to the transformation relationship, it is an interior point. The iteration result with most interior points is the correct matching result. Then, the homography transformation between images is calculated, and the fundamental matrix is calculated by it. The interior points are determined by epipolar geometric constraints, and the fundamental matrix with the most interior points is obtained. Finally, the effects of the visual odometry optimization algorithm are verified by the TUM data set from two aspects: feature matching and fundamental matrix calculation. The experimental results show that an improved feature matching algorithm can effectively remove mismatched feature points while improving the operation efficiency. At the same time, the accuracy of feature point matching is increased by 15.8%. The fundamental matrix estimation algorithm not only improves the calculation accuracy of the fundamental matrix but also increases the interior point rate by 11.9%. A theoretical basis for improving the accuracy estimation of visual odometry will be provided.

A Global Fundamental Matrix Estimation Method of Planar Motion Based on Inlier Updating.

A fundamental matrix estimation based on matching points is a critical problem in epipolar geometry. In this paper, a global fundamental matrix estimation method based on inlier updating is proposed. Firstly, the coplanar constraint was incorporated into the solution of the fundamental matrix to reduce the number of parameters to be solved. Subsequently, an inlier updating matrix was introduced according to the threshold of the epipolar geometry distance to eliminate the potential outliers and obtain a reliable initial value of the fundamental matrix. On this basis, we employed a four-point iterative method to estimate the fundamental matrix and make it satisfy the rank constraint at the same time. Finally, the epipolar geometry in binocular vision was extended to triple-view, and the fundamental matrix obtained in the previous step was globally optimized by minimizing the coordinate deviation between the intersection point and feature point in each group of images. The experiments show that the proposed fundamental matrix estimation method is robust to noise and outliers. In the attitude measurement, the maximum static error was 0.104° and dynamic measurement error was superior to 0.273°, which improved the reconstruction accuracy of feature points. Indoor images were further used to test the method, and the mean rotation angle error was 0.362°. The results demonstrate that the estimation method proposed in this paper has a good practical application prospect in multi-view 3D reconstruction and visual localization.

FM-Net: Deep Learning Network for the Fundamental Matrix Estimation from Biplanar Radiographs

Background and ObjectiveThe fundamental matrix estimation is a classic problem in computer vision. The traditional algorithms require high-precision correspondences. However, correspondences in biplanar radiographs are difficult to match accurately. MethodsWe propose an end-to-end network to estimate the F-Matrix directly from BR, which includes feature extraction and regression prediction. There is no publicly available dataset of biplanar radiographs. We produce the dataset in this paper to train and test the proposed network. Four metrics, Mean Square Error, Calculating R-squared, Square Value of Extreme Constraint, and Absolute Value of Extreme Constraint are used to measure the performance of the approaches. ResultsThe best Square Value of Extreme Constraint and Absolute Value of Extreme Constraint values we obtained on the datasets were 0.20 and 0.43, respectively. Compared with other methods, the estimation accuracy of FM-Net is improved by more than 53.53%. ConclusionsThe results of experiments demonstrate that the proposed network can estimate the fundamental matrix successfully. It outperforms the classical algorithms and other deep learning-based methods.

Automatic RANSAC by Likelihood Maximization

In computer vision, and particularly in 3D reconstruction from images, it is customary to be faced with regression problems contaminated by outlying data. The standard and efficient method to deal with them is the Random Sample Consensus (RANSAC) algorithm. The procedure is simple and versatile, drawing random minimal samples from the data to estimate parameterized candidate models and ranking them based on the amount of compatible data. Such evaluation involves some threshold that separates inliers from outliers. In presence of unknown level of noise, as is usual in practice, it is desirable to remove the dependency on this fixed threshold and to estimate it as an additional unknown. Among the numerous variants of RANSAC, few, that we call 'automatic', propose this approach, which involves changing the maximization criterion of consensus, as it is naturally increasing with the varying threshold. An algorithm of Zach and Cohen (ICCV 2015) uses the likelihood statistics. We present the details and the implementation of their method along with quantitative and qualitative tests on standard stereovision tasks: estimation of homography, fundamental and essential matrix.

Locality-Guided Global-Preserving Optimization for Robust Feature Matching.

Feature matching is a fundamental problem in many computer vision tasks. This paper proposes a novel effective framework for mismatch removal, named LOcality-guided Global-preserving Optimization (LOGO). To identify inliers from a putative matching set generated by feature descriptor similarity, we introduce a fixed-point progressive approach to optimize a graph-based objective, which represents a two-class assignment problem regarding an affinity matrix containing global structures. We introduce a strategy that a small initial set with a high inlier ratio exploits the topology of the affinity matrix to elicit other inliers based on their reliable geometry, which enhances the robustness to outliers. Geometrically, we provide a locality-guided matching strategy, i.e., using local topology consensus as a criterion to determine the initial set, thus expanding to yield the final feature matching set. In addition, we apply local affine transformations based on reference points to determine the local consensus and similarity scores of nodes and edges, ensuring the validity and generality for various scenarios including complex nonrigid transformations. Extensive experiments demonstrate the effectiveness and robustness of the proposed LOGO, which is competitive with the current state-of-the-art methods. It also exhibits favorable potential for high-level vision tasks, such as essential and fundamental matrix estimation, image registration and loop closure detection.

Guided neighborhood affine subspace embedding for feature matching

Feature matching, which refers to determining reliable correspondences between two sets of feature points, is a fundamental component of numerous visual tasks. This paper proposes a novel method, termed as guided neighborhood affine subspace embedding (NASE), to eliminate false matches from the given tentative feature matches. Its essential philosophy is to preserve the underlying intrinsic manifold of potential true matches. Specifically, we aim to approximate the manifold of an inlier with an affine subspace fitted on its neighbors by imposing a motion-consistency constraint. Considering that the “corresponding manifold” of inliers may be biased by gross outliers, we introduce a density-based seed point selection strategy for neighborhood refinement. Based on the above two strategies, we further formulate the general feature matching problem into a mathematical optimization model and deduce a closed-form solution with linearithmic time complexity (i.e., O(NlogN)) for mismatch removal. Additionally, we devise a multi-scale strategy for neighborhood construction, making our method more robust to various degradations. Extensive experiments on general feature matching, fundamental matrix estimation, and loop closure detection demonstrate the clear superiority of NASE over the state-of-the-arts.

A General Framework of Remote Sensing Epipolar Image Generation

Epipolar images can improve the efficiency and accuracy of dense matching by restricting the search range of correspondences from 2-D to 1-D, which play an important role in 3-D reconstruction. As most of the satellite images in archives are incidental collections, which do not have rigorous stereo properties, in this paper, we propose a general framework to generate epipolar images for both in-track and cross-track stereo images. We first investigate the theoretical epipolar constraints of single-sensor and multi-sensor images and then introduce the proposed framework in detail. Considering large elevation changes in mountain areas, the publicly available digital elevation model (DEM) is applied to reduce the initial offsets of two stereo images. The left image is projected into the image coordinate system of the right image using the rational polynomial coefficients (RPCs). By dividing the raw images into several blocks, the epipolar images of each block are parallel generated through a robust feature matching method and fundamental matrix estimation, in which way, the horizontal disparity can be drastically reduced while maintaining negligible vertical disparity for epipolar blocks. Then, stereo matching using the epipolar blocks can be easily implemented and the forward intersection method is used to generate the digital surface model (DSM). Experimental results on several in-track and cross-track images, including optical-optical, SAR-SAR, and SAR-optical pairs, demonstrate the effectiveness of the proposed framework, which not only has obvious advantages in mountain areas with large elevation changes but also can generate high-quality epipolar images for flat areas. The generated epipolar images of a ZiYuan-3 pair in Songshan are further utilized to produce a high-precision DSM.

EGOF-Net: epipolar guided optical flow network for unrectified stereo matching.

It is challenging to realize stereo matching in dynamic stereo vision systems. We present an epipolar guided optical flow network (EGOF-Net) for unrectified stereo matching by estimating robust epipolar geometry with a deep cross-checking-based fundamental matrix estimation method (DCCM) and then surpassing false matches with a 4D epipolar modulator (4D-EM) module. On synthetic and real-scene datasets, our network outperforms the state-of-the-art methods by a substantial margin. Also, we test the network in an existing dynamic stereo system and successfully reconstruct the 3D point clouds. The technique can simplify the stereo vision pipeline by ticking out rectification operations. Moreover, it suggests a new opportunity for combining heuristic algorithms with neural networks. The code is available on https://github.com/psyrocloud/EGOF-Net.

Image Detection System Using Image Processing

The increasing popularity of Social Networks makes change the way people interact. These interactions produce a huge amount of data and it opens the door to new strategies and marketing analysis. According to Instagram and Tumblr, an average of 80 and 59 million photos respectively are published every day, and those pictures contain several implicit or explicit brand logos. Image recognition is one of the most important fields of image processing and computer vision. The CNNs are a very effective class of neural networks that is highly effective at the task of image classifying, object detection and other computer vision problems.in recent years, several scale- invariant features have been proposed in literature, this paper analyzes the usage of Speeded Up Robust Features (SURF) as local descriptors, and as we will see, they are not only scale-invariant features, but they also offer the advantage of being computed very efficiently. Furthermore, a fundamental matrix estimation method based on the RANSAC is applied.

Graph-Cut RANSAC: Local Optimization on Spatially Coherent Structures

We propose Graph-Cut RANSAC, GC-RANSAC in short, a new robust geometric model estimation method where the local optimization step is formulated as energy minimization with binary labeling, applying the graph-cut algorithm to select inliers. The minimized energy reflects the assumption that geometric data often form spatially coherent structures - it includes both a unary component representing point-to-model residuals and a binary term promoting spatially coherent inlier-outlier labelling of neighboring points. The proposed local optimization step is conceptually simple, easy to implement, efficient with a globally optimal inlier selection given the model parameters. Graph-Cut RANSAC, equipped with "the bells and whistles" of USAC and MAGSAC++, was tested on a range of problems using a number of publicly available datasets for homography, 6D object pose, fundamental and essential matrix estimation. It is more geometrically accurate than state-of-the-art robust estimators, fails less often and runs faster or with speed similar to less accurate alternatives. The source code is available at https://github.com/danini/graph-cut-ransac.

An Epipolar Resampling Method for Multi-View High Resolution Satellite Images Based on Block

As the basis of stereo observation, epipolar resampling is a critical technology for 3D reconstruction. With the development of the 3D reconstruction using multi-view satellite images, the epipolar resampling of the multi-view satellite images has become a new challenging problem. In this paper, we propose an epipolar resampling method for multi-view high resolution satellite images based on block. Firstly, we establish the relationship between the vertical parallax and image block size for the epipolar resampling based on block. Using this relationship, we can choose the block size to limit the maximum vertical parallax in epipolar resampling. Secondly, we use the rational function model (RFM) to generate virtual corresponding points for fundamental matrix estimation, so image pairs with large stereo angles and poor consistency in multi-view satellite images which are difficult to extract feature matching point can be rectified. Experiments with multi-view satellite images demonstrated the universality and effectiveness of the proposed method. All the image pairs in the multi-view satellite image dataset were rectified successfully. The average epipolar resampling error is only 0.25 pixels when we rectify the multi-view satellite images acquired by WorldView-3 with the block size of 3000*3000 pixels, significantly less than the similar method.

A hybrid algorithm for calculating the global solution of the fundamental matrix problem

It is well-known that the fundamental matrix estimation problem is a non-convex problem and to date only local solutions can be claimed. The hybrid method is mainly based on some recent progress in global polynomial optimization techniques. By solving a hierarchy of semi-definite programming problems, the global solutions can be achieved or approximated as closely as desired at some relaxation order. To reduce the computational complexity and avoid numerical difficulties of this convex relaxation method when the number of unknown variables is large, the singular value decomposition (SVD) method combined with the interleaving parameterization method are integrated. The experimental results show that the proposed algorithm can calculate the global solution after only a few iterations and can achieve almost the same accuracy as the original convex relaxation method with much less time.

Ray-Space Epipolar Geometry for Light Field Cameras.

Light field essentially represents rays in space. The epipolar geometry between two light fields is an important relationship that captures ray-ray correspondences and relative configuration of two views. Unfortunately, so far little work has been done in deriving a formal epipolar geometry model that is specifically tailored for light field cameras. This is primarily due to the high-dimensional nature of the ray sampling process with a light field camera. This paper fills in this gap by developing a novel ray-space epipolar geometry which intrinsically encapsulates the complete projective relationship between two light fields, while the generalized epipolar geometry which describes relationship of normalized light fields is the specialization of the proposed model to calibrated cameras. With Plücker parameterization, we propose the ray-space projection model involving a 6×6 ray-space intrinsic matrix for ray sampling of light field camera. Ray-space fundamental matrix and its properties are then derived to constrain ray-ray correspondences for general and special motions. Finally, based on ray-space epipolar geometry, we present two novel algorithms, one for fundamental matrix estimation, and the other for calibration. Experiments on synthetic and real data have validated the effectiveness of ray-space epipolar geometry in solving 3D computer vision tasks with light field cameras.

On evaluating consensus in RANSAC surface registration

AbstractRandom Sample Consensus is a powerful paradigm that was successfully applied in various contexts, including Location Determination Problem, fundamental matrix estimation and global 3D surface registration, where many previously proposed algorithms can be interpreted as a particular implementation of this concept. In general, a set of candidate transformations is generated by some simple procedure, and an aligning transformation is chosen within this set, such that it aligns the largest portion of the input data.We observe that choosing the aligning transformation may also be interpreted as finding consensus among the candidates, which in turn involves measuring similarity of candidate rigid transformations. While it is not difficult to construct a metric that provides reasonable results, most approaches come with certain limitations and drawbacks. In this paper, we investigate possible means of measuring distances in SE(3) and compare their properties both theoretically and experimentally in a model RANSAC registration algorithm. We also propose modifications to existing measures and propose a novel method of locating the consensus transformation based on Vantage Point Tree data structure.

Soft decision optimization method for robust fundamental matrix estimation

It is easy to show that in computer vision, there is the closely coupled relation between feature matching and fundamental matrix estimation. The widely used robust methods such as RANSAC and its improved versions separately deal with feature matching and fundamental matrix estimation. Although these methods are simple to implement, their performance may be relatively low in the presence of gross outliers. By exploiting such coupled relation, the soft decision optimization method is proposed in this paper to estimate the fundamental matrix and find the inlier correspondence set together. Combing feature matching and fundamental matrix estimation, a soft decision objective function is developed to automatically remove the interference of the outliers in the candidate correspondence set. Moreover, an efficient expectation–maximization algorithm is established to find the solution to the fundamental matrix and the inlier correspondence set. Experiments on both synthesized data and real images show that the proposed method can cope with large noise and high ratio of outliers and is superior to some state-of-the-art robust methods in precision, recall, and residual error.

Accelerated Guided Sampling for Multistructure Model Fitting.

The performance of many robust model fitting techniques is largely dependent on the quality of the generated hypotheses. In this paper, we propose a novel guided sampling method, called accelerated guided sampling (AGS), to efficiently generate the accurate hypotheses for multistructure model fitting. Based on the observations that residual sorting can effectively reveal the data relationship (i.e., determine whether two data points belong to the same structure), and keypoint matching scores can be used to distinguish inliers from gross outliers, AGS effectively combines the benefits of residual sorting and keypoint matching scores to efficiently generate accurate hypotheses via information theoretic principles. Moreover, we reduce the computational cost of residual sorting in AGS by designing a new residual sorting strategy, which only sorts the top-ranked residuals of input data, rather than all input data. Experimental results demonstrate the effectiveness of the proposed method in computer vision tasks, such as homography matrix and fundamental matrix estimation.