Abstract
The Perspective-n-Point-and-Line (PnPL) problem, as a cornerstone in geometric computer vision, seeks to estimate the absolute pose of a calibrated camera from 3D-to-2D point and line correspondences. In this paper, we present a certifiably globally optimal and robust solution to the PnPL problem with a large number of outliers among the correspondences. Our first contribution is to reformulate the general PnPL problem as a novel constrained global optimization problem of Sum-of-Squares (SOS) polynomials using generalized geometric distances for both points and lines. Our second contribution is to efficiently solve this non-convex optimization problem by reducing it to an equivalent convex Linear Matrix Inequality (LMI) problem via a series of SOS relaxations. With these two contributions, we can develop a non-minimal solver, named SPnPL, for the outlier-free cases. The third contribution is to further add a Graduated Non-Convexity (GNC) cost function to SPnPL so as to remove outliers through closed-form iterations, which leads to a robust solver, named GNC-SPnPL. Both synthetic and real-data experiments confirm that SPnPL can mostly outperform the existing state-of-the-art PnPL algorithms and GNC-SPnPL can render accurate results even when 85% of the correspondences are outliers.
Highlights
The absolute pose estimation of a calibrated camera from 3D-to-2D correspondences ranks among the most fundamental problems in computer vision, since it has extensive applications in Augmented Reality (AR), Structure-from-Motion (SfM) pipelines, and Simultaneous Localization and Mapping (SLAM) systems
In the PnP cases, we compare our SPnP with Direct Linear Transformation (DLT) [1], EPnP [2], Optimal PnP (OPnP) [4], Direct Least Squares (DLS) [3]
In the PnL cases, we compare our SPnL with DLT, RPnL [45], and EPnL and OPnL [34]
Summary
The absolute pose estimation of a calibrated camera from 3D-to-2D correspondences ranks among the most fundamental problems in computer vision, since it has extensive applications in Augmented Reality (AR), Structure-from-Motion (SfM) pipelines, and Simultaneous Localization and Mapping (SLAM) systems. We develop the first non-minimal solver that can globally integrate the geometric distances of both points and lines as the projection errors to minimize, and further provide an exact and computationally efficient solution to its NP-hard polynomial minimization problem with a reliable guarantee of optimality. Another aspect lies in the approach for robust estimation. The first contribution is that we derive a novel generalized formulation of geometric distances for point and line correspondences in the PnPL problem, which can be further recast as the objective function of Sum of Squares (SOS) polynomials that globally quantify the 3D-to-2D projection errors from all correspondences. The experimental results unanimously demonstrate that SPnPL is mostly superior to other state-of-the-art PnPL algorithms in accuracy and GNC-SPnPL shows strong robustness against as many as 85% outliers
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Free) 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
