Robust Camera Pose Estimation Research Articles

Outliers rejection for robust camera pose estimation using graduated non‐convexity

AbstractCamera pose estimation plays a crucial role in computer vision, which is widely used in augmented reality, robotics and autonomous driving. However, previous studies have neglected the presence of outliers in measurements, so that even a small percentage of outliers will significantly degrade precision. In order to deal with outliers, this paper proposes using a graduated non‐convexity (GNC) method to suppress outliers in robust camera pose estimation, which serves as the core of GNCPnP. The authors first reformulate the camera pose estimation problem using a non‐convex cost, which is less affected by outliers. Then, to apply a non‐minimum solver to solve the reformulated problem, the authors use the Black‐Rangarajan duality theory to transform it. Finally, to address the dependence of non‐convex optimisation on initial values, the GNC method was customised according to the truncated least squares cost. The results of simulation and real experiments show that GNCPnP can effectively handle the interference of outliers and achieve higher accuracy compared to existing state‐of‐the‐art algorithms. In particular, the camera pose estimation accuracy of GNCPnP in the case of a low percentage of outliers is almost comparable to that of the state‐of‐the‐art algorithm in the case of no outliers.

Regression-Based Camera Pose Estimation through Multi-Level Local Features and Global Features.

Accurate and robust camera pose estimation is essential for high-level applications such as augmented reality and autonomous driving. Despite the development of global feature-based camera pose regression methods and local feature-based matching guided pose estimation methods, challenging conditions, such as illumination changes and viewpoint changes, as well as inaccurate keypoint localization, continue to affect the performance of camera pose estimation. In this paper, we propose a novel relative camera pose regression framework that uses global features with rotation consistency and local features with rotation invariance. First, we apply a multi-level deformable network to detect and describe local features, which can learn appearances and gradient information sensitive to rotation variants. Second, we process the detection and description processes using the results from pixel correspondences of the input image pairs. Finally, we propose a novel loss that combines relative regression loss and absolute regression loss, incorporating global features with geometric constraints to optimize the pose estimation model. Our extensive experiments report satisfactory accuracy on the 7Scenes dataset with an average mean translation error of 0.18 m and a rotation error of 7.44° using image pairs as input. Ablation studies were also conducted to verify the effectiveness of the proposed method in the tasks of pose estimation and image matching using the 7Scenes and HPatches datasets.

Align to locate: Registering photogrammetric point clouds to BIM for robust indoor localization

Indoor localization is critical for many smart applications in built environments such as service robot navigation and facility management. Building information models (BIMs) provide new streams of spatial and visual information about building interiors that can be exploited for robust indoor localization. However, previous localization methods that used BIM were unable to achieve high precision and accuracy, limiting their practical applications. To address this challenge, a new approach, “align-to-locate (A2L)”, is proposed in this study to leverage BIM as a reference to rectify and fine-tune coarse camera poses estimated by photogrammetry. The camera pose rectification is achieved using a new registration algorithm that aligns a photogrammetric point cloud with a BIM-referenced point cloud. The experiments demonstrated the effectiveness of the proposed A2L approach, which outperformed the state of the art with a localization error of 1.07 m and an orientation deviation of 3.7°. It was also found that query point clouds generated from photographs taken along the lateral or longitude directions are more conducive for registration. While increasing the number of data collection locations and images from each location can provide higher accuracy, this approach may compromise the computational speed. This study contributes to the challenging indoor localization problem by proposing the A2L approach and evaluating its applicability for more robust camera pose estimation through point-cloud-to-BIM registration. The developed A2L approach can be integrated as a post-processing module in existing vision-based localization methods to fine-tune their estimated camera poses.

Certifiably Optimal and Robust Camera Pose Estimation From Points and Lines

The Perspective-n-Point-and-Line (PnPL) problem, as a cornerstone in geometric computer vision, seeks to estimate the absolute pose of a calibrated camera from 3D-to-2D point and line correspondences. In this paper, we present a certifiably globally optimal and robust solution to the PnPL problem with a large number of outliers among the correspondences. Our first contribution is to reformulate the general PnPL problem as a novel constrained global optimization problem of Sum-of-Squares (SOS) polynomials using generalized geometric distances for both points and lines. Our second contribution is to efficiently solve this non-convex optimization problem by reducing it to an equivalent convex Linear Matrix Inequality (LMI) problem via a series of SOS relaxations. With these two contributions, we can develop a non-minimal solver, named SPnPL, for the outlier-free cases. The third contribution is to further add a Graduated Non-Convexity (GNC) cost function to SPnPL so as to remove outliers through closed-form iterations, which leads to a robust solver, named GNC-SPnPL. Both synthetic and real-data experiments confirm that SPnPL can mostly outperform the existing state-of-the-art PnPL algorithms and GNC-SPnPL can render accurate results even when 85% of the correspondences are outliers.

Globally Consistent Indoor Mapping via a Decoupling Rotation and Translation Algorithm Applied to RGB-D Camera Output

This paper presents a novel RGB-D 3D reconstruction algorithm for the indoor environment. The method can produce globally-consistent 3D maps for potential GIS applications. As the consumer RGB-D camera provides a noisy depth image, the proposed algorithm decouples the rotation and translation for a more robust camera pose estimation, which makes full use of the information, but also prevents inaccuracies caused by noisy depth measurements. The uncertainty in the image depth is not only related to the camera device, but also the environment; hence, a novel uncertainty model for depth measurements was developed using Gaussian mixture applied to multi-windows. The plane features in the indoor environment contain valuable information about the global structure, which can guide the convergence of camera pose solutions, and plane and feature point constraints are incorporated in the proposed optimization framework. The proposed method was validated using publicly-available RGB-D benchmarks and obtained good quality trajectory and 3D models, which are difficult for traditional 3D reconstruction algorithms.

A Hierarchical Optimization Algorithm Based on GPU for Real-Time 3D Reconstruction

In machine vision sensing system, it is important to realize high-quality real-time 3D reconstruction in large-scale scene. The recent online approach performed well, but scaling up the reconstruction, it causes pose estimation drift, resulting in the cumulative error, usually requiring a large number of off-line operation to completely correct the error, reducing the reconstruction performance. In order to optimize the traditional volume fusion method and improve the old frame-to-frame pose estimation strategy, this paper presents a real-time CPU to Graphic Processing Unit reconstruction system. Based on a robust camera pose estimation strategy, the algorithm fuses all the RGB-D input values into an effective hierarchical optimization framework, and optimizes each frame according to the global camera attitude, eliminating the serious dependence on the tracking timeliness and continuously tracking globally optimized frames. The system estimates the global optimization of gestures (bundling) in real-time, supports for robust tracking recovery (re-positioning), and re-estimation of large-scale 3D scenes to ensure global consistency. It uses a set of sparse corresponding features, geometric and ray matching functions in one of the parallel optimization systems. The experimental results show that the average reconstruction time is 415 ms per frame, the ICP pose is estimated 20 times in 100.0 ms. For large scale 3D reconstruction scene, the system performs well in online reconstruction area, keeping the reconstruction accuracy at the same time.

Robust camera pose estimation by viewpoint classification using deep learning

Camera pose estimation with respect to target scenes is an important technology for superimposing virtual information in augmented reality (AR). However, it is difficult to estimate the camera pose for all possible view angles because feature descriptors such as SIFT are not completely invariant from every perspective. We propose a novel method of robust camera pose estimation using multiple feature descriptor databases generated for each partitioned viewpoint, in which the feature descriptor of each keypoint is almost invariant. Our method estimates the viewpoint class for each input image using deep learning based on a set of training images prepared for each viewpoint class. We give two ways to prepare these images for deep learning and generating databases. In the first method, images are generated using a projection matrix to ensure robust learning in a range of environments with changing backgrounds. The second method uses real images to learn a given environment around a planar pattern. Our evaluation results confirm that our approach increases the number of correct matches and the accuracy of camera pose estimation compared to the conventional method.

Identification of Errors in 3D Building Models by a Robust Camera Pose Estimation

Abstract. This paper presents a method for identification of errors in 3D building models which are results of inaccurate creation process. Error detection is carried out within the camera pose estimation. As observations, parameters of the building corners and of the line segments detected in the image are used and conditions for the coplanarity of corresponding edges are defined. For the estimation, the uncertainty of the 3D building models and image features are taken into account.

Exhaustive Linearization for Robust Camera Pose and Focal Length Estimation

We propose a novel approach for the estimation of the pose and focal length of a camera from a set of 3D-to-2D point correspondences. Our method compares favorably to competing approaches in that it is both more accurate than existing closed form solutions, as well as faster and also more accurate than iterative ones. Our approach is inspired on the EPnP algorithm, a recent O(n) solution for the calibrated case. Yet we show that considering the focal length as an additional unknown renders the linearization and relinearization techniques of the original approach no longer valid, especially with large amounts of noise. We present new methodologies to circumvent this limitation termed exhaustive linearization and exhaustive relinearization which perform a systematic exploration of the solution space in closed form. The method is evaluated on both real and synthetic data, and our results show that besides producing precise focal length estimation, the retrieved camera pose is almost as accurate as the one computed using the EPnP, which assumes a calibrated camera.

Robust camera pose estimation from unknown or known line correspondences

We address the model-to-image registration problem with line features in the following two ways. (a) We present a robust solution to simultaneously recover the camera pose and the three-dimensional-to-two-dimensional line correspondences. With weak pose priors, our approach progressively verifies the pose guesses with a Kalman filter by using a subset of recursively found match hypotheses. Experiments show our method is robust to occlusions and clutter. (b) We propose a new line feature based pose estimation algorithm, which iteratively optimizes the objective function in the object space. Experiments show that the algorithm has strong robustness to noise and outliers and that it can attain very accurate results efficiently.

A New 3D Model-Based Tracking Technique for Robust Camera Pose Estimation

In this paper we present a new robust camera pose estimation approach based on 3D lines features. The proposed method is well adapted for mobile augmented reality applications We used an Extended Kalman Filter (EKF) to incrementally update the camera pose in real-time. The principal contributions of our method include first, the expansion of the RANSAC scheme in order to achieve a robust matching algorithm that associates 2D edges from the image with the 3D line segments from the input model. And second, a new powerful framework for camera pose estimation using only 2D-3D straight-lines within an EKF. Experimental results on real image sequences are presented to evaluate the performances and the feasibility of the proposed approach in indoor and outdoor environments.