Absolute Camera Pose Research Articles

Retraction Note: Fast and robust absolute camera pose estimation with known focal length

Retraction Note: Fast and robust absolute camera pose estimation with known focal length

Learning single and multi-scene camera pose regression with transformer encoders

Contemporary state-of-the-art localization methods perform feature matching against a structured scene model or learn to regress the scene 3D coordinates. The resulting matches between 2D query pixels and 3D scene coordinates are used to estimate the camera pose using PnP and RANSAC, requiring the camera intrinsics for both the query and reference images. An alternative approach is to directly regress the camera pose from the query image. Although less accurate, absolute camera pose regression does not require any additional information at inference time and is typically lightweight and fast. Recently, Transformers were proposed for learning multi-scene camera pose regression, employing encoders to attend to spatially varying deep features while using decoders to embed multiple scene queries at once. In this work, we show that Transformer Encoders can aggregate and extract task-informative latent representations for learning both single- and multi- scene camera pose regression, without Transformer-Decoders. Our approach is shown to reduce the runtime and memory of previous Transformer-based multi-scene solutions, while comparing favorably with contemporary pose regression schemes and achieving state-of-the-art accuracy on multiple indoor and outdoor regression benchmarks. In particular, to the best of our knowledge, our approach is the first absolute regression approach to attain sub-meter average accuracy across outdoor scenes. We make our code publicly available at: https://github.com/yolish/transposenet.

Coarse-to-Fine Multi-Scene Pose Regression with Transformers.

Absolute camera pose regressors estimate the posi-tion and orientation of a camera given the captured image alone. Typically, a convolutional backbone with a multi-layer perceptron (MLP) head is trained using images and pose labels to embed a single reference scene at a time. Recently, this scheme was extended to learn multiple scenes by replacing the MLP head with a set of fully connected layers. In this work, we propose to learn multi-scene absolute camera pose regression with Transformers, where encoders are used to aggregate activation maps with self-attention and decoders transform latent features and scenes encoding into pose predictions. This allows our model to focus on general features that are informative for localization, while em-bedding multiple scenes in parallel. We extend our previous MS-Transformer approach [1] by introducing a mixed classification-regression architecture that improves the localization accuracy. Our method is evaluated on commonly benchmark indoor and outdoor datasets and has been shown to exceed both multi-scene and state-of-the-art single-scene absolute pose regressors. We make our code publicly available from here.

TransAPR: Absolute Camera Pose Regression With Spatial and Temporal Attention

Visual relocalization aims to estimate the absolute camera pose from an image or sequential images. Recent works tackle this problem by exploiting deep neural networks to regress camera poses. However, spatial and temporal clues from sequential images still remain underexplored, resulting in inaccurate poses and large outliers. In this work, we introduce a novel vision Transformer based absolute pose regression model, TransAPR, to tackle this problem. Upon the traditional CNN backbone, we design Transformer based spatial and temporal fusion modules respectively to realize sufficient feature interaction among the neighboring images in the sequence. A hierarchical feature aggregation (HFA) module is further designed to aggregate multi-scale and multi-level features in the pose regressor. Benefiting from these delicate designs, our model is able to generate reliable image representations for absolute pose regression, resulting in more robust localization under challenging environments. We conduct extensive experiments on various indoor and outdoor datasets and show that our method achieves state-of-the-art performance.

A Generalized 2-Point Solution for Absolute Camera Pose With Known Rotation Axis

A Generalized 2-Point Solution for Absolute Camera Pose With Known Rotation Axis

WatchPose: A View-Aware Approach for Camera Pose Data Collection in Industrial Environments.

Collecting correlated scene images and camera poses is an essential step towards learning absolute camera pose regression models. While the acquisition of such data in living environments is relatively easy by following regular roads and paths, it is still a challenging task in constricted industrial environments. This is because industrial objects have varied sizes and inspections are usually carried out with non-constant motions. As a result, regression models are more sensitive to scene images with respect to viewpoints and distances. Motivated by this, we present a simple but efficient camera pose data collection method, WatchPose, to improve the generalization and robustness of camera pose regression models. Specifically, WatchPose tracks nested markers and visualizes viewpoints in an Augmented Reality- (AR) based manner to properly guide users to collect training data from broader camera-object distances and more diverse views around the objects. Experiments show that WatchPose can effectively improve the accuracy of existing camera pose regression models compared to the traditional data acquisition method. We also introduce a new dataset, Industrial10, to encourage the community to adapt camera pose regression methods for more complex environments.

Robust Estimation of Absolute Camera Pose via Intersection Constraint and Flow Consensus.

Estimating the absolute camera pose requires 3D-to-2D correspondences of points and/or lines. However, in practice, these correspondences are inevitably corrupted by outliers, which affects the pose estimation. Existing outlier removal strategies for robust pose estimation have some limitations. They are only applicable to points, rely on prior pose information, or fail to handle high outlier ratios. By contrast, we propose a general and accurate outlier removal strategy. It can be integrated with various existing pose estimation methods originally vulnerable to outliers, and is applicable to points, lines, and the combination of both. Moreover, it does not rely on any prior pose information. Our strategy has a nested structure composed of the outer and inner modules. First, our outer module leverages our intersection constraint, i.e., the projection rays or planes defined by inliers intersect at the camera center. Our outer module alternately computes the inlier probabilities of correspondences and estimates the camera pose. It can run reliably and efficiently under high outlier ratios. Second, our inner module exploits our flow consensus. The 2D displacement vectors or 3D directed arcs generated by inliers exhibit a common directional regularity, i.e., follow a dominant trend of flow. Our inner module refines the inlier probabilities obtained at each iteration of our outer module. This refinement improves the accuracy and facilitates the convergence of our outer module. Experiments on both synthetic data and real-world images have shown that our method outperforms state-of-the-art approaches in terms of accuracy and robustness.

RETRACTED ARTICLE: Fast and robust absolute camera pose estimation with known focal length

Some 3D computer vision techniques such as structure from motion (SFM) and augmented reality (AR) depend on a specific perspective-n-point (PnP) algorithm to estimate the absolute camera pose. However, existing PnP algorithms are difficult to achieve a good balance between accuracy and efficiency, and most of them do not make full use of the internal camera information such as focal length. In order to attack these drawbacks, we propose a fast and robust PnP (FRPnP) method to calculate the absolute camera pose for 3D compute vision. In the proposed FRPnP method, we firstly formulate the PnP problem as the optimization problem in the null space that can avoid the effects of the depth of each 3D point. Secondly, we can easily get the solution by the direct manner using singular value decomposition. Finally, the accurate information of camera pose can be obtained by optimization strategy. We explore four ways to evaluate the proposed FRPnP algorithm with synthetic dataset, real images, and apply it in the AR and SFM system. Experimental results show that the proposed FRPnP method can obtain the best balance between computational cost and precision, and clearly outperforms the state-of-the-art PnP methods.

Computationally efficient approach to three-dimensional point cloud reconstruction from video image sequences

This paper presents a computationally efficient solution to three-dimensional point cloud reconstruction from video image sequences that are captured by a hand-held camera. Our solution starts with a frame selection step to remove frames that cause physically nonrealizable reconstruction outcomes. Then, a computationally efficient approach for obtaining the absolute camera pose is introduced based on pairwise relative camera poses. This is followed by a computationally efficient rotation registration to update the absolute camera pose. The reconstruction results obtained based on actual video sequences indicate lower computation times and lower reprojection errors of the introduced approach compared to the conventional approach.

Estimation of the absolute camera pose for environment recognition of industrial robotics

The problem of estimating and predicting the absolute camera pose (the position and orientation of a camera with respect to the world coordinate system) is approached by fusion of measurements from inertial sensors (accelerometers and gyroscopes) and robot control system. The sensor fusion approach described in this paper is based on non-linear filtering of multi-rate extended Kalman filter. In this way, camera pose estimates, with improved accuracy and sampling rate as well as reduced computation complexity, are available. Experiments that an industrial robot moves the sensors (camera and inertial measurement unit) in an indoor-global positioning system (GPS)-based global referencing system are presented. The absolute camera pose, provided by indoor-GPS, allows for a performance evaluation. The experimental results confirm also the dynamics improvement of the estimated absolute camera pose.

A Novel Solution to the P4P Problem for an Uncalibrated Camera

We present a novel solution for the absolute camera pose and the camera calibration (effective focal length and aspect ratio) based on perspective four point (P4P) problem. By converting perspective transformation to affine transformation and using invariance to 3D affine transformation, we explore the relationship between the dual image of the absolute conic (DIAC) and the world coordinate of camera optical center and show how the coplanar and noncoplanar cases are cast into the problems of solving a quadratic polynomial equation and an eighth degree polynomial equation in a single variable respectively using only linear algebra. In particular, geometric configurations for infinite solutions of the coplanar case are explored. We also confirm the conclusion that the upper bound of eight real solutions for noncoplanar case is attainable by an example. The performance and usefulness of our novel solution are demonstrated by thorough testing on both synthetic and real data.