Absolute Pose Research Articles

Image Uncertainty-Based Absolute Camera Pose Estimation with Fibonacci Outlier Elimination

Estimating the pose of a camera is a vital requirement in real-world applications like virtual reality, structure from motion, vision-assisted robot localization and manipulation. The existing Perspective-n-Point (PnP) based pose estimation algorithms have poor accuracy in presence of noise and outliers. Hence, they are combined with the Random Sample Consensus strategies to eliminate the outliers prior to pose estimation and to produce accurate results at the expense of computation time. With this concern, an Image Uncertainty-based Perspective-n-Point (IUPnP) with Fibonacci-based outlier rejection is proposed to accurately estimate the absolute pose of a calibrated camera with a minimum computation load. The uncertainties of the spherically normalized camera coordinates are formulated in the tangent space of the camera coordinate system and the initial pose is estimated using Singular Value Decomposition. The correspondences with tangent space residual exceeding the threshold values, are classified as outliers and then rejected iteratively. In order to prevent the inlier rejections and to improve the pose estimates, the threshold values are updated eventually using the Fibonacci technique. Finally, the estimated pose values are refined, using Gauss-Newton optimization. The proposed IUPnP algorithm is tested with synthetic data and real image data to validate its performance by comparing with the existing PnP algorithms in terms of accuracy. The results show that the proposed technique produces better pose estimates for correspondences with 50% outliers, than the state-of-art techniques do.

Rolling Shutter Camera Absolute Pose

We present minimal, non-iterative solutions to the absolute pose problem for images from rolling shutter cameras. The absolute pose problem is a key problem in computer vision and rolling shutter is present in a vast majority of today's digital cameras. We discuss several camera motion models and propose two feasible rolling shutter camera models for a polynomial solver. In previous work a linearized camera model was used that required an initial estimate of the camera orientation. We show how to simplify the system of equations and make this solver faster. Furthermore, we present a first solution of the non-linearized camera orientation model using the Cayley parameterization. The new solver does not require any initial camera orientation estimate and therefore serves as a standalone solution to the rolling shutter camera pose problem from six 2D-to-3D correspondences. We show that our algorithms outperform P3P followed by a non-linear refinement using a rolling shutter model.

A universal, closed-form approach for absolute pose problems

We propose a general approach for absolute pose problems including the well known perspective-n-point (PnP) problem, its generalized variant (GPnP) with and without scale, and the pose from 2D line correspondences (PnL). These have received a tremendous attention in the computer vision community during the last decades. However, it was only recently that efficient, globally optimal, closed-form solutions have been proposed, which can handle arbitrary numbers of correspondences including minimal configurations as well as over-constrained cases with linear complexity. We follow the general scheme by eliminating the linear parameters first, which results in a least squares error function that only depends on the non-linear rotation and a small symmetric coefficient matrix of fixed size. Then, in a second step the rotation is solved with algorithms which are derived using methods from algebraic geometry such as the Gröbner basis method. We propose a unified formulation based on a representation with orthogonal complements which allows to combine different types of constraints elegantly in one single framework. We show that with our unified formulation existing polynomial solvers can be interchangeably applied to problem instances other than those they were originally proposed for. It becomes possible to compare them on various registrations problems with respect to accuracy, numerical stability, and computational speed. Our compression procedure not only preserves linear complexity, it is even faster than previous formulations. For the second step we also derive an own algebraic equation solver, which can additionally handle the registration from 3D point-to-point correspondences, where other rotation solvers fail. Finally, we also present a marker-based SLAM approach with automatic registration to a target coordinate system based on partial and distributed reference information. It represents an application example that goes beyond classical camera pose estimation from image measurements and also serves for evaluation on real data.

An efficient error prediction and compensation method for coordinated five-axis machine tools under variable temperature

I n a dual-machine system, the positioning accuracy of coordinated machine tools is one of the most important performances. Considering the quasi-static errors of the developed system, the paper presents an effective error prediction and compensation method for coordinated five-axis machine tools under variable temperature. Firstly, for error prediction at a constant temperature, the workspace of coordinated machines is divided into the subspaces of translation and rotation axes, and an error prediction algorithm is presented with the known errors at sampled points. When the temperature changes, the thermal errors of the coordinated machines are analyzed by the simulation and error measurement, and an interpolation method is proposed to quickly determine the sampled data at any temperature. For error compensation, a recursive algorithm is proposed to compensate the absolute and relative pose error of dual machines. Validation experiments have been conducted to verify the proposed method on the developed system. The results have shown that, compared with previous method, the method presented in this paper is more accurate and time-efficient for error prediction and compensation under variable temperature conditions.

Reflectance Intensity Assisted Automatic and Accurate Extrinsic Calibration of 3D LiDAR and Panoramic Camera Using a Printed Chessboard

This paper presents a novel method for fully automatic and convenient extrinsic calibration of a 3D LiDAR and a panoramic camera with a normally printed chessboard. The proposed method is based on the 3D corner estimation of the chessboard from the sparse point cloud generated by one frame scan of the LiDAR. To estimate the corners, we formulate a full-scale model of the chessboard and fit it to the segmented 3D points of the chessboard. The model is fitted by optimizing the cost function under constraints of correlation between the reflectance intensity of laser and the color of the chessboard’s patterns. Powell’s method is introduced for resolving the discontinuity problem in optimization. The corners of the fitted model are considered as the 3D corners of the chessboard. Once the corners of the chessboard in the 3D point cloud are estimated, the extrinsic calibration of the two sensors is converted to a 3D-2D matching problem. The corresponding 3D-2D points are used to calculate the absolute pose of the two sensors with Unified Perspective-n-Point (UPnP). Further, the calculated parameters are regarded as initial values and are refined using the Levenberg-Marquardt method. The performance of the proposed corner detection method from the 3D point cloud is evaluated using simulations. The results of experiments, conducted on a Velodyne HDL-32e LiDAR and a Ladybug3 camera under the proposed re-projection error metric, qualitatively and quantitatively demonstrate the accuracy and stability of the final extrinsic calibration parameters.

Bidirectional invariant representation of rigid body motions and its application to gesture recognition and reproduction

In this paper we propose a new bidirectional invariant motion descriptor of a rigid body. The proposed invariant representation is not affected by rotations, translations, time, linear and angular scaling. Invariant properties of the proposed representation enable to recognize gestures in realistic scenarios with unexpected variations (e.g., changes in user’s initial pose, execution time or an observation point), while Cartesian trajectories are sensitive to these changes. The proposed invariant representation also allows reconstruction of the original motion trajectory, which is useful for human-robot interaction applications where a robot recognizes human actions and executes robot’s proper behaviors using same descriptors. By removing the dependency on absolute pose and scaling factors of the Cartesian trajectories the proposed descriptor achieves flexibility to generate different motion instances from the same invariant representation. In order to illustrate the effectiveness of our proposed descriptor in motion recognition and generation, it is tested on three datasets and experiments on a NAO humanoid robot and a KUKA LWR IV $$+$$ manipulator and compared with other existing invariant representations.

Online absolute pose compensation and steering control of industrial robot based on six degrees of freedom laser measurement

In-situ intelligent manufacturing for large-volume equipment requires industrial robots with absolute high-accuracy positioning and orientation steering control. Conventional robots mainly employ an offline calibration technology to identify and compensate key robotic parameters. However, the dynamic and static parameters of a robot change nonlinearly. It is not possible to acquire a robot’s actual parameters and control the absolute pose of the robot with a high accuracy within a large workspace by offline calibration in real-time. This study proposes a real-time online absolute pose steering control method for an industrial robot based on six degrees of freedom laser tracking measurement, which adopts comprehensive compensation and correction of differential movement variables. First, the pose steering control system and robot kinematics error model are constructed, and then the pose error compensation mechanism and algorithm are introduced in detail. By accurately achieving the position and orientation of the robot end-tool, mapping the computed Jacobian matrix of the joint variable and correcting the joint variable, the real-time online absolute pose compensation for an industrial robot is accurately implemented in simulations and experimental tests. The average positioning error is 0.048 mm and orientation accuracy is better than 0.01 deg. The results demonstrate that the proposed method is feasible, and the online absolute accuracy of a robot is sufficiently enhanced.

Trajectory-Based Place-Recognition for Efficient Large Scale Localization

Place recognition is a core competency for any visual simultaneous localization and mapping system. Identifying previously visited places enables the creation of globally accurate maps, robust relocalization, and multi-user mapping. To match one place to another, most state-of-the-art approaches must decide a priori what constitutes a place, often in terms of how many consecutive views should overlap, or how many consecutive images should be considered together. Unfortunately, such threshold dependencies limit their generality to different types of scenes. In this paper, we present a placeless place recognition algorithm using a novel match-density estimation technique that avoids heuristically discretizing the space. Instead, our approach considers place recognition as a problem of continuous matching between image streams, automatically discovering regions of high match density that represent overlapping trajectory segments. The algorithm uses well-studied statistical tests to identify the relevant matching regions which are subsequently passed to an absolute pose algorithm to recover the geometric alignment. We demonstrate the efficiency and accuracy of our methodology on three outdoor sequences, including a comprehensive evaluation against ground-truth from publicly available datasets that shows our approach outperforms several state-of-the-art algorithms for place recognition. Furthermore we compare our overall algorithm to the currently best performing system for global localization and show how we outperform the approach on challenging indoor and outdoor datasets.

Automatic calibration of camera sensor networks based on 3D texture map information

To construct an intelligent space with a distributed camera sensor network, pre-calibration of all cameras (i.e., determining the absolute poses of each camera) is an essential task that is extremely tedious. This paper considers the automatic calibration method for camera sensor networks based on 3D texture map information of a given environment. In other words, this paper solves a global localization problem for the poses of the camera sensor networks given the 3D texture map information. To manage the complete calibration problem, we propose a novel image descriptor based on quantized line parameters in the Hough space (QLH) to perform a particle filter-based matching process between line features extracted from both a distributed camera image and the 3D texture map information. We evaluate the proposed method in a simulation environment with a virtual camera network and in a real environment with a wireless camera sensor network. The results demonstrate that the proposed system can calibrate complete external camera parameters successfully.

Unscented Kalman Filter for Spacecraft Pose Estimation Using Twistors

This study presents a new method to solve the pose estimation problem of the combined spacecraft translational and rotational motion using twistors. The twistor representation enables the use of the unscented Kalman filter in the proposed pose estimation method to achieve a higher estimation accuracy than can be achieved with the method using a dual-quaternion-based extended Kalman filter. The unscented Kalman filter is difficult to design using dual quaternions because of the normalization constraint problem. To overcome this problem, the twistor representation in the framework of dual numbers is introduced to describe the spacecraft local pose error, whereas dual quaternions are used to describe the absolute pose. The unscented Kalman filter schemes, each of which uses different measurements, are designed based on the twistor representation. The proposed unscented Kalman filter schemes using twistors are compared with the dual-quaternion-based multiplicative extended Kalman filter through simulations. The results demonstrate the effectiveness of the proposed method and indicate that the twistor-based unscented Kalman filter is superior to the dual-quaternion-based multiplicative extended Kalman filter when both the initial estimation errors are large and the observation model is highly nonlinear.

Robust hazard matching approach for visual navigation application in planetary landing

Visual navigation is favored in future planetary landing mission for its ability of providing the absolute pose of lander with respect to the landing spot. However, such method often brings much computational burden to spacecraft, whose hardware is specially degraded for reliability issue. In addition, unlike natural ground environment, planetary surface is often barren and short of distinctive landmarks, which also poses great challenge to visual navigation. Planetary terrain hazard poses potential threat to planetary landing mission, some large-size hazards like boulders and craters must be targeted before a landing mission and detected in real-time during the landing phase for spacecraft to make necessary avoiding moves. It is expected that an autonomous safe landing and hazard avoidance system will bring much more scientific rewards to us. To cater for the requirements of such system, a robust and fast terrain hazard matching approach designed for visual navigation is proposed in this paper. First, image positions of detected hazards are used to generate affine invariants, after that, affine invariants are exploited to generate representative sets to describe hazard, thirdly, two finite set based hazard matching algorithms that based upon seeking similarity of representative sets are proposed, in which the advantage of set based feature matching technique is highlighted. Validity of our proposed algorithm is demonstrated by the comparison with state-of-the-art feature matching algorithms using real planetary images. Finally, different visual navigation scenarios were simulated by using a landscape generation toolbox, from which the effect of hazard detection on proposed hazard matching algorithm is further examined.

A new 2-point absolute pose estimation algorithm under planar motion

Several pose estimation algorithms, such as n-point and perspective n-point (PnP), have been introduced over the last few decades to solve the relative and absolute pose estimation problems in robotics research. Since the n-point algorithms cannot decide the real scale of robot motion, the PnP algorithms are often addressed to find the absolute scale of motion. This paper introduce a new PnP algorithm which use only two 3D–2D correspondences by considering only planar motion. Experiment results prove that the proposed algorithm solves the absolute motion in real scale with high accuracy and less computational time compared to previous algorithms.

Characterization of the Performance of Memetic Algorithms for the Automation of Bone Tracking With Fluoroscopy

Reliable knowledge of in vivo joint kinematics is fundamental in clinical medicine. Fluoroscopic motion tracking theoretically permits a millimeter/degree level of accuracy in 3-D joint motion analysis, but the reliability of the local optimization algorithm [Levenberg–Marquardt (LMA)], typically used for the pose estimation, is highly operator dependent. A new memetic algorithm (MA), hybridizing global evolution and a local search metaphor for learning, is proposed to automate the analysis and improve its reliability and robustness. The performance of MA was assessed for in silico and in vivo elbow kinematics, with and without user supervision. The best learning strategy between Lamarckian and Baldwinian evolution was identified. MA's accuracy and repeatability was quantified and compared with LMA's. The algorithm performed best using a partial Lamarckian learning strategy. The geometric symmetry of analyzed bony segments influenced the accuracy, whereas the absolute bone pose with respect to the projection geometry affected the repeatability. In contrast to LMA, MA provided robust, repeatable, and operator independent pose estimations, even for in vivo analyses. The pose can be automatically estimated with errors lower than 1 mm and 1 $^{\circ}$ for all the pose parameters except the depth position, if the investigated motion task avoids symmetric bony projection silhouettes.

Automatic Camera Pose Estimation Based on Textured 3D Map Information

To construct an intelligent space which has a distributed camera network, pre-calibration of all cameras (i.e., determining the absolute poses of each camera) is an essential task that is extremely tedious. This paper deals with automatic calibration method for the distributed camera based on 3D map information of an environment. Parameterized line features that are extracted from both a distributed camera image and the map information are transformed to Hough space and utilized for matching process in particle filter-based estimation. We evaluate the proposed method in a simulation environment with a virtual camera.

2A1-P06 移動ロボットによるカメラネットワークの自動キャリブレーション : 知能化空間における地図情報による性能向上

Pre-calibration of camera network system consisting of distributed camera sensors (i.e., determining the absolute poses of each camera) is an essential task to construct an intelligent space. This research deals with automatic calibration method of external parameters of distributed cameras by utilizing easily obtainable grid map information of the environment as prior information. The map information help seek a global minimum solution (i.e., camera parameters) within the objective function containing many local minimums. We evaluate the proposed method with a mobile robot in a wireless camera network.

Review on kinematics calibration technology of serial robots

Robot kinematics calibration is of great significance for improving robot absolute pose accuracy, which can be divided into modelbased and non-parametric kinematics calibration. For model-based kinematics calibration, kinematics calibration modeling, pose measurement of end-effector, kinematics parameters calibration and error compensation are systematically analyzed. And with the increasing demand for autonomy, the autonomous kinematics calibration is introduced. On the other hand, research status of nonparametric kinematics calibration is elaborated, the advantages and disadvantage of which are discussed respectively. On the basis, comprehensive and organized description and summarization of the three kinds of calibrations are given. Overall, aiming at robot kinematics calibration, existing achievements are summarized, a typical calibration process is carried out and calibration precautions are detailed, then calibration results of various methods are compared and analyzed. Finally, existing problems in kinematics calibration are analyzed and development prospects are presented.

Computationally efficient approach to three-dimensional point cloud reconstruction from video image sequences

This paper presents a computationally efficient solution to three-dimensional point cloud reconstruction from video image sequences that are captured by a hand-held camera. Our solution starts with a frame selection step to remove frames that cause physically nonrealizable reconstruction outcomes. Then, a computationally efficient approach for obtaining the absolute camera pose is introduced based on pairwise relative camera poses. This is followed by a computationally efficient rotation registration to update the absolute camera pose. The reconstruction results obtained based on actual video sequences indicate lower computation times and lower reprojection errors of the introduced approach compared to the conventional approach.

In vitro quantification of the performance of model-based mono-planar and bi-planar fluoroscopy for 3D joint kinematics estimation

Model-based mono-planar and bi-planar 3D fluoroscopy methods can quantify intact joints kinematics with performance/cost trade-off. The aim of this study was to compare the performances of mono- and bi-planar setups to a marker-based gold-standard, during dynamic phantom knee acquisitions. Absolute pose errors for in-plane parameters were lower than 0.6mm or 0.6° for both mono- and bi-planar setups. Mono-planar setups resulted critical in quantifying the out-of-plane translation (error<6.5mm), and bi-planar in quantifying the rotation along bone longitudinal axis (error<1.3°). These errors propagated to joint angles and translations differently depending on the alignment of the anatomical axes and the fluoroscopic reference frames. Internal-external rotation was the least accurate angle both with mono- (error<4.4°) and bi-planar (error<1.7°) setups, due to bone longitudinal symmetries. Results highlighted that accuracy for mono-planar in-plane pose parameters is comparable to bi-planar, but with halved computational costs, halved segmentation time and halved ionizing radiation dose. Bi-planar analysis better compensated for the out-of-plane uncertainty that is differently propagated to relative kinematics depending on the setup. To take its full benefits, the motion task to be investigated should be designed to maintain the joint inside the visible volume introducing constraints with respect to mono-planar analysis.

Efficient solutions to the absolute pose of cameras with unknown focal length and radial distortion by decomposition to planar and non-planar cases

Efficient solutions to the absolute pose of cameras with unknown focal length and radial distortion by decomposition to planar and non-planar cases

Comparison of Pose Correspondence Methods of Master-Slave Manipulators for Neurosurgical Robotic Systems

Performing microsurgery in the field of neurosurgery is very challenging because neurosurgeons have to suture fine vessels by maneuvering long, thin surgical instruments inserted through a small hole in the skull. In order to assist neurosurgeons, a novel master-slave surgical robotic system has been developed. The objective of the surgical robotic system is to assist neurosurgeons in performing micro surgery in deep surgical fields by providing high dexterity. However, a method of correspondence between master and slave manipulators has not yet been studied, though this is strongly related to the operability and usability of robotic surgery. In this paper, we propose two pose correspondence methods for the master and slave manipulators, axis-based relative pose correspondence and vector-based absolute pose correspondence, and their usability and operability are verified by performing pointing and suturing tasks. The experimental results show that there is a trade-off between the two correspondence methods in terms of time, length of trajectory, and the singular point problem.