Iterative Pose Estimation Research Articles

A general mobile manipulator automation framework for flexible tasks in controlled environments

To enable a mobile manipulator to perform human tasks from a single teaching demonstration is vital to flexible automation. We call our proposed method MMPA (Mobile Manipulator Process Automation with One-shot Teaching). Currently, there is no effective and robust MMPA framework which is not influenced by the mobile base’s parking precision. The proposed MMPA framework consists of two stages: collecting data (mobile base’s location, environment information, end-effector’s path) in the teaching stage for robot learning; letting the end-effector repeat the nearly same path as the reference path in the world frame to reproduce the work in the automation stage. More specifically, in the automation stage, the robot navigates to the specified location without the need of a precise parking. Then, based on colored point cloud registration, the proposed IPE (Iterative Pose Estimation by Eye & Hand) algorithm could estimate the accurate 6D relative parking pose of the robot arm base without the need of any marker. Finally, the robot could learn the error compensation from the parking pose’s bias to modify the end-effector’s path to make it repeat a nearly same path in the world coordinate system as recorded in the teaching stage. Hundreds of trials have been conducted with a real mobile manipulator to show the superior robustness of the system and the accuracy of the process automation regardless of the parking precision. For the released code, please contact AI@amigaga.com.

Iterative Pose Estimation for a Planar Object Using Virtual Sphere

This article proposed an iterative pose estimation for the planar object to deal with the pose ambiguity in the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Perspective n Points</i> (PnP) problem. Specifically, by utilizing the unit virtual sphere, the PnP pose estimation problem can be performed as a minimization of the error function with three independent transitional parameters, referred to as the coupling position. Then, Levenberg–Marquardt (LM) optimization algorithm is applied to acquire the coupling position of the first local minimum. Furthermore, the coupling position is represented as the two Euler angles and the vector length, which are combined with the planar points to compute the second local minimum approximation as the initialization of the second LM algorithm. Consequently, once the global coupling position in the two local minimum is decided by the lower error, the orientation and position are directly decoupled by using the singular value decomposition. It is shown that the designed pose estimation is able to achieve prescribed performance for locating and distinguishing two local minimum, and meanwhile guarantee the superior computation behavior under the pose ambiguity for the planar object. Finally, numerical simulation and physical experiment are conducted to validate the effectiveness of the proposed method, compared with the state-of-the-art PnP methods.

An iterative pose estimation algorithm based on epipolar geometry with application to multi-target tracking

This paper introduces a new algorithm for estimating the relative pose of a moving camera using consecutive frames of a video sequence. State-of-the-art algorithms for calculating the relative pose between two images use matching features to estimate the essential matrix. The essential matrix is then decomposed into the relative rotation and normalized translation between frames. To be robust to noise and feature match outliers, these methods generate a large number of essential matrix hypotheses from randomly selected minimal subsets of feature pairs, and then score these hypotheses on all feature pairs. Alternatively, the algorithm introduced in this paper calculates relative pose hypotheses by directly optimizing the rotation and normalized translation between frames, rather than calculating the essential matrix and then performing the decomposition. The resulting algorithm improves computation time by an order of magnitude. If an inertial measurement unit &#x0028 IMU &#x0029 is available, it is used to seed the optimizer, and in addition, we reuse the best hypothesis at each iteration to seed the optimizer thereby reducing the number of relative pose hypotheses that must be generated and scored. These advantages greatly speed up performance and enable the algorithm to run in real-time on low cost embedded hardware. We show application of our algorithm to visual multi-target tracking &#x0028 MTT &#x0029 in the presence of parallax and demonstrate its real-time performance on a 640 &#x00D7 480 video sequence captured on a UAV.

Fast, robust, and accurate monocular peer-to-peer tracking for surgical navigation

PurposeThis work presents a new monocular peer-to-peer tracking concept overcoming the distinction between tracking tools and tracked tools for optical navigation systems. A marker model concept based on marker triplets combined with a fast and robust algorithm for assigning image feature points to the corresponding markers of the tracker is introduced. Also included is a new and fast algorithm for pose estimation.MethodsA peer-to-peer tracker consists of seven markers, which can be tracked by other peers, and one camera which is used to track the position and orientation of other peers. The special marker layout enables a fast and robust algorithm for assigning image feature points to the correct markers. The iterative pose estimation algorithm is based on point-to-line matching with Lagrange–Newton optimization and does not rely on initial guesses. Uniformly distributed quaternions in 4D (the vertices of a hexacosichora) are used as starting points and always provide the global minimum.ResultsExperiments have shown that the marker assignment algorithm robustly assigns image feature points to the correct markers even under challenging conditions. The pose estimation algorithm works fast, robustly and always finds the correct pose of the trackers. Image processing, marker assignment, and pose estimation for two trackers are handled in less than 18 ms on an Intel i7-6700 desktop computer at 3.4 GHz.ConclusionThe new peer-to-peer tracking concept is a valuable approach to a decentralized navigation system that offers more freedom in the operating room while providing accurate, fast, and robust results.

Projection Distortion-based Object Tracking in Shader Lamp Scenarios.

Shader lamp systems augment the real environment by projecting new textures on known target geometries. In dynamic scenes, object tracking maintains the illusion if the physical and virtual objects are well aligned. However, traditional trackers based on texture or contour information are often distracted by the projected content and tend to fail. In this paper, we present a model-based tracking strategy, which directly takes advantage from the projected content for pose estimation in a projector-camera system. An iterative pose estimation algorithm captures and exploits visible distortions caused by object movements. In a closed-loop, the corrected pose allows the update of the projection for the subsequent frame. Synthetic frames simulating the projection on the model are rendered and an optical flow-based method minimizes the difference between edges of the rendered and the camera image. Since the thresholds automatically adapt to the synthetic image, a complicated radiometric calibration can be avoided. The pixel-wise linear optimization is designed to be easily implemented on the GPU. Our approach can be combined with a regular contour-based tracker and is transferable to other problems, like the estimation of the extrinsic pose between projector and camera. We evaluate our procedure with real and synthetic images and obtain very precise registration results.

Vision navigation for aircrafts based on 3D reconstruction from real-time image sequences

In this paper, we propose a novel vision navigation method based on three-dimensional (3D) reconstruction from real-time image sequences. It adapts 3D reconstruction and terrain matching to establish the correspondence between image points and 3D space points and the terrain reference (by using a digital elevation map (DEM)). An adaptive weighted orthogonal iterative pose estimation method is employed to calculate the position and attitude angle of the aircraft. Synthesized and real experiments show that the proposed method is capable of providing accurate navigation parameters for a long-endurance flight without using a global positioning system or an inertial navigation system (INS). Moreover, it can be combined with an INS to achieve an improved navigation result.

Contour-based iterative pose estimation of 3D rigid object

Estimating pose parameters of a 3D rigid object based on a 2D monocular image is a fundamental problem in computer vision. State-of-the-art methods usually assume that certain feature correspondences are available a priori between the input image and object's 3D model. This presumption makes the problem more algebraically tractable. However, when there is no feature correspondence available a priori, how to estimate the pose of a truly 3D object using just one 2D monocular image is still not well solved. In this article, a new contour-based method which solves both the pose estimation problem and the feature correspondence problem simultaneously and iteratively is proposed. The outer contour of the object is firstly extracted from the input 2D grey-level image, then a tentative point correspondence relationship is established between the extracted contour and object's 3D model, based on which object's pose parameters will be estimated; the newly estimated pose parameters are then used to revise the tentative point correspondence relationship, and the process is iterated until convergence. Experiment results are promising, showing that the authors' method has fast convergence speed and good convergence radius.

Monocular vision-based iterative pose estimation algorithm from corresponding feature points

To determine the relative pose between an object and a single camera using 2D-to-3D point correspondences, a kind of iterative methods based on inverse projection ray approach is proposed. An iterative algorithm which is divided into depth recovery stage and absolute orientation stage is also proposed. In the first stage, the optimal translation vector is first computed in terms of rotation matrix via least square method, then the depths of the observed points are estimated by projecting the estimated point orthogonally to the inverse projection ray defined by the image point, and finally 3D points are reconstructed using the estimated depths from previous step. In the second stage, the optimal rotation matrix is estimated by applying Umeyama algorithm to fitting of the 3D model points and 3D estimated points. The above two stages are repeated until the result converges. The global convergence of the two-stage iterative algorithm is proven based on the global convergence theorem. Finally, a spacecraft docking application is implemented to test the effectiveness and convergence of the proposed algorithm by mathematical simulation and physical simulation.

Accurate and robust iterative pose estimation from line correspondences

The methods for pose estimation problem from 2D to 3D feature correspondences are mainly based on point correspondences.However,it is advantageous to compute pose from line correspondences.From a novel point of view based on the Euclidean space,we presented an iterative algorithm for solving pose estimation problem by using line correspondences.The main idea is that,firstly the best rotation matrix is iteratively computed,and then the translation vector is obtained.For different groups of line correspondences,we presented different methods for getting an initial guess.Synthetic experimental data show the proposed algorithm is less sensitive to initialization,accurate and robust.

Improved iterative pose estimation algorithm using three-dimensional feature points

In a real-time vision navigation system, an accurate and fast convergent pose estimation algorithm is required for the video guidance sensor. The orthogonal iteration (OI) algorithm is fast and globally convergent, but its results have a large translation error at a close range, and sometimes it fails to give a correct rotation matrix when the data are severely corrupted, when using the 3-D feature points. When the rotation matrix solution in the OI algorithm has been refined, an efficient pose estimation algorithm is derived. Simulation of the improved algorithm shows that the rotation matrix is always proper, which in turn improves the accuracy of the translation vector. The noise resistance and the outlier tolerance are enhanced by using the improved algorithm. The two algorithms are applied to our experimental system for an unmanned vehicle rendezvous and docking simulation separately. The comparison experiments show that the relative distance error is less than 0.28% from 1.5 to 5 m, and the rotation angle error is within ±0.7 deg in 5 m using the improved algorithm. These are better than the results using the OI algorithm.

Mapping textures on 3D geometric model using reflectance image

Texture mapping on scanned objects, that is, the method of mapping current color images on a 3D geometric model measured by a range sensor, is a key technique of photometric modeling for virtual reality. Usually range and color images are obtained from different viewing positions, through two independent range and color sensors. Thus, in order to map those color images on the geometric model, it is necessary to determine relative relations between these two viewpoints. This paper focuses on the reflectance image of the laser scanner which is given as side products of range images for most of the range sensors, and proposes a new calibration method for the texture mapping using reflectance images and the iterative pose estimation using robust M-estimator. © 2005 Wiley Periodicals, Inc. Syst Comp Jpn, 36(13): 92–101, 2005; Published online in Wiley InterScience (www.interscience.wiley.com). DOI 10.1002/scj.10392

Multiple camera model-based 3-D visual servo

A multiple camera, visual servo system is presented that tracks and aligns rigid objects in 3D using wire-frame models. The system is capable of achieving frame-rate (30 Hz between two cameras) object tracking while using nonlinear iterative pose estimation. Tracking begins with a rough estimate of the positions of the objects. The relative rotation and translation of the cameras (external calibration) is calculated from the images. Six degree-of-freedom object pose estimates are computationally fused to provide tracking that is both stable and robust to occlusion. The tracking data is used to move (servo) the objects together while performing online robot Jacobian estimation. The robot is controlled in real time by joint angles with no prior knowledge of its kinematics.