Pose Estimation Error Research Articles

Vision-Based Pose Estimation for Textureless Space Objects by Contour Points Matching

This paper presents a novel vision-based method to solve the 6-degree-of-freedom pose estimation problem of textureless space objects from a single monocular image. Our approach follows a coarse-to-fine procedure, utilizing only shape and contour information of the input image. To achieve invariance to initialization, we select a series of projection images that are similar to the input image and establish many-to-one 2D–3D correspondences by contour feature matching. Intensive attention is focused on outlier rejection and we introduce an innovative strategy to fully utilize geometric matching information to guide pose calculation. Experiments based on simulated images are carried out, and the results manifest that pose estimation error of our approach is about $1\%$ even in situations with heavy outlier correspondences.

Human-inspired robotic grasping of flat objects

In this paper, we propose a human-inspired framework for grasping domestic flat objects placed on planar support surfaces. In particular, three grasp strategies are proposed which aim to pinch small flat objects from different scenes. The framework uses representations of the robotic hand, the support surface and the target object which encapsulate rough information for the scene. Furthermore, the strategies exploit the environmental constraint of the support surface by establishing compliant contact with it, which leads to increased robustness against object geometry uncertainties as well as pose estimation errors possibly introduced by the perception system. This is inspired by how humans perform relative grasping tasks with object pose and geometry uncertainties by using compliant contact with the support surfaces. Finally, the strategy selection is determined by a decision making procedure which uses the current scene representation.

Tracking Human Pose During Robot-Assisted Dressing Using Single-Axis Capacitive Proximity Sensing

Dressing is a fundamental task of everyday living and robots offer an opportunity to assist people with motor impairments. While several robotic systems have explored robot-assisted dressing, few have considered how a robot can manage errors in human pose estimation, or adapt to human motion in real time during dressing assistance. In addition, estimating pose changes due to human motion can be challenging with vision-based techniques since dressing is often intended to visually occlude the body with clothing. We present a method to track a person's pose in real time using capacitive proximity sensing. This sensing approach gives direct estimates of distance with low latency, has a high signal-to-noise ratio, and has low computational requirements. Using our method, a robot can adjust for errors in the estimated pose of a person and physically follow the contours and movements of the person while providing dressing assistance. As part of an evaluation of our method, the robot successfully pulled the sleeve of a hospital gown and a cardigan onto the right arms of 10 human participants, despite arm motions and large errors in the initially estimated pose of the person's arm. We also show that a capacitive sensor is unaffected by visual occlusion of the body and can sense a person's body through cotton clothing.

Fuzzy adaptive extended Kalman filter for robot 3D pose estimation

PurposeEstimating the pose – position and orientation – of a moving object such as a robot is a necessary task for many applications, e.g., robot navigation control, environment mapping, and medical applications such as robotic surgery. The purpose of this paper is to introduce a novel method to fuse the information from several available sensors in order to improve the estimated pose from any individual sensor and calculate a more accurate pose for the moving platform.Design/methodology/approachPose estimation is usually done by collecting the data obtained from several sensors mounted on the object/platform and fusing the acquired information. Assuming that the robot is moving in a three-dimensional (3D) world, its location is completely defined by six degrees of freedom (6DOF): three angles and three position coordinates. Some 3D sensors, such as IMUs and cameras, have been widely used for 3D localization. Yet, there are other sensors, like 2D Light Detection And Ranging (LiDAR), which can give a very precise estimation in a 2D plane but they are not employed for 3D estimation since the sensor is unable to obtain the full 6DOF. However, in some applications there is a considerable amount of time in which the robot is almost moving on a plane during the time interval between two sensor readings; e.g., a ground vehicle moving on a flat surface or a drone flying at an almost constant altitude to collect visual data. In this paper a novel method using a “fuzzy inference system” is proposed that employs a 2D LiDAR in a 3D localization algorithm in order to improve the pose estimation accuracy.FindingsThe method determines the trajectory of the robot and the sensor reliability between two readings and based on this information defines the weight of the 2D sensor in the final fused pose by adjusting “extended Kalman filter” parameters. Simulation and real world experiments show that the pose estimation error can be significantly decreased using the proposed method.Originality/valueTo the best of the authors’ knowledge this is the first time that a 2D LiDAR has been employed to improve the 3D pose estimation in an unknown environment without any previous knowledge. Simulation and real world experiments show that the pose estimation error can be significantly decreased using the proposed method.

Single‐camera pose estimation using mirage

Recently, mirage pose estimation method was proposed for multi-camera systems. Multi-camera mirage analytically solves a system of linear equations for six pose parameters in O(n) time. Mirage promises to execute in real time with high accuracy and shows lower rotational and translational errors compared to eight other well-known perspective-n-points (PnP) methods. However, the simulated tests and real experiments showed that, in case of a single camera, the analytical system of linear equations is not solvable due to the reduced rank of the linear system that is obtained by the formulation. In this study, an important revision to mirage is proposed to support single camera systems properly. The results of simulations and real experiments demonstrate smaller pose estimation errors compared to a group of eight well-known state-of-the-art PnP methods.

Robust and Accurate Vision-Based Pose Estimation Algorithm Based on Four Coplanar Feature Points.

Vision-based pose estimation is an important application of machine vision. Currently, analytical and iterative methods are used to solve the object pose. The analytical solutions generally take less computation time. However, the analytical solutions are extremely susceptible to noise. The iterative solutions minimize the distance error between feature points based on 2D image pixel coordinates. However, the non-linear optimization needs a good initial estimate of the true solution, otherwise they are more time consuming than analytical solutions. Moreover, the image processing error grows rapidly with measurement range increase. This leads to pose estimation errors. All the reasons mentioned above will cause accuracy to decrease. To solve this problem, a novel pose estimation method based on four coplanar points is proposed. Firstly, the coordinates of feature points are determined according to the linear constraints formed by the four points. The initial coordinates of feature points acquired through the linear method are then optimized through an iterative method. Finally, the coordinate system of object motion is established and a method is introduced to solve the object pose. The growing image processing error causes pose estimation errors the measurement range increases. Through the coordinate system, the pose estimation errors could be decreased. The proposed method is compared with two other existing methods through experiments. Experimental results demonstrate that the proposed method works efficiently and stably.

Face morphology: Can it tell us something about body weight and fat?

This paper proposes a method for an automatic extraction of geometric features, related to weight parameters, from 3D facial data acquired with low-cost depth scanners. The novelty of the method relies both on the processing of the 3D facial data and on the definition of the geometric features which are conceptually simple, robust against noise and pose estimation errors, computationally efficient, invariant with respect to rotation, translation, and scale changes. Experimental results show that these measurements are highly correlated with weight, BMI, and neck circumference, and well correlated with waist and hip circumference, which are markers of central obesity.Therefore the proposed method strongly supports the development of interactive, non obtrusive systems able to provide a support for the detection of weight-related problems.

ENHANCEMENT STRATEGIES FOR FRAME-TO-FRAME UAS STEREO VISUAL ODOMETRY

Autonomous navigation of indoor unmanned aircraft systems (UAS) requires accurate pose estimations usually obtained from indirect measurements. Navigation based on inertial measurement units (IMU) is known to be affected by high drift rates. The incorporation of cameras provides complementary information due to the different underlying measurement principle. The scale ambiguity problem for monocular cameras is avoided when a light-weight stereo camera setup is used. However, also frame-to-frame stereo visual odometry (VO) approaches are known to accumulate pose estimation errors over time. Several valuable real-time capable techniques for outlier detection and drift reduction in frame-to-frame VO, for example robust relative orientation estimation using random sample consensus (RANSAC) and bundle adjustment, are available. This study addresses the problem of choosing appropriate VO components. We propose a frame-to-frame stereo VO method based on carefully selected components and parameters. This method is evaluated regarding the impact and value of different outlier detection and drift-reduction strategies, for example keyframe selection and sparse bundle adjustment (SBA), using reference benchmark data as well as own real stereo data. The experimental results demonstrate that our VO method is able to estimate quite accurate trajectories. Feature bucketing and keyframe selection are simple but effective strategies which further improve the VO results. Furthermore, introducing the stereo baseline constraint in pose graph optimization (PGO) leads to significant improvements.

ENHANCEMENT STRATEGIES FOR FRAME-TO-FRAME UAS STEREO VISUAL ODOMETRY

Abstract. Autonomous navigation of indoor unmanned aircraft systems (UAS) requires accurate pose estimations usually obtained from indirect measurements. Navigation based on inertial measurement units (IMU) is known to be affected by high drift rates. The incorporation of cameras provides complementary information due to the different underlying measurement principle. The scale ambiguity problem for monocular cameras is avoided when a light-weight stereo camera setup is used. However, also frame-to-frame stereo visual odometry (VO) approaches are known to accumulate pose estimation errors over time. Several valuable real-time capable techniques for outlier detection and drift reduction in frame-to-frame VO, for example robust relative orientation estimation using random sample consensus (RANSAC) and bundle adjustment, are available. This study addresses the problem of choosing appropriate VO components. We propose a frame-to-frame stereo VO method based on carefully selected components and parameters. This method is evaluated regarding the impact and value of different outlier detection and drift-reduction strategies, for example keyframe selection and sparse bundle adjustment (SBA), using reference benchmark data as well as own real stereo data. The experimental results demonstrate that our VO method is able to estimate quite accurate trajectories. Feature bucketing and keyframe selection are simple but effective strategies which further improve the VO results. Furthermore, introducing the stereo baseline constraint in pose graph optimization (PGO) leads to significant improvements.

Point Cloud Based Relative Pose Estimation of a Satellite in Close Range.

Determination of the relative pose of satellites is essential in space rendezvous operations and on-orbit servicing missions. The key problems are the adoption of suitable sensor on board of a chaser and efficient techniques for pose estimation. This paper aims to estimate the pose of a target satellite in close range on the basis of its known model by using point cloud data generated by a flash LIDAR sensor. A novel model based pose estimation method is proposed; it includes a fast and reliable pose initial acquisition method based on global optimal searching by processing the dense point cloud data directly, and a pose tracking method based on Iterative Closest Point algorithm. Also, a simulation system is presented in this paper in order to evaluate the performance of the sensor and generate simulated sensor point cloud data. It also provides truth pose of the test target so that the pose estimation error can be quantified. To investigate the effectiveness of the proposed approach and achievable pose accuracy, numerical simulation experiments are performed; results demonstrate algorithm capability of operating with point cloud directly and large pose variations. Also, a field testing experiment is conducted and results show that the proposed method is effective.

A Multisensor Technique for Gesture Recognition Through Intelligent Skeletal Pose Analysis

Recent advances in smart sensor technology and computer vision techniques have made the tracking of unmarked human hand and finger movements possible with high accuracy and at sampling rates of over 120 Hz. However, these new sensors also present challenges for real-time gesture recognition due to the frequent occlusion of fingers by other parts of the hand. We present a novel multisensor technique that improves the pose estimation accuracy during real-time computer vision gesture recognition. A classifier is trained offline, using a premeasured artificial hand, to learn which hand positions and orientations are likely to be associated with higher pose estimation error. During run-time, our algorithm uses the prebuilt classifier to select the best sensor-generated skeletal pose at each time step, which leads to a fused sequence of optimal poses over time. The artificial hand used to establish the ground truth is configured in a number of commonly used hand poses such as pinches and taps. Experimental results demonstrate that this new technique can reduce total pose estimation error by over 30% compared with using a single sensor, while still maintaining real-time performance. Our evaluations also demonstrate that our approach significantly outperforms many other alternative approaches such as weighted averaging of hand poses. An analysis of our classifier performance shows that the offline training time is insignificant, and our configuration achieves about 90.8% optimality for the dataset used. Our method effectively increases the robustness of touchless display interactions, especially in high-occlusion situations by analyzing skeletal poses from multiple views.

Simultaneous pose estimation and patient-specific model reconstruction from single image using maximum penalized likelihood estimation (MPLE)

Pose estimation and shape reconstruction are two common problems in pattern recognition, which oftentimes are tackled separately. But in some medical applications, both pose and shape of a target anatomy are crucial and have to be estimated from intra-operative two-dimensional images. As pose estimation and shape reconstruction are two coupled problems, previous feature-based methods solved the problems in consecutive stages utilizing statistical shape models (SSMs). Only the mean shape of SSM is used to estimate the pose by finding paired correspondences in the first stage, based on which SSM-regularized surface deformations are performed in the following stages. Such a strategy heavily depends on the paired correspondences. In this paper, bypassing correspondence establishment, a novel method is proposed to simultaneously optimize pose and shape by formulating the coupled problems as a maximum penalized likelihood estimation (MPLE). It models oriented object contours as a mixture of von Mises–Fisher Gaussian distributions, and solves the MPLE effectively using a global optimizer. It utilize the entire knowledge of SSM in both solving pose and reconstructing shape, providing robustness to large offsets in initializations. Leave-one-out cross-validations on 19 dry cadaveric femurs were performed using simulated X-ray images with accurate ground-truth, under various initial conditions. Our method achieved sub-degree rotational and sub-millimeter in-plane translational pose estimation errors, and an approximately one millimeter average mean surface-to-surface distance in shape reconstruction. The reconstruction accuracy is comparable to those reported in the literature using two or more images. The experiment results are encouraging and indicate that an accurate simultaneous 3D–2D pose estimation and surface reconstruction is achievable from one single image.

Laser-Based Online Sliding-Window Approach for UAV Loop-Closure Detection in Urban Environments

Online loop-closure detection serves as an essential task for Unmanned Aerial Vehicles (UAVs) equipped with laser scanners. Due to the inherent errors in UAVs' pose estimation, a 3-D reconstruction algorithm is adopted to perform 3-D map building, which establishes probabilistic models of the system according to the assumption of errors. To meet the demand of online loop-closure detection using sequential 2-D laser data, a robust ISW-NDT (incremental sliding-window-based NDT) approach is proposed, which compares the appearance similarity between two scans by sliding a window with fixed size. Compared with the conventional 3-D NDT approach, the proposed loop-closure detection algorithm is capable of providing superior performance in large-scale outdoor environments, achieving higher recall rate at 100% precision and ensuring successful online implementation. Experimental results show the validity and robustness of the proposed method.

Mobile robot navigation using grid line patterns via probabilistic measurement modeling

Mobile robots are generally equipped with proprioceptive motion sensors such as odometers and inertial sensors. These sensors are used for dead-reckoning navigation in an indoor environment where GPS is not available. However, this dead-reckoning scheme is susceptible to drift error in position and heading. This study proposes using grid line patterns which are often found on the surface of floors or ceilings in an indoor environment to obtain pose (i.e., position and orientation) fix information without additional external position information by artificial beacons or landmarks. The grid lines can provide relative pose information of a robot with respect to the grid structure and thus can be used to correct the pose estimation errors. However, grid line patterns are repetitive in nature, which leads to difficulties in estimating its configuration and structure using conventional Gaussian filtering that represent the system uncertainty using a unimodal function (e.g., Kalman filter). In this study, a probabilistic sensor model to deal with multiple hypotheses is employed and an online navigation filter is designed in the framework of particle filtering. To demonstrate the performance of the proposed approach, an experiment was performed in an indoor environment using a wheeled mobile robot, and the results are presented.

Decentralized Cooperative Localization Approach for Autonomous Multirobot Systems

This study proposes the use of a split covariance intersection algorithm (Split-CI) for decentralized multirobot cooperative localization. In the proposed method, each robot maintains a local cubature Kalman filter to estimate its own pose in a predefined coordinate frame. When a robot receives pose information from neighbouring robots, it employs a Split-CI based approach to fuse this received measurement with its local belief. The computational and communicative complexities of the proposed algorithm increase linearly with the number of robots in the multirobot systems (MRS). The proposed method does not require fully connected synchronous communication channels between robots; in fact, it is applicable for MRS with asynchronous and partially connected communication networks. The pose estimation error of the proposed method is bounded. As the proposed method is capable of handling independent and interdependent information of the estimations separately, it does not generate overconfidence state estimations. The performance of the proposed method is compared with several multirobot localization approaches. The simulation and experiment results demonstrate that the proposed algorithm outperforms the single-robot localization algorithms and achieves approximately the same estimation accuracy as the centralized cooperative localization approach, but with reduced computational and communicative cost.

Robust camera localisation with depth reconstruction for bronchoscopic navigation.

Bronchoscopy is a standard technique for airway examination, providing a minimally invasive approach for both diagnosis and treatment of pulmonary diseases. To target lesions identified pre-operatively, it is necessary to register the location of the bronchoscope to the CT bronchial model during the examination. Existing vision-based techniques rely on the registration between virtually rendered endobronchial images and videos based on image intensity or surface geometry. However, intensity-based approaches are sensitive to illumination artefacts, while gradient-based approaches are vulnerable to surface texture. In this paper, depth information is employed in a novel way to achieve continuous and robust camera localisation. Surface shading has been used to recover depth from endobronchial images. The pose of the bronchoscopic camera is estimated by maximising the similarity between the depth recovered from a video image and that captured from a virtual camera projection of the CT model. The normalised cross-correlation and mutual information have both been used and compared for the similarity measure. The proposed depth-based tracking approach has been validated on both phantom and in vivo data. It outperforms the existing vision-based registration methods resulting in smaller pose estimation error of the bronchoscopic camera. It is shown that the proposed approach is more robust to illumination artefacts and surface texture and less sensitive to camera pose initialisation. A reliable camera localisation technique has been proposed based on depth information for bronchoscopic navigation. Qualitative and quantitative performance evaluations show the clinical value of the proposed framework.

Near laser-scan quality 3-D face reconstruction from a low-quality depth stream

We propose a method to produce near laser-scan quality 3-D face models of a freely moving user with a low-cost, low resolution range sensor in real-time. Our approach does not require any prior knowledge about the geometry of a face and can produce faithful geometric models of any star-shaped object. We use a cylindrical representation, which enables us to efficiently process the 3-D mesh by applying 2-D filters.We use the first frame as a reference and incrementally build the model by registering each subsequent cloud of 3-D points to the reference using the ICP (Iterative Closest Point) algorithm implemented on a GPU (Graphics Processing Unit). The registered point clouds are merged into a single image through a cylindrical representation. The noise from the sensor and from the pose estimation error is removed with a temporal integration and a spatial smoothing of the successively incremented model. To validate our approach, we quantitatively compare our model to laser scans, and show comparable accuracy.11This paper extends the method presented in [15].

Head Pose Estimation From a 2D Face Image Using 3D Face Morphing With Depth Parameters

This paper presents estimation of head pose angles from a single 2D face image using a 3D face model morphed from a reference face model. A reference model refers to a 3D face of a person of the same ethnicity and gender as the query subject. The proposed scheme minimizes the disparity between the two sets of prominent facial features on the query face image and the corresponding points on the 3D face model to estimate the head pose angles. The 3D face model used is morphed from a reference model to be more specific to the query face in terms of the depth error at the feature points. The morphing process produces a 3D face model more specific to the query image when multiple 2D face images of the query subject are available for training. The proposed morphing process is computationally efficient since the depth of a 3D face model is adjusted by a scalar depth parameter at feature points. Optimal depth parameters are found by minimizing the disparity between the 2D features of the query face image and the corresponding features on the morphed 3D model projected onto 2D space. The proposed head pose estimation technique was evaluated on two benchmarking databases: 1) the USF Human-ID database for depth estimation and 2) the Pointing'04 database for head pose estimation. Experiment results demonstrate that head pose estimation errors in nodding and shaking angles are as low as 7.93° and 4.65° on average for a single 2D input face image.

Point cloud modeling using the homogeneous transformation for non-cooperative pose estimation

A modeling process to simulate point cloud range data that a lidar (light detection and ranging) sensor produces is presented in this paper in order to support the development of non-cooperative pose (relative attitude and position) estimation approaches which will help improve proximity operation capabilities between two adjacent vehicles. The algorithms in the modeling process were based on the homogeneous transformation, which has been employed extensively in robotics and computer graphics, as well as in recently developed pose estimation algorithms. Using a flash lidar in a laboratory testing environment, point cloud data of a test article was simulated and compared against the measured point cloud data. The simulated and measured data sets match closely, validating the modeling process. The modeling capability enables close examination of the characteristics of point cloud images of an object as it undergoes various translational and rotational motions. Relevant characteristics that will be crucial in non-cooperative pose estimation were identified such as shift, shadowing, perspective projection, jagged edges, and differential point cloud density. These characteristics will have to be considered in developing effective non-cooperative pose estimation algorithms. The modeling capability will allow extensive non-cooperative pose estimation performance simulations prior to field testing, saving development cost and providing performance metrics of the pose estimation concepts and algorithms under evaluation. The modeling process also provides “truth” pose of the test objects with respect to the sensor frame so that the pose estimation error can be quantified.

2A1-T03 ステレオビジョンを備えたロボットヘッドのリアルタイム位置姿勢推定

We describe a location and pose estimation of a robotic head based on three dimensional coordinate of three point markers. Computation for the location and pose estimation is carried out at 30 fps, which is a frame rate of camera, and with low latency by using FPGA. In experiments for accuracy evaluation, an error in pose estimation was less than 3 degrees. This can be used for a pose estimation of humanoid head by combining complementary other methods such as one with inertial sensors.