6-DOF Pose Estimation Research Articles

Low-cost and Retrofittable Pose Estimation of Rigid Objects Using Infrared Markers

We present a system for the 6-DOF pose estimation of rigid objects which can be used for tool tracking in manual manufacturing applications like quality assurance or assembly assistance. To be practical and widely adaptable, the system must be low-cost, passive, impact resistant, retrofittable and applicable to arbitrary rigid objects. Therefore we attach circular markers of retroreflective self-adhesive film to relevant objects. A stereo camera system with IR illumination tracks the 3D positions of these markers and uses known marker constellations to estimate the object pose. Additionally a simple teach-in procedure which learns marker constellations of new objects is presented.

Latent-Class Hough Forests for 6DoF Object Pose Estimation.

In this paper we present Latent-Class Hough Forests, a method for object detection and 6 DoF pose estimation in heavily cluttered and occluded scenarios. We adapt a state of the art template matching feature into a scale-invariant patch descriptor and integrate it into a regression forest using a novel template-based split function. We train with positive samples only and we treat class distributions at the leaf nodes as latent variables. During testing we infer by iteratively updating these distributions, providing accurate estimation of background clutter and foreground occlusions and, thus, better detection rate. Furthermore, as a by-product, our Latent-Class Hough Forests can provide accurate occlusion aware segmentation masks, even in the multi-instance scenario. In addition to an existing public dataset, which contains only single-instance sequences with large amounts of clutter, we have collected two, more challenging, datasets for multiple-instance detection containing heavy 2D and 3D clutter as well as foreground occlusions. We provide extensive experiments on the various parameters of the framework such as patch size, number of trees and number of iterations to infer class distributions at test time. We also evaluate the Latent-Class Hough Forests on all datasets where we outperform state of the art methods.

Surface-Based Detection and 6-DoF Pose Estimation of 3-D Objects in Cluttered Scenes

In this paper, we propose a novel approach for both 3-D object detection and six-degree-of-freedom (6-DoF) pose estimation based on smooth 3-D surface segments and their visual signatures in cluttered scenes with objects partially occluded. It is an appearance-based approach that does not require precise geometric models of the objects. We introduce a robust and flexible strategy to build object models based on segmented smooth surfaces of RGB-D images. We next introduce a strategy to detect objects and estimate their 6-DoF poses from a single image by taking full advantage of the surface-based object models. Our approach can reconstruct objects in cluttered scenes from a single view, where different objects or multiple instances of the same object can be stacked together or physically attached and occlude one another. It detects objects and estimates their 6-DoF poses even if the objects have transparent regions. Experimental results and comparison with related work demonstrate the effectiveness of our approach.

An Indoor Backpack System for 2-D and 3-D Mapping of Building Interiors

This letter presents a backpack mapping system for creating indoor 2-D and 3-D maps of building interiors. For many applications, indoor mobile mapping provides a 3-D structure via an indoor map. Because there are significant roll and pitch motions of the indoor mobile mapping system, the need arises for a moving mobile system with 6 degrees of freedom (DOFs) ( $x$ , $y$ , and $z$ positions and roll, yaw, and pitch angles). First, we present a 6-DOF pose estimation algorithm by fusing 2-D laser scanner data with inertial sensor data using an extended Kalman filter-based method. The estimated 6-DOF pose is used as the initialized transformation for consecutive map alignment in 3-D map building. The 6-DOF pose gives a full 3-D estimation of the system pose and is used to accelerate the map alignment process and also align the two maps directly when there are few or no overlapping areas between the maps. Our results show that the proposed system effectively builds a consistent 2-D grid map and a 3-D point cloud map of an indoor environment.

Co-Robotic Cane: A New Robotic Navigation Aid for the Visually Impaired.

This paper presents a new robotic navigation aid, called Co-Robotic Cane (CRC). The CRC uses a 3D camera for both pose estimation and object recognition in an unknown indoor environment. The 6-DOF pose estimation method determines the CRC's pose change by an egomotion estimation method and the iterative closest point algorithm and reduces the pose integration error by a pose graph optimization algorithm. The pose estimation method does not require any prior knowledge of the environment. The object recognition method detects indoor structures such as stairways, doorways, etc. and objects such as tables, computer monitors, etc. by a Gaussian Mixture Model based pattern recognition method. Some structures/objects (e.g., stairways) can be used as navigational waypoints and the others for obstacle avoidance. The CRC can be used in either robot cane (active) mode or white cane (passive) mode. In the active mode it guides the user by steering itself into the desired direction of travel, while in the passive mode it functions as a computer-vision-enhanced white cane. The CRC is a co-robot. It can detect human intent and use the intent to select a suitable mode automatically.

6-DOF Pose Estimation of a Robotic Navigation Aid by Tracking Visual and Geometric Features.

This work was motivated by the limitations of the existing navigation technology for the visually impaired. Most of the existing methods use a point/line measurement sensor for indoor object detection. Therefore, they lack capability in detecting 3D objects and positioning a blind traveler. Stereovision has been used in recent research. However, it cannot provide reliable depth data for object detection. Also, it tends to produce a lower localization accuracy because its depth measurement error quadratically increases with the true distance. This paper suggests a new approach for navigating a blind traveler. The method uses a single 3D time-of-flight camera for both 6-DOF PE and 3D object detection and thus results in a small-sized but powerful RNA. Due to the camera's constant depth accuracy, the proposed egomotion estimation method results in a smaller error than that of existing methods. A new EKF method is proposed to integrate the egomotion into the RNA's 6-DOF pose in the world coordinate system by tracking both visual and geometric features of the operating environment. The proposed method substantially reduces the pose error of a standard EKF method and thus supports a longer range navigation task. One limitation of the method is that it requires a feature-rich environment to work well.

Robust and accurate monocular visual navigation combining IMU for a quadrotor

In this paper, we present a multi-sensor fusion based monocular visual navigation system for a quadrotor with limited payload, power and computational resources. Our system is equipped with an inertial measurement unit (IMU), a sonar and a monocular down-looking camera. It is able to work well in GPS-denied and markerless environments. Different from most of the keyframe-based visual navigation systems, our system uses the information from both keyframes and keypoints in each frame. The GPU-based speeded up robust feature (SURF) is employed for feature detection and feature matching. Based on the flight characteristics of quadrotor, we propose a refined preliminary motion estimation algorithm combining IMU data. A multi-level judgment rule is then presented which is beneficial to hovering conditions and reduces the error accumulation effectively. By using the sonar sensor, the metric scale estimation problem has been solved. We also present the novel IMU+3P (IMU with three point correspondences) algorithm for accurate pose estimation. This algorithm transforms the 6-DOF pose estimation problem into a 4-DOF problem and can obtain more accurate results with less computation time. We perform the experiments of monocular visual navigation system in real indoor and outdoor environments. The results demonstrate that the monocular visual navigation system performing in real-time has robust and accurate navigation results of the quadrotor.

2A2-M02 ICPアルゴリズムと3軸慣性情報を用いた位置・姿勢変化に頑健な3次元地図作成

2A2-M02 ICPアルゴリズムと3軸慣性情報を用いた位置・姿勢変化に頑健な3次元地図作成

Global Localization from Monocular SLAM on a Mobile Phone

We propose the combination of a keyframe-based monocular SLAM system and a global localization method. The SLAM system runs locally on a camera-equipped mobile client and provides continuous, relative 6DoF pose estimation as well as keyframe images with computed camera locations. As the local map expands, a server process localizes the keyframes with a pre-made, globally-registered map and returns the global registration correction to the mobile client. The localization result is updated each time a keyframe is added, and observations of global anchor points are added to the client-side bundle adjustment process to further refine the SLAM map registration and limit drift. The end result is a 6DoF tracking and mapping system which provides globally registered tracking in real-time on a mobile device, overcomes the difficulties of localization with a narrow field-of-view mobile phone camera, and is not limited to tracking only in areas covered by the offline reconstruction.

CLIPS – a camera and laser-based indoor positioning system

This article presents a detailed description of an optical indoor positioning system named CLIPS, short for camera and laser-based indoor positioning system. The main objective of CLIPS is pose estimation of a mobile camera with respect to a static projector on a tripod. For that reason, a reference field of red and green laser spots is projected on any surface in an indoor environment. When the reference field is captured by the mobile camera, the geometric relationship between the camera pose and the projector can be established by the coplanarity constraint. Our approach enables full 6 DOF pose estimation of the mobile camera. This article describes the basic concept of CLIPS, the projector calibration, the camera pose estimation and the introduction of the system scale. In addition, the performance of CLIPS regarding achievable accuracy is analysed.

6DoF object pose measurement by a monocular manifold-based pattern recognition technique

In this paper, a novel solution to the compound problem of object recognition and 3D pose estimation is presented. An accurate measurement of the geometrical configuration of a recognized target, relative to a known coordinate system, is of fundamental importance and constitutes a prerequisite for several applications such as robot grasping or obstacle avoidance. The proposed method lays its foundations on the following assumptions: (a) the same object captured under varying viewpoints and perspectives represents data that could be projected onto a well-established and highly distinguishable subspace; (b) totally different objects observed under the same viewpoints and perspectives share identical 3D pose that can be sufficiently modeled to produce a generalized model. Toward this end, we propose an advanced architecture that allows both recognizing patterns and providing efficient solution for 6DoF pose estimation. We employ a manifold modeling architecture that is grounded on a part-based representation of an object, which in turn, is accomplished via an unsupervised clustering of the extracted visual cues. The main contributions of the proposed framework are: (a) the proposed part-based architecture requires minimum supervision, compared to other contemporary solutions, whilst extracting new features encapsulating both appearance and geometrical attributes of the objects; (b) contrary to related projects that extract high-dimensional data, thus, increasing the complexity of the system, the proposed manifold modeling approach makes use of low dimensionality input vectors; (c) the formulation of a novel input–output space mapping that outperforms the existing dimensionality reduction schemes. Experimental results justify our theoretical claims and demonstrate the superiority of our method comparing to other related contemporary projects.

Handheld Augmented Reality involving gravity measurements

This article is a revised version of an earlier work on Gravity-Aware Handheld Augmented Reality (AR) (Kurz and Benhimane, 2011 [1]), which investigates how different stages in handheld AR applications can benefit from knowing the direction of the gravity measured with inertial sensors. It presents approaches to improve the description and matching of feature points, detection and tracking of planar templates, and the visual quality of the rendering of virtual 3D objects by incorporating the gravity vector. In handheld AR, both the camera and the display are located in the user's hand and therefore can be freely moved. The pose of the camera is generally determined with respect to piecewise planar objects that have a static and known orientation with respect to gravity.In the presence of (close to) vertical surfaces, we show how Gravity-Aligned Feature Descriptors (GAFDs) improve the initialization of tracking algorithms relying on feature point descriptor-based approaches in terms of quality and performance. For (close to) horizontal surfaces, we propose to use the gravity vector to rectify the camera image and detect and describe features in the rectified image. The resulting Gravity-Rectified Feature Descriptors (GREFDs) provide an improved precision–recall characteristic and enable faster initialization, in particular under steep viewing angles. Gravity-rectified camera images also allow for real-time 6 DoF pose estimation using an edge-based object detection algorithm handling only 4 DoF similarity transforms. Finally, the rendering of virtual 3D objects can be made more realistic and plausible by taking into account the orientation of the gravitational force in addition to the relative pose between the handheld device and a real object.In comparison to the original paper, this work provides a more elaborate evaluation of the presented algorithms. We propose a method enabling the evaluation of inertial-sensor aided visual tracking methods without real inertial sensor data. By synthesizing gravity measurements from ground truth camera poses, we benchmark our algorithms on a large existing dataset. Based on this approach, we also develop and evaluate a gravity-adaptive approach that performs image-rectification only when beneficial.

Vision-based Positioning with a Single Camera and 3D Maps: Accuracy and Reliability Analysis

In an effort to supplement the available satelite-based positioning technology and extend such high level positioning capability to GPS-denied environments, a method of vision-based positioning with the use of single camera and newly defined 3D maps is proposed. Besides, only natural landmarks are required in the proposed method. Absolute position and orientation information can be provided in six degree of freedom. Our work here is to address the accuracy and reliability concerns of such a vision-based navigation system. The main contribution will be the newly defined 3D map and the adoption of photogrammetric 6DOF pose estimation method to improve positioning accuracy. Dilution of Precisions (DOPs) are introduced to evaluate positioning precision within the vision-based positioning domain. Quality control strategies are also applied to detect outliers in the observation and strengthen system reliability

A probabilistic framework for object search with 6-DOF pose estimation

This article presents a systematic approach to the problem of autonomous 3D object search in indoor environments, using a two-wheeled non-holonomic robot equipped with an actuated stereo-camera head and processing done on a single laptop. A probabilistic grid-based map encodes the likelihood of object existence in each cell and is updated after each sensing action. The updating schema incorporates characteristic parameters modeled after the robot’s sensing modalities and allows for sequential updating via Bayesian recursion methods. Two types of sensing modalities are used to update the map: a coarse search method (global search) based on a color histogram approach, and a more refined search method (local search) based on Scale-Invariant Feature Transform (SIFT) feature matching. If the local search correctly locates the desired object, its 6-DOF pose is estimated using stereo applied to each SIFT feature (i.e. 3D SIFT feature), which is then fed as measurements into an Extended Kalman Filter (EKF) for sustained tracking. If the local search fails to locate the desired object in a particular cell, the cell is updated in the probability map and the next peak probability cell is identified and planned to using a separate grid-based costmap populated via obstacle detection from stereo, with planning done using an A* planner. Experimental results obtained from the use of this method on a mobile robot are presented to illustrate and validate the approach, confirming that the search strategy can be carried out with modest computation on a single laptop.

SLAM for an AUV using vision and an acoustic beacon

The estimation of the position and attitude of an autonomous underwater vehicle (AUV) is a challenging and important problem in marine robotics. It is well known that the underwater environment posses considerable problems, that include i) the fact that there is no GPS signal, ii) the communication is usually done through acoustic signals, which suffers from faults, delays and low bandwidth, and iii) the use of vision and/or laser is very limited due to poor visibility. In this paper, we combine a multiple set of sensors to address the full state 6DOF pose estimation of an AUV. The problem is formulated assuming that we have partial measurements from an Inertial Measurement Unit (IMU), an acoustic ranging from a single beacon buoy, and a monocular camera attached to the AUV. Using multiple model estimation techniques and the concept of Extended Kalman Filters with Simultaneous Localization and Mapping (EKF-SLAM), we propose an algorithm that integrates the AUV measurements (that arrive at different sampling-times) and compute in real time an estimate of the position and attitude of the AUV. Simulation results are presented and discussed.

Precision potential of photogrammetric 6DOF pose estimation with a single camera

The article discusses the theoretical precision for the measurement of six degrees of freedom (6DOF = position and orientation in 3D space) of an object with respect to a reference system using one camera only. Single camera solutions are of interest in applications where the application is restricted to only one camera, e.g. due restrictions in terms of costs, synchronisation demands, or spatial observation conditions. A pure photogrammetric approach is discussed in which two sequential space resections generate the required six transformation values. The paper describes the mathematical model and precision evaluations based on mathematical simulations, and some practical examples as well. With suitable configurations of camera position, object size and reference system, accuracies of better than 1:10 000 of the measuring volume can be achieved whereby translations in Z and rotations around ω and φ are the most critical to measure.