Real‐time Robust Six Degrees of Freedom Object Pose Estimation with a Time‐of‐flight Camera and a Color Camera

Abstract

We present an algorithm for estimating the six degrees of freedom (6DOF) pose for a rigid object of arbitrary shape, which can move fast in cluttered environments. The RGBD data input is obtained by fusing a time‐of‐flight (TOF) camera and a color camera. The proposed approach is composed of a coarse estimation stage for prealignment and an accurate estimation stage for refining the coarse pose. The most important contribution is the textured iterative closest point (ICP) in the accurate stage, where the Lukas‐Kanade method is incorporated into the point‐to‐plane ICP framework, by which geometrically symmetric objects can be handled. In addition, the pose estimation performance under motion artifacts that are common for a TOF sensor can be significantly improved. Another important contribution is the tailored sparse representation under an annealed particle filtering framework for effectively extracting the target from the background clutter and providing a coarse 6DOF pose. The entire algorithm is implemented with graphics processing unit acceleration and shows real‐time performance. The approach is verified on a variety of targets in both indoor and outdoor scenarios.