6DoF Pose Estimation of Transparent Object from a Single RGB-D Image.

Chi Xu,Jun Zhou,Lijun Zhang,Jiale Chen,Mengyang Yao,Yi Liu

doi:10.3390/s20236790

Abstract

6DoF object pose estimation is a foundation for many important applications, such as robotic grasping, automatic driving, and so on. However, it is very challenging to estimate 6DoF pose of transparent object which is commonly seen in our daily life, because the optical characteristics of transparent material lead to significant depth error which results in false estimation. To solve this problem, a two-stage approach is proposed to estimate 6DoF pose of transparent object from a single RGB-D image. In the first stage, the influence of the depth error is eliminated by transparent segmentation, surface normal recovering, and RANSAC plane estimation. In the second stage, an extended point-cloud representation is presented to accurately and efficiently estimate object pose. As far as we know, it is the first deep learning based approach which focuses on 6DoF pose estimation of transparent objects from a single RGB-D image. Experimental results show that the proposed approach can effectively estimate 6DoF pose of transparent object, and it out-performs the state-of-the-art baselines by a large margin.

Highlights

IntroductionTransparent objects (e.g., glasses, plastic bottles, bowls, etc.) are commonly seen in our daily environments, such as kitchen, office, living room, canteen, and so on
Transparent objects are commonly seen in our daily environments, such as kitchen, office, living room, canteen, and so on
Different from Sajjan et al [12], we focus on 6DoF pose estimation, while ClearGrasp aims at depth reconstruction

Summary

Introduction

Transparent objects (e.g., glasses, plastic bottles, bowls, etc.) are commonly seen in our daily environments, such as kitchen, office, living room, canteen, and so on. Existing pose estimation methods for ordinary objects cannot deal with transparent ones correctly (please refer to Figure 1), because the optical characteristics of transparent material lead to significant depth error [12] in D-channel of RGB-D image DenseFusion [2] is one of the state-of-the-art methods for 6DoF object pose estimation from a single RGB-D image. We propose an accurate and efficient approach for 6DoF pose estimation of transparent object from a single RGB-D image. The proposed approach contains two stages: In the first stage, we eliminate the influence of depth errors by transparent segmentation, surface normal recovering, and RANSAC plane estimation. The extended point-cloud and the color feature extracted are fed into a DenseFusion-like network structure [2] for 6DoF pose estimation

Objectives

Methods

Results

Conclusion