TGF-Net: Sim2Real Transparent Object 6D Pose Estimation Based on Geometric Fusion

Abstract

Transparent objects are a common part of daily life, but their unique optical properties make estimating their 6D pose a challenging task. In this letter, we propose TGF-Net, a monocular instance-level 6D pose estimation method for transparent objects based on geometric fusion. TGF-Net learns the edge features and surface fragments of transparent objects as intermediate features and reduces the influence of appearance changes on the 6D pose estimation of transparent objects by fusing rich geometric features in the network. Moreover, we propose an approach for generating high-fidelity large-scale synthetic datasets of transparent objects using Blender and use this approach to generate a synthetic dataset Trans6D-32K. Trans6D-32K contains rendered RGB images and poses information about transparent objects in a variety of different backgrounds, perspectives, and lighting conditions. To evaluate the performance of TGF-Net on 6D pose estimation of transparent objects, we compare with multiple related works on the dataset Trans6D-32K. TGF-Net can be trained entirely on synthetic datasets without fine-tuning and applied directly to real-world scenarios. Multiple challenging real-scene experiments demonstrate the good performance of TGF-Net, while grasping experiments demonstrate the application value of TGF-Net in transparent object manipulation.