Towards Learning 3d Object Detection and 6d Pose Estimation from Synthetic Data

Abstract

Deep Learning-based approaches for 3d object detection and 6d pose estimation typically require large amounts of labeled training data. Labeling image data is expensive and particularly the 6d pose information is difficult to obtain, as it requires a complex setup during image acquisition. Training with synthetic data is therefore very attractive. Large amounts of synthetic, labeled data can be generated, but it is not yet fully understood how certain aspects of data generation affect the detection and pose estimation performance. Our work therefore focuses on creating synthetic training data and investigating the effects on detection performance. We present two methods for data generation: rendering object views and pasting them on random background images, and simulating realistic scenes. The former is computationally simpler and achieved better results, but the detection performance is still very sensitive to small changes, e.g. the type of background images.