The impact of domain randomization on cross-device monocular deep 6DoF detection

Abstract

• We propose an evaluation of synthetic datas influence when training deep learning models to handle unseen scenarios. • We provide an analysis of how the models behave when faced with different environmental challenges and cameras. • We describe use cases for 3D printed objects and how the method facilitates the development of real-world applications. • We introduce the eC3Po dataset with over 110,000 annotated frames (real and synthetic), that will be made publicly available. This work evaluates the use of synthetic data to train deep 6DoF pose estimation models that use a monocular RGB camera as input. We have compared different training strategies combining real and synthetic data (with domain randomization) to investigate how to better handle real-world challenges. We show that it is possible to obtain accurate models using less real data and suggest how to utilize this strategy. In this work, we have captured and made available two datasets: one real and one synthetic, totaling over 110,000 annotated frames. These datasets are organized according to the different cameras used and the challenges present in the sequences, all featuring textureless 3D printed objects. We also show that synthetic data can help models generalize, handling challenges such as fast motion, occlusion, illumination changes, color variation, scale changes, and unexpected geometry. Finally, we evaluated 70 different models to understand how a model trained for one camera sensor performs when used with a different sensor. To this end, we also suggest how to handle this challenge better by using synthetic simulations to supplement training.