Generating Images with Physics-Based Rendering for an Industrial Object Detection Task: Realism versus Domain Randomization.

Leon Eversberg,Jens Lambrecht

doi:10.3390/s21237901

Abstract

Limited training data is one of the biggest challenges in the industrial application of deep learning. Generating synthetic training images is a promising solution in computer vision; however, minimizing the domain gap between synthetic and real-world images remains a problem. Therefore, based on a real-world application, we explored the generation of images with physics-based rendering for an industrial object detection task. Setting up the render engine’s environment requires a lot of choices and parameters. One fundamental question is whether to apply the concept of domain randomization or use domain knowledge to try and achieve photorealism. To answer this question, we compared different strategies for setting up lighting, background, object texture, additional foreground objects and bounding box computation in a data-centric approach. We compared the resulting average precision from generated images with different levels of realism and variability. In conclusion, we found that domain randomization is a viable strategy for the detection of industrial objects. However, domain knowledge can be used for object-related aspects to improve detection performance. Based on our results, we provide guidelines and an open-source tool for the generation of synthetic images for new industrial applications.

Highlights

Synthetic data are one of the most promising areas of research in modern deep learning as this approach tries to solve the problem of insufficient training data [1]
We want to detect a texture-less turbine blade at a manual working station in a shopfloor. With this exemplary use case we study the domain gap between synthetic images based on a 3D model and real-world images from a camera for an industrial object detection task
In this work we presented an image generation pipeline based on Physically based rendering (PBR) that can generate synthetic training images for deep learning object detection models

Summary

Introduction

Synthetic data are one of the most promising areas of research in modern deep learning as this approach tries to solve the problem of insufficient training data [1]. For industrial applications, synthetic images can be generated based on already available 3D CAD models. For those reasons, synthetic images offer a solution to the problem of limited labeled data in industrial deep learning. Application domain images or random 3D models. We believe this is due to the position and orientation of the virtual camera, which is often looking towards the indoor ceiling or floor of the scene. Random material textures performed only slightly worse; domain randomization of the object texture seems to be a viable alternative if no appropriate material textures are available. We believe this is due to the fact that the YCB tools already have complex textures, there is no benefit in randomizing them

Objectives

Methods

Results

Discussion

Conclusion