Generating real-world-like labelled synthetic datasets for construction site applications

Abstract

Having synthetic image generation and automatic labelling as two separate processes remains one of the main limitations of automatic generation of large real-world synthetic datasets. To overcome this drawback, a methodology to perform both tasks in a simultaneous and automatic manner is proposed. To resemble real-world scenarios, a diverse set of rendering configurations of illumination, locations, and sizes are presented. For testing, three synthetic datasets (S, M and SM), oriented to the construction field, were generated. Faster R-CNN, RetinaNet, and YoloV4 detection algorithms were used to independently evaluate the datasets using the COCO evaluation metrics and the PascalVOC Mean Average Accuracy metric. Results show that, in general, the S dataset performed 1.2% better in the evaluation metrics and that the SM dataset obtained better plots of training and validation loss curves in each detector; highlighting the combinational usage of images with single and multiple objects as a better generalisation approach.