Image generation of hazardous situations in construction sites using text-to-image generative model for training deep neural networks

Abstract

There has been a persistent challenge in acquiring sufficient training image data for deep neural networks (DNNs) to enhance safety monitoring on construction sites. Given the prevalence of textual data in the construction sector and the capabilities of multi-modal AI systems, this paper presents the use of text-to-image models to generate training images that capture the relationships between objects involved in construction accidents. Through a systematic prompt design process, a synthetic dataset of 3585 images across 27 hazardous scenarios was generated. The efficacy of this method is demonstrated by the performance of DNN models trained on these virtual images, which achieved a mean Average Precision (mAP) of approximately 64% in object detection and 60% in segmentation tasks. This paper demonstrates the potential of text-to-image models in mitigating the scarcity of training data and enhancing the capability of DNNs to identify potential hazards.