Visual blockage assessment at culverts using synthetic images to mitigate blockage-originated floods

Abstract

Abstract The assessment of visual blockages in cross-drainage hydraulic structures, such as culverts and bridges, is crucial for ensuring their efficient functioning and preventing flash flooding incidents. The extraction of blockage-related information through computer vision algorithms can provide valuable insights into the visual blockage. However, the absence of comprehensive datasets has posed a significant challenge in effectively training computer vision models. In this study, we explore the use of synthetic data, the synthetic images of culvert (SIC) and the visual hydraulics lab dataset (VHD), in combination with a limited real-world dataset, the images of culvert openings and blockage (ICOB), to evaluate the performance of a culvert opening detector. The Faster Region-based Convolutional Neural Network (Faster R-CNN) model with a ResNet50 backbone was used as the culvert opening detector. The impact of synthetic data was evaluated through two experiments. The first involved training the model with different combinations of synthetic and real-world data, while the second involved training the model with reduced real-world images. The results of the first experiment revealed that structured training, where the synthetic images of culvert (SIC) were used for initial training and the ICOB was used for fine-tuning, resulted in slightly improved detection performance. The second experiment showed that the use of synthetic data, in conjunction with a reduced number of real-world images, resulted in significantly improved degradation rates.