Simulation-based decision support system for earthmoving operations using computer vision

Abstract

A reliable Decision Support System (DSS), particularly in the construction domain, can be driven by quality input information. Although vision-based methods have been widely utilized to retrieve contextual information, their potential is not fully leveraged in construction simulation yet. This study introduces an automated framework that utilizes multi-view video footage for vision-based input modeling within simulation domains. The proposed framework addresses project uncertainties (e.g., equipment performance, operators’ skills, road network, and weather status) using a proactive approach where project task durations are modeled as probabilistic distributions. The modeled distributions are continuously calibrated using the Markov Chain Monte Carlo Bayesian Inference (MCMCBI) approach. A simulation-based Simulated Annealing (SA) optimization is also employed to provide an efficient resource assignment. The extracted vision-based data is validated statistically against actual and spatiotemporal data. The results demonstrate that the suggested vision-based approach can provide qualified DSS input. Statistical analysis also confirms that vision-based data is more consistent with actual data than spatiotemporal data. The presented approach is successfully applied to an actual case study of a large-scale earthmoving project.