Semi-Automated Visual Quality Control Inspection During Construction or Renovation of Railways Using Deep Learning Techniques and Augmented Reality Visualization

Abstract

The construction industry stands to greatly benefit from the technological advancements in deep learning and computer vision, which can automate time-consuming tasks such as quality control. In this paper, we introduce a framework that incorporates two advanced tools - the Visual Quality Control (VQC) tool and the Digital Twin visualization with Augmented Reality (DigiTAR) tool - to perform semi-automated visual quality control in the construction site during the execution phase of the project. The VQC tool is a backend service that detects potential defects on images captured on-site using the Mask R-CNN algorithm trained on annotated images of concrete and railway defects. The surveyor, aided by the Augmented Reality (AR) technology through the DigiTAR tool, can in-situ confirm/reject the detected defects and propose remedial actions. All the quality control results are recorded in the relevant BIM model and can be viewed on-site overlaid on the physical construction elements. This solution offers a semi-automated visual inspection that can speed up and simplify the quality control process, especially in case of large linear infrastructures, illustrating the added value of AR-based applications in Digital Twins