Automatic segmentation of TBM muck images via a deep-learning approach to estimate the size and shape of rock chips

Abstract

Real-time muck analysis is of great importance for assisting tunnel boring machines (TBMs) in intelligent tunneling. Typically, muck images are characterized by low contrast, large appearance differences, and object overlap, posing a great challenge to image segmentation. In this study, a deep learning-based approach, composed of a dual UNet with multi-scale inputs and side-output (MSD-UNet) and a post-processing algorithm, was proposed to solve the automatic segmentation of muck images and estimate the size and shape of rock chips. The MSD-UNet used a dual structure with two decoders to segment the regions and boundaries of rock chips in a unified network, aiming to solve the overlapping problem of rock chips by introducing the boundary information. It also integrated a multi-scale input and side outputs to enhance low-level image features and supervise the training of early layers of the network, respectively. An integrated loss function based on generalized dice loss was developed to solve the class imbalance problem. The multi-radius erosion and seed filling algorithms were employed to further separate the connected chips in the post-processing. To evaluate the effectiveness of the method, a dataset containing various muck images collected from a TBM construction site was set up, and the MSD-UNet was trained and tested. Experimental results showed that the segmentation using the proposed approach outperformed those of using U-Net and comparable conventional methods. It achieved the highest F1-score of 0.867 and 0.640 on the region and boundary task respectively, and an average Hausdorff distance of 3.59 mm for the rock chip instance. The proposed approach can process a 2,048 × 2,048 image in about 4 s and can nearly meet the requirement of real-time TBM muck image analysis.