Deep learning visual interpretation of structural damage images

Abstract

In the past decade, Deep Convolutional Neural Network (DCNN) achieved the state-of-the-art performance in computer vision tasks. However, DCNN is usually treated as a “black box”, whose internal working principle is hard to understand. This drawback significantly limits its usage in real-world applications, e.g., vision-based Structural Health Monitoring (SHM), where wrong predictions may lead to catastrophic consequences. To resolve this problem, a framework for the interpretation of the Deep Learning (DL) results called Structural Image Guided Map Analysis Box (SIGMA-Box or Σ-Box) is proposed. In the Σ-Box, visual interpretation results (saliency maps) are produced and used for model quality evaluation along with human experts’ domain knowledge. In this study, the use of the Σ-Box is explored in vision-based SHM applications. Firstly, understanding trained DCNN’s performance in concrete cover spalling detection is investigated. Besides, learning procedure at different epochs, learned feature from different network depths, influence of training techniques, and level of semantic abstraction are studied. The experiments demonstrate the good interpretable performance of the Σ-Box which facilitates the understanding of the DCNN models’ recognition capabilities, preferences, and limitations. In conclusion, this study sheds light on the high potential of interpreting the trained DCNN in vision-based SHM, providing confidence to the engineers for practical engineering applications involving DL.