Deep learning-based automated tile defect detection system for Portuguese cultural heritage buildings

Abstract

A prominent feature in Portuguese historic architecture is Portugal’s azulejos or tiles that cover cultural heritage buildings with colorful patterns. However, tiles are prone to deterioration due to the quality of masonry materials, exposure over time, and natural and human factors. A careful approach is necessary to detect and assess tile damage in time to conserve cultural heritage. Deep learning (DL) methods are applied to detect deterioration and damage by automating vision-based monitoring. This study uses the You Only Look Once (YOLO), method to detect deterioration in tiles automatically. To obtain the initial dataset, over 5000 images of damage were collected, including cracks, craters, glaze detachment, and tile lacunae, as well as images with no defects. Additionally, a MobileNet model was used for binary classification of damaged and intact tiles to compare classification and detection approaches. Through the fine-tuning of hyperparameters and updating the dataset, an overall accuracy of over 72% for YOLO (multiple classification) and 97% accuracy for binary classification was achieved, demonstrating the adequacy of the tool for real-world applications.