Introduction of Deep Learning in Thermographic Monitoring of Cultural Heritage and Improvement by Automatic Thermogram Pre-Processing Algorithms.

Iván Garrido,Stefano Sfarra,Susana Lagüela,Gianfranco Gargiulo,Pedro Arias,Elena Pivarčiová,Jorge Erazo-Aux,Xavier Maldague,Clemente Ibarra-Castanedo

doi:10.3390/s21030750

Abstract

The monitoring of heritage objects is necessary due to their continuous deterioration over time. Therefore, the joint use of the most up-to-date inspection techniques with the most innovative data processing algorithms plays an important role to apply the required prevention and conservation tasks in each case study. InfraRed Thermography (IRT) is one of the most used Non-Destructive Testing (NDT) techniques in the cultural heritage field due to its advantages in the analysis of delicate objects (i.e., undisturbed, non-contact and fast inspection of large surfaces) and its continuous evolution in both the acquisition and the processing of the data acquired. Despite the good qualitative and quantitative results obtained so far, the lack of automation in the IRT data interpretation predominates, with few automatic analyses that are limited to specific conditions and the technology of the thermographic camera. Deep Learning (DL) is a data processor with a versatile solution for highly automated analysis. Then, this paper introduces the latest state-of-the-art DL model for instance segmentation, Mask Region-Convolution Neural Network (Mask R-CNN), for the automatic detection and segmentation of the position and area of different surface and subsurface defects, respectively, in two different artistic objects belonging to the same family: Marquetry. For that, active IRT experiments are applied to each marquetry. The thermal image sequences acquired are used as input dataset in the Mask R-CNN learning process. Previously, two automatic thermal image pre-processing algorithms based on thermal fundamentals are applied to the acquired data in order to improve the contrast between defective and sound areas. Good detection and segmentation results are obtained regarding state-of-the-art IRT data processing algorithms, which experience difficulty in identifying the deepest defects in the tests. In addition, the performance of the Mask R-CNN is improved by the prior application of the proposed pre-processing algorithms.

Highlights

IntroductionIntroduction of Deep Learning in ThermographicMonitoring of Cultural Heritage and Improvement by Automatic ThermogramPre-Processing AlgorithmsIván Garrido 1 , Jorge Erazo-Aux 2,3 , Susana Lagüela 4 , Stefano Sfarra 5, * , Clemente Ibarra-Castanedo 6 , Elena Pivarčiová 7 , Gianfranco Gargiulo 8 , Xavier Maldague 6 and Pedro Arias 1Escuela de Ingeniería Eléctrica y Electrónica, Universidad del Valle, Cali 760032, VA, Colombia; Facultad de Ingeniería, Institución Universitaria Antonio José Camacho, Cali 760046, VA, Colombia
Introduction of Deep Learning in ThermographicMonitoring of Cultural Heritage and Improvement by Automatic ThermogramPre-Processing AlgorithmsIván Garrido 1, Jorge Erazo-Aux 2,3, Susana Lagüela 4, Stefano Sfarra 5, *, Clemente Ibarra-Castanedo 6, Elena Pivarčiová 7, Gianfranco Gargiulo 8, Xavier Maldague 6 and Pedro Arias 1Citation: Garrido, I.; Erazo-Aux, J.; Lagüela, S.; Sfarra, S.; Ibarra-Castanedo, *Escuela de Ingeniería Eléctrica y Electrónica, Universidad del Valle, Cali 760032, VA, Colombia; Facultad de Ingeniería, Institución Universitaria Antonio José Camacho, Cali 760046, VA, ColombiaDepartment of Cartographic and Terrain Engineering, University of Salamanca, Calle Hornos Caleros, 50, Department of Industrial and Information Engineering and Economics (DIIIE), University of L’Aquila, Piazzale E
As for the monitoring, two different experiments have been applied to each marquetry, one heating the marqueteries with pulsed heat and the other heating the marqueteries with waved heat

Summary

Introduction

Introduction of Deep Learning in ThermographicMonitoring of Cultural Heritage and Improvement by Automatic ThermogramPre-Processing AlgorithmsIván Garrido 1 , Jorge Erazo-Aux 2,3 , Susana Lagüela 4 , Stefano Sfarra 5, * , Clemente Ibarra-Castanedo 6 , Elena Pivarčiová 7 , Gianfranco Gargiulo 8 , Xavier Maldague 6 and Pedro Arias 1Escuela de Ingeniería Eléctrica y Electrónica, Universidad del Valle, Cali 760032, VA, Colombia; Facultad de Ingeniería, Institución Universitaria Antonio José Camacho, Cali 760046, VA, Colombia. Preventive and conservation interventions in cultural heritage are essential tasks to protect objects of incalculable human value. The use of the most up-to-date technologies and the corresponding most advanced data processing algorithms is necessary to apply the required preventive and conservation tasks in each specific case under analysis, facing the inevitable object deterioration due to the passage of time. Inspection technologies must be able to avoid producing new defects in the whole 3D structure, and (their algorithms) to identify defects from their initial growth phase, in order to implement prevention tasks. Both technologies and algorithms should be able to identify the most damaged parts in order to implement conservation tasks. In case of a late, wrong, or absent intervention, the damage can be irreversible by leading to an anticipated degradation of the artistic object

Methods

Results

Conclusion