An Artificial Intelligence System for Automatic Recognition of Punches in Fourteenth-Century Panel Painting

Abstract

In Late-Medieval panel paintings from the Tuscan area, mechanical tools called punches were used to impress repeated motifs on gold foils to create decorative patterns. Such patterns can be used as clues to objectively support the attribution of the paintings, as proposed by art historian Erling S. Skaug in his decades-long study on punches. We investigate the feasibility of employing automatic pattern recognition techniques for accelerating the process of classification of punches by experts working in the field. We propose a system composed of (a) a Convolutional Neural Network for categorizing a punch contained in a frame, and (b) an additional component for uncertainty estimation, aimed at recognizing possible Out-of-Distribution (OOD) samples. After collecting a set of 14th century panel paintings from Tuscany, we train a Convolutional Neural Network which achieves very high test-set accuracy. As far as the uncertainty estimation is concerned, we experiment with two techniques, OpenGAN and II-loss, both exhibiting very positive results. The former seems to work better on specific data extracted from images of panel paintings, while the latter showcases a more consistent behavior when considering additional OOD data obtained randomly. These outcomes indicate that an application of our system in support of experts is feasible, although we subsequently show that additional experiments on larger datasets might be required.