Evaluation of visual descriptors for painting categorisation

Abstract

The ever increasing availability of digital data from the Arts and cultural heritage calls for efficient methods to organise, categorise, and retrieve such information in an effective and reliable way. In this context, painting classification has attracted significant research interest in recent years. In this work we address the problem of style classification, which involves determining the school, period and art movement to which a painting belongs. Notably, this job is peculiarly different from other machine vision applications – such as material, object and scene recognition – since the concept of ‘similarity’ is much more difficult to define in this case. For this specific task we evaluate, in this study, the effectiveness of an array of hand-designed visual descriptors against a set of feature extractors based on last-generation convolutional neural networks. We also investigate the effect of pre-processing methods such as image split and pyramidal decomposition. The experiments are based on the open-access Pandora dataset. The results show that pre-trained models can significantly outperform hand-designed descriptors with overall accuracy surpassing 67%. This represents an improvement on the state-of-the-art by ≈12 percentage points.