Abstract
Abstract. In the past few decades, a number of scholars studied painting classification based on image processing or computer vision technologies. Further, as the machine learning technology rapidly developed, painting classification using machine learning has been carried out. However, due to the lack of information about brushstrokes in the photograph, typical models cannot use more precise information of the painters painting style. We hypothesized that the visualized depth information of brushstroke is effective to improve the accuracy of the machine learning model for painting classification. This study proposes a new data utilization approach in machine learning with Reflectance Transformation Imaging (RTI) images, which maximizes the visualization of a three-dimensional shape of brushstrokes. Certain artist’s unique brushstrokes can be revealed in RTI images, which are difficult to obtain with regular photographs. If these new types of images are applied as data to train in with the machine learning model, classification would be conducted including not only the shape of the color but also the depth information. We used the Convolution Neural Network (CNN), a model optimized for image classification, using the VGG-16, ResNet-50, and DenseNet-121 architectures. We conducted a two-stage experiment using the works of two Korean artists. In the first experiment, we obtained a key part of the painting from RTI data and photographic data. In the second experiment on the second artists work, a larger quantity of data are acquired, and the whole part of the artwork was captured. The result showed that RTI-trained model brought higher accuracy than Non-RTI trained model. In this paper, we propose a method which uses machine learning and RTI technology to analyze and classify paintings more precisely to verify our hypothesis.
Highlights
In recent years, as the art database expands rapidly, automatic painting classification based on color and morphological features have been gaining much attention (Berns, 2001) (Barni et al, 2005)
Painting classification studies are proceeding from the classical Support Vector Machine (SVM) method to Convolution Neural Network (CNN), which is optimized for image learning (Cortes, Vapnik, 1995) (Krizhevsky et al, 2012)
In the case of group A, VGG-16 with Non-Reflectance Transformation Imaging (RTI) dataset, group B, DenseNet-121 with RTI dataset, and group C’, ResNet-50 with RTI dataset showed the best performance. This result suggests that there is a need for future research to develop more suitable machine learning architectures and optimizers for utilizing RTI data
Summary
As the art database expands rapidly, automatic painting classification based on color and morphological features have been gaining much attention (Berns, 2001) (Barni et al, 2005). In order to classify paintings, the Image processing technique has been studied to extract the characteristics of paintings such as the shape, directions, and the pattern of brushstrokes (Li et al, 2011). Painting classification studies are proceeding from the classical Support Vector Machine (SVM) method to Convolution Neural Network (CNN), which is optimized for image learning (Cortes, Vapnik, 1995) (Krizhevsky et al, 2012). Artists’ brushstroke is one of the characteristics which reflects their unique painting styles (Li et al, 2011) It contains a combination of color, pattern, and texture, and the collaboration of each characteristic element shows much information in oil paintings with pigments (Berezhnoy et al, 2009) (Johnson et al, 2008).
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callMore From: The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences
Disclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
