Analyzing cultural relationships visual cues through deep learning models in a cross-dataset setting

Abstract

AbstractTo study the evolution of specific cultures and times different kinds of pictures could be adopted. Family album photos may reveal socio-historical insights regarding those specific cultures and times. Along this path, this work addresses the problem of automatically dating an image by resorting to the analysis of an analog family album photo dataset. In particular, the IMAGO collection, which contains Italian photos shot in the 20th century, was considered. Thanks to the IMAGO dataset, it was possible to apply different deep learning-based architectures to date images belonging to photo albums without needing any other sources of information. In addition, we carried out cross-dataset experiments, which also involved models trained on American datasets, observing temporal shifts which may be due to known intercultural influences. We further explore such a possibility by qualitatively analyzing the cross-dataset interpretation of the trained deep-learning models with the Uniform Manifold Approximation and Projection (UMAP) algorithm. In conclusion, deep learning models revealed their potential in terms of possible applications to intercultural research, from different points of view.