Abstract
<p class="first" id="d1036043e76">This paper contributes to the discussion about the opportunities and challenges of applying computer vision and machine learning to archival image collections of significant cultural heritage value. We explore these questions from an institutional perspective. Our case study is a pilot project developed at Dumbarton Oaks, a research institute and library, museum, and historic garden affiliated with Harvard University and located in Washington, DC. The project focused on a collection of 10,000 images of Syrian monuments in the institution’s Image Collections and Fieldwork Archives (ICFA). Drawing on that project, as well as the broader landscape of AI-based categorisation efforts in the fields of art and architecture, we will share our insights on the potential of AI to facilitate and enhance archival image access and recording. Many of the Syrian sites in the Dumbarton Oaks collection have been inaccessible to researchers and the public for over a decade and/or have been damaged or destroyed. The pilot project, undertaken in 2019-2020, was a collaboration between Dumbarton Oaks; a commercial tech partner, ArthurAI Inc.; and a computer science research team from the University of Maryland. For Dumbarton Oaks, the primary goal was to explore whether AI can improve the speed and efficiency of sharing collections and allow for more sophisticated curation by subject experts who, thanks to automation, would be relieved of the burden of rote processing. For the technology partners, the experimental value of the project lay in the availability of a collection that could be shared open access (no privacy or copyright issues) and was focused enough to yield a domain-specific training set. The methods and techniques explored included multi-label classification, multi-task classification, unsupervised image clustering, and explainability.
Highlights
Artificial Intelligence could prove of enormous value for archival image collections like the ones held at the Dumbarton Oaks research institute in Washington, DC
Extensive efforts have been directed toward the creation of digital resources for cultural heritage management, ranging from digitisation initiatives to the development of sophisticated, first-line response actions, such as the extraction of pertinent information from social media postings
Deep learning techniques (LeCun, Bengio & Hinton 2015), and especially convolutional neural networks (CNNs), have been applied to images of cultural heritage value to perform classification and annotation tasks that are critical to documentation (Llamas et al 2017), as well as to facilitate public access to this content
Summary
Deep learning techniques (LeCun, Bengio & Hinton 2015), and especially convolutional neural networks (CNNs), have been applied to images of cultural heritage value to perform classification and annotation tasks that are critical to documentation (Llamas et al 2017), as well as to facilitate public access to this content. The Metropolitan Museum of Art has announced a recent partnership with Microsoft and the Massachusetts Institute of Technology to pursue a number of AI-based use cases These aim to enrich user experiences of the Met’s Open Access artworks through the detection of hidden patterns, as well as broaden access to the museum collections through automation of the tagging process and associated cost efficiencies (Choi 2019). There is an increasing number of individual technical studies involving the use of deep learning in the processing of images of cultural heritage value, the adoption of such techniques by institutional holders is in its nascent stage. We focus on the challenges and opportunities of the application of such mature and well-known computer vision and machine learning applications in the context of a research library, to streamline the cataloguing of vast archival collections and render them more quickly and openly accessible
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Free) 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
