Cultural Heritage Domain Research Articles

CLEF. A Linked Open Data Native System for Crowdsourcing

Collaborative data collection initiatives are increasingly becoming pivotal to cultural institutions and scholars, to boost the population of born-digital archives. For over a decade, organisations have been leveraging Semantic Web technologies to design their workflows, ensure data quality, and a means for sharing and reusing (Linked Data). Crucially, scholarly projects that leverage cultural heritage data to collaboratively develop new resources would benefit from agile solutions to simplify the Linked Data production workflow via user-friendly interfaces. To date, only a few pioneers have abandoned legacy cataloguing and archiving systems to fully embrace the Linked Open Data (LOD) paradigm and manage their catalogues or research products through LOD-native management systems. In this article we present Crowdsourcing Linked Entities via web Form (CLEF) , an agile LOD-native platform for collaborative data collection, peer-review, and publication. We detail design choices as motivated by two case studies, from the Cultural Heritage and scholarly domains respectively, and we discuss benefits of our solution in the light of prior works. In particular, the strong focus on user-friendly interfaces for producing FAIR data, the provenance-aware editorial process, and the integration with consolidated data management workflows, distinguish CLEF as a novel attempt to develop Linked Data platforms for cultural heritage.

TTProfiler: Types and Terms Profile Building for Online Cultural Heritage Knowledge Graphs

As more and more knowledge graphs (KGs) are published on the Web, there is a need for tools that show their content. This implies showing the schema-level patterns instantiated in the graph, but also the terms used to qualify its entities. In this article, we present a new profiling tool that we call TTprofiler. It shows the predicates that relate types in the KG, and also the terms present in this KG, because of their paramount importance in most KGs, especially in the Cultural Heritage (CH) domain. We recall the role of terminologies and how they are implemented and used on the Web, we give the algorithm for building a TT profile from an online KG’s endpoint, and we report on experiments performed over a set of Cultural Heritage Web KGs. A tool for visualizing TT profiles is also provided.

Around the GLOBE: Numerical Aggregation Question-answering on Heterogeneous Genealogical Knowledge Graphs with Deep Neural Networks

One of the key AI tools for textual corpora exploration is natural language question-answering (QA). Unlike keyword-based search engines, QA algorithms receive and process natural language questions and produce precise answers to these questions, rather than long lists of documents that need to be manually scanned by the users. State-of-the-art QA algorithms based on DNNs were successfully employed in various domains. However, QA in the genealogical domain is still underexplored, and researchers in this field (and other fields in humanities and social sciences) can highly benefit from the ability to ask questions in natural language, receive concrete answers, and gain insights hidden within large corpora. While some research has been recently conducted for factual QA in the genealogical domain, to the best of our knowledge, there is no previous research on the more challenging task of numerical aggregation QA (i.e., answering questions combining aggregation functions, e.g., count, average, max). Numerical aggregation QA is critical for distant reading and analysis for researchers (and the general public) interested in investigating cultural heritage domains. Therefore, in this study, we present a new end-to-end methodology for numerical aggregation QA for genealogical trees that includes (1) an automatic method for training dataset generation, (2) a transformer-based table selection method, and (3) an optimized transformer-based numerical aggregation QA model. The findings indicate that the proposed architecture, GLOBE, outperforms the state-of-the-art models and pipelines by achieving 87% accuracy for this task compared to only 21% by current state-of-the-art models. This study may have practical implications for genealogical information centers and museums, making genealogical data research easy and scalable for experts as well as the general public.

PLANTS’ PHYTOCHEMICALS – A CRITICAL DISCUSSION ON THEIR POTENTIAL TO PROTECT CULTURAL HERITAGE OBJECTS

Since ancient times, plants were used for different purposes: even we talk about medicine, construction materials, food or feed. In modern times, plants are a huge factory of compounds which can replace synthesis and hazardous compounds, being appreciated like raw materials in different fields. In the field of cultural heritage protection there is an urgent need to develop innovative, eco-sustainable, and safe solutions, so, plants’ constituents can be an excellent choice. In this context, our short review intends to present a critical opinion on aspects regarding obtaining methods of plants extract, some aspects regarding chemical active compounds, and their application in the cultural heritage domain, being a starting step for a future trans disciplinary study, which offers links between multy disciplinary sciences: chemistry, biology, restoration and conservation.

VISCOUNTH: A Large-scale Multilingual Visual Question Answering Dataset for Cultural Heritage

Visual question answering has recently been settled as a fundamental multi-modal reasoning task of artificial intelligence that allows users to get information about visual content by asking questions in natural language. In the cultural heritage domain, this task can contribute to assisting visitors in museums and cultural sites, thus increasing engagement. However, the development of visual question answering models for cultural heritage is prevented by the lack of suitable large-scale datasets. To meet this demand, we built a large-scale heterogeneous and multilingual (Italian and English) dataset for cultural heritage that comprises approximately 500K Italian cultural assets and 6.5M question-answer pairs. We propose a novel formulation of the task that requires reasoning over both the visual content and an associated natural language description, and present baselines for this task. Results show that the current state of the art is reasonably effective but still far from satisfactory; therefore, further research in this area is recommended. Nonetheless, we also present a holistic baseline to address visual and contextual questions and foster future research on the topic.

NEURAL RADIANCE FIELDS (NERF): REVIEW AND POTENTIAL APPLICATIONS TO DIGITAL CULTURAL HERITAGE

Abstract. Neural Radiance Fields (NeRF or NeRFs) are to date emerging as a novel method for synthesizing novel views of complex 3D scenes, leveraging an artificial neural network to optimize a volumetric scene function using a set of input views. We conduct a preliminary critical review of the scientific and technical literature on NeRFs, and we highlight possible applications of the latter in the Cultural Heritage domain, for the image-based reconstruction of 3D models of real, multi-scale objects, even in combination with the more well-established photogrammetric techniques. A comparison is made between NeRFs and photogrammetry in terms of operating procedures and outputs (volumetric renderings vs. point clouds or meshes). It is demonstrated that NeRFs could be conveniently used for rendering objects (sculptures, archaeological remains, sites, paintings etc.) that are challenging for photogrammetry, typically: i) metallic, translucent, and/or transparent surfaces; ii) objects that present homogeneous textures; iii) occlusions, vegetation, and elements of very fine detail.

FEW SHOT PHOTOGRAMETRY: A COMPARISON BETWEEN NERF AND MVS-SFM FOR THE DOCUMENTATION OF CULTURAL HERITAGE

Abstract. 3D documentation methods for Digital Cultural Heritage (DCH) domain is a field that becomes increasingly interdisciplinary, breaking down boundaries that have long separated experts from different domains. In the past, there has been an ambiguous claim for ownership of skills, methodologies, and expertise in the heritage sciences. This study aims to contribute to the dialogue between these different disciplines by presenting a novel approach for 3D documentation of an ancient statue. The method combines TLS acquisition and MVS pipeline using images from a DJI Mavic 2 drone. Additionally, the study compares the accuracy and final product of the Deep Points (DP) and Neural Radiance Fields (NeRF) methods, using the TLS acquisition as validation ground truth. Firstly, a TLS acquisition was performed on an ancient statue using a Faro Focus 2 scanner. Next, a multi-view stereo (MVS) pipeline was adopted using 2D images captured by a Mini-2 DJI Mavic 2 drone from a distance of approximately 1 meter around the statue. Finally, the same images were used to train and run the NeRF network after being reduced by 90%. The main contribution of this paper is to improve our understanding of this method and compare the accuracy and final product of two different approaches - direct projection (DP) and NeRF - by exploiting a TLS acquisition as the validation ground truth. Results show that the NeRF approach outperforms DP in terms of accuracy and produces a more realistic final product. This paper has important implications for the field of CH preservation, as it offers a new and effective method for generating 3D models of ancient statues. This technology can help to document and preserve important cultural artifacts for future generations, while also providing new insights into the history and culture of different civilizations. Overall, the results of this study demonstrate the potential of combining TLS and NeRF for generating accurate and realistic 3D models of ancient statues.

Towards terminological resources tailored to the users’ needs: Terminology extraction based on appositive constructions

Abstract Terminological resources (TRs) are indispensable tools for accessing a specialized domain of knowledge. In this article, we propose a methodology for extracting terms and relevant linguistic information, useful for different users, hinging on the nature of some special linguistic structures: appositive constructions. The case study for our proof of concept in this investigation is the domain of Cultural Heritage. For such purpose, we compile a parallel Italian–English domain corpus in the specialized sub-domain of archaeology, for extracting data to be later stored in TRs tailored to three target users: (1) translators and interpreters, (2) technical communicators, and (3) the general public or laypeople.

An Experiment-based Comparative Analysis of Pigment Classification Algorithms using Hyperspectral Imaging

Hyperspectral imaging techniques are widely used in cultural heritage for documentation and material analysis. Pigment classification of an artwork is an essential task. Several algorithms have been used for hyperspectral data classification, and the effectiveness of each algorithm depends on the application domain. However, very few have been applied for pigment classification tasks in the cultural heritage domain. Most of these algorithms work effectively for spectral shape differences and might not perform well for spectra with differences in magnitude or for spectra that are nearly similar in shape but might belong to two different pigments. In this work, we evaluate the performance of different supervised-based algorithms and few machine learning models for the pigment classification of a mockup using hyperspectral imaging. The result obtained shows the importance of choosing appropriate algorithms for pigment classification.

Blockchain and NFTs in the Cultural Heritage Domain: A Review of Current Research Topics

In the last few years, blockchain technology and NFTs have been the subject of much research in different sectors ranging from informatics, to medicine, to economics. Although it is most often associated with cryptocurrencies, due to its features of immutability and durability, this technology has found its place in various fields, including GLAM institutions. This article will review the literature from 2017 to 2022 dealing with blockchain and NFTs in the heritage sector. Topics covered, proposed models, and projects will be highlighted. Archives are currently leading the research into the use of blockchain technology and have already developed models such as TrustChain. However, libraries, museums, and galleries are also beginning to show an interest in the new technology and its potential benefits. Therefore, we also approached the GLAM sector as a whole, to emphasize the importance of the joint development on the advancement of shared approaches and protocols in utilizing blockchain technology to enhance the trustworthy management and preservation of digital resources. This is particularly important because GLAM institutions care for a shared heritage and serve a common audience. In the second part of the article we will discuss the proposed uses of the technology and highlight still unexplored topics that should be elaborated in further research. The aim of this paper is to make a synthesis of previous research and bring the potential of blockchain technology and NFTs closer to experts in the heritage field, given that they are still quite unknown.

A storytelling framework based on multimedia knowledge graph using linked open data and deep neural networks

Automatic storytelling is a broad challenge in research contexts such as Natural Language Processing and Contend Based Image Analysis. Despite the considerable achievements of machine learning techniques in these research fields, combining different approaches to fill the gap between an automatic generated story and human handwriting is hard. This work proposes a novel storytelling framework in the Cultural Heritage domain. We developed our framework based on a Multimedia Knowledge Graph (MKG), a crucial point of our work. Furthermore, we populated our Multimedia Knowledge Graph with a focused crawler that employs deep learning techniques to recognise a multimedia object from web resources. Furthermore, we used a combined approach of deep learning techniques and Linked Open Data (LOD) to retrieve information about images and depicted figures using Instance Segmentation. The system has a dynamic, user-friendly interface that guides the user during the storytelling process. Finally, we evaluated the system from a qualitative and quantitative point of view.

Augmented, Virtual and Mixed Reality Research in Cultural Heritage: A Bibliometric Study

Heritage allows us to learn about the different monuments of importance and the inherited traditions from our ancestors. However, many times the monuments get partly ruined due to natural wear and tear, sometimes due to attacks by invaders. To preserve the cultural heritage virtually, many researchers have used augmented, virtual and mixed reality for bringing ancient environments to live in those heritage sites. This study aims to identify publications related to virtual, augmented and mixed reality in cultural heritage and to present the bibliometric analysis of these studies. The research articles on virtual, augmented, and mixed reality in cultural heritage are retrieved using the Scopus database. The analysis is performed using VOSviewer and various parameters such as bibliographic coupling of the countries, publications, journals, authors, and co-occurrences of the author keywords are performed. From the analysis done in this study, it is discovered that the augmented, virtual and mixed reality research in the domain of cultural heritage is mostly concentrated in Italy and surrounding European countries. However, it is also found that the research in this domain is lagging in many countries even if those countries are the homes of various heritage sites. This study provides an extensive analysis of the recent literature related to augmented, virtual and mixed reality research in cultural heritage. This information science based analysis will help researchers to identify the prominent journals in this domain, recognize stalwarts in this field and follow their works, find out path-breaking publications to refer to, and predict the direction of future studies.

WHY SHOULD YOU CARE ABOUT THE LINK BETWEEN SPORTS, AS AN INTANGIBLE CULTURAL HERITAGE AND GLOBAL SUSTAINABLE HEALTH DEVELOPMENT (UN SDG3)?

Introduction. The truth is I did not care until the global pandemic and fake news in the media MADE me care! Why? Because UN SDGs are vital to the survival of human life, indeed all lives, on the Anthropocene. What better time to research this as when the whole planet was locked down by one virus? This is a working paper using a subset of the data from a larger longitudinal study. The purpose of this paper was to investigate key questions around the transmission of Sports as a major domain of Intangible Cultural Heritage (ICH). Aim of the study. The study shed light on the “spontaneous and extemporary” way High Performance Sports/eSports ICH-creators transmit to its global community and in an unscripted and unprompted way, included in their transmission process contents that enriched the trust-building aspects of Sustainable strategic health development (SDG3). I isolated and studied this phenomenon by collecting data at an eSports Field configuring event (FCE), studying my subjects who were the highest-ranking world champions of their generation. Material and methods. Through an abductive qualitative analysis of “living culture” at a major FCE, studying the theories of a “living culture”-transmission (ICH) in the field, a paradoxical framework inquisites the unscripted trust-building commentaries regarding the health services provided at large sports venues, during ICH-creators’ Live global broadcast. Results. The initial findings are based on field data collected at high-level e-sports FCE event (World Chess championship 2020). My ICH-transmission framework cross-validated in practice, the implications that ICH like Sports/eSports have ties with UN SDG3 in strategic trans-disciplinary way. Conclusions. This research address theoretical assumptions around “tangibility/intangibility” of cultural manifestations. This will contribute long-term to the study of paradox in creative ICH-transmissions. In the short-term, the knowledge improves UN SDG Index-metrics for Sustainable Health (SDG3). My research challenges assumptions around the requirement of material manifestation of an ICH e.g., a tangible sport played intangibly as an e-sport can still transmit important wellbeing narratives in unscripted and vibrant ICH manifestations. When critically examined the ICH-transmission process of an eSport contributed to sustainable health promotion agenda by disarming polarity (conflict) caused by viral anti-vax disinformation campaigns; potentially steering the fearful towards an SDG-centric wellbeing mind-frame amidst an uncertainty of Sports and eSports event in the middle of a global health crisis.

CIDOC CRM as the basis of the Electronic State Register of Immovable Cultural Heritage of Ukraine

Abstract. The article is the final in a series of articles on Conceptual Provisions for the Creation of a New Electronic State Register of Immovable Cultural Heritage (CH) of Ukraine. These provisions correspond to the components of the Solutions Framework (SoFr) of special Spatial Information Systems (SpIS) defined in the monograph [15]. The special SpIS of the new registry of the CH of Ukraine should belong to the class of Atlas Geoinformation Systems (AGIS), which is described in [14]. The first queue of AGIS – AGIS-CH1 – is proposed as the first queue of the new electronic State Register of Immovable CH of Ukraine. The first queue should include, at least, three components that are simultaneously SoFr packages: Products-Processes-Basics. The conceptual provisions of AGIS-CH1 describe these three most important components of the architectural pattern of AGIS-CH1: AGIS-CH1.Products, Part 1 [17]; AGIS-CH1.Processes, Part 2 [18], AGIS-CH1.Basics, Part 3, consisting of two subparts, 3.1 and 3.2. Subpart 3.1 is described in the article [19]. This article describes subpart 3.2, which is called "Basics. CIDOC CRM". It consists of two main sections. The first of these initially examines the prerequisites that lead to the use of CIDOC CRM. Such prerequisites are two evolutions: system and subject. System evolution claims that the time has come to consider the registry of the CH of Ukraine as a SpIS of the Web 3.0 Formation, also known as the Semantic Web, especially if we have in mind the creation of a new registry. Subject evolution refers to the evolution of understanding of the domain of cultural heritage. From the review of this issue in the monograph [4], it is obvious that modern CH registers should be "process" rather than "product". In order to proceed to the consideration of CIDOC CRM with a better understanding of the essence of the issue, the CHARM model (Cultural Heritage Abstract Reference Model) was considered. CIDOC CRM can also be considered as such, but CHARM is described in an excellent monograph [4] that is practically applicable, unlike the scattered articles on CIDOC CRM. The second of the two main sections of the article deals with CIDOC CRM and its use. We do not describe CIDOC CRM completely. Attention is paid only to its "spatial" and "process" parts. In addition, attention is paid to the use of CIDOC CRM. For this, the information from the website (https://www.cidoc-crm.org/how-i-can-use-cidoc-crm, 2023-jun-26) is used first. Then there is some initial information about the Arches software platform. We offer the Arches platform for the implementation of AGIS-CH1.

Move cultural heritage knowledge graphs in everyone’s pocket

Last years witnessed a shift from the potential utility in digitisation to a crucial need to enjoy activities virtually. In fact, before 2019, data curators recognised the utility of performing data digitisation, while during the lockdown caused by the COVID-19, investing in virtual and remote activities to make culture survive became crucial as no one could enjoy Cultural Heritage in person. The Cultural Heritage community heavily invested in digitisation campaigns, mainly modelling data as Knowledge Graphs by becoming one of the most successful Semantic Web technologies application domains. Despite the vast investment in Cultural Heritage Knowledge Graphs, the syntactic complexity of RDF query languages, e.g., SPARQL, negatively affects and threatens data exploitation, risking leaving this enormous potential untapped. Thus, we aim to support the Cultural Heritage community (and everyone interested in Cultural Heritage) in querying Knowledge Graphs without requiring technical competencies in Semantic Web technologies. We propose an engaging exploitation tool accessible to all without losing sight of developers’ technological challenges. Engagement is achieved by letting the Cultural Heritage community leave the passive position of the visitor and actively create their Virtual Assistant extensions to exploit proprietary or public Knowledge Graphs in question-answering. By accessible to all, we mean that the proposed software framework is freely available on GitHub and Zenodo with an open-source license. We do not lose sight of developers’ technical challenges, which are carefully considered in the design and evaluation phases. This article first analyses the effort invested in publishing Cultural Heritage Knowledge Graphs to quantify data developers can rely on in designing and implementing data exploitation tools in this domain. Moreover, we point out challenges developers may face in exploiting them in automatic approaches. Second, it presents a domain-agnostic Knowledge Graph exploitation approach based on virtual assistants as they naturally enable question-answering features where users formulate questions in natural language directly by their smartphones. Then, we discuss the design and implementation of this approach within an automatic community-shared software framework (a.k.a. generator) of virtual assistant extensions and its evaluation in terms of performance and perceived utility according to end-users. Finally, according to a taxonomy of the Cultural Heritage field, we present a use case for each category to show the applicability of the proposed approach in the Cultural Heritage domain. In overviewing our analysis and the proposed approach, we point out challenges that a developer may face in designing virtual assistant extensions to query Knowledge Graphs, and we show the effect of these challenges in practice.

Semantic models and services for conservation and restoration of cultural heritage: A comprehensive survey

Over the last decade, the Cultural Heritage (CH) domain has gradually adopted Semantic Web (SW) technologies for organizing information and for tackling interoperability issues. Several semantic models have been proposed which accommodate essential aspects of information management: retrieval, integration, reuse and sharing. In this context, the CH subdomain of Conservation and Restoration (CnR) exhibits an increasing interest in SW technologies, in an attempt to effectively handle the highly heterogeneous and often secluded CnR information. This paper investigates semantic models relevant to the CnR knowledge domain. The scope, development methodology and coverage of CnR aspects are described and discussed. Furthermore, the evaluation, deployment and current exploitation of each model are examined, with focus on the types and variety of services provided to support the CnR professional. Through this study, the following research questions are investigated: To what extent the various aspects of CnR are covered by existing CnR models? To what extent existing CnR models incorporate models of the broader CH domain and of relevant disciplines (e.g., Chemistry)? In what ways and to what extent services built upon the reviewed models facilitate CnR professionals in their various tasks? Finally, based on the findings, fields of interest that merit further investigation are suggested.

Generation of training data for named entity recognition of artworks

As machine learning techniques are being increasingly employed for text processing tasks, the need for training data has become a major bottleneck for their application. Manual generation of large scale training datasets tailored to each task is a time consuming and expensive process, which necessitates their automated generation. In this work, we turn our attention towards creation of training datasets for named entity recognition (NER) in the context of the cultural heritage domain. NER plays an important role in many natural language processing systems. Most NER systems are typically limited to a few common named entity types, such as person, location, and organization. However, for cultural heritage resources, such as digitized art archives, the recognition of fine-grained entity types such as titles of artworks is of high importance. Current state of the art tools are unable to adequately identify artwork titles due to unavailability of relevant training datasets. We analyse the particular difficulties presented by this domain and motivate the need for quality annotations to train machine learning models for identification of artwork titles. We present a framework with heuristic based approach to create high-quality training data by leveraging existing cultural heritage resources from knowledge bases such as Wikidata. Experimental evaluation shows significant improvement over the baseline for NER performance for artwork titles when models are trained on the dataset generated using our framework.

Multimodal metadata assignment for cultural heritage artifacts

We develop a multimodal classifier for the cultural heritage domain using a late fusion approach and introduce a novel dataset. The three modalities are Image, Text, and Tabular data. We based the image classifier on a ResNet convolutional neural network architecture and the text classifier on a multilingual transformer architecture (XML-Roberta). Both are trained as multitask classifiers. Tabular data and late fusion are handled by Gradient Tree Boosting. We also show how we leveraged a specific data model and taxonomy in a Knowledge Graph to create the dataset and to store classification results.

Bridging Digital Approaches and Legacy in Archaeology

The emergence of the ubiquitous digital ecosystem has provided new momentum for research in archaeology and the cultural heritage domain [...]

Extraction of Event-Related Information from Text for the Representation of Cultural Heritage

In knowledge representation systems for Cultural Heritage (CH) there is a vast amount of curated textual information for CH objects and sites. However, the large-scale study of the accumulated knowledge is difficult as long as it is provided in the form of free text. By extracting the most significant pieces of information from textual descriptions of CH objects and sites and compiling them in a single comprehensive knowledge graph, conforming to a standard would facilitate its exploitation from multiple perspectives including study, presentation and narratives. The method proposed by this research work was to employ Natural Language Processing, and reinforcement learning for semantic knowledge extraction, and a knowledge representation standard of the CH domain for the knowledge graph thus making the extracted knowledge directly compatible with linked open data platforms and CH representation systems.