Abstract
Cultural heritage sites are precious and fragile resources that hold significant historical, esthetic, and social values in our society. However, the increasing frequency and severity of natural and man-made disasters constantly strike the cultural heritage sites with significant damages. In this article, we focus on a cultural heritage damage assessment (CHDA) problem where the goal is to accurately locate the damaged area of a cultural heritage site using the imagery data posted on social media during a disaster event by exploring the collective strengths of both AI and human intelligence from crowdsourcing systems. Unlike other infrastructure-based solutions, social media platforms provide a more pervasive and scalable solution to acquire timely cultural heritage damage information during disaster events. Our work is motivated by the limitation of current AI solutions that fail to accurately model the complex cultural heritage damage due to the lack of essential human cultural knowledge to differentiate various damage types and identify the actual causes of the damage. Two critical technical challenges exist in solving our problem: 1) it is challenging to effectively detect the problematic cultural heritage damage estimation of AI in the absence of ground truth labels and 2) it is nontrivial to acquire accurate cultural background knowledge from the potentially unreliable crowd workers to effectively address the failure cases of AI. To address the above-mentioned challenges, we develop CollabLearn, an uncertainty-aware crowd-AI collaborative assessment system that explicitly explores the human intelligence from crowdsourcing systems to identify and fix AI failure cases and boost the damage assessment accuracy in CHDA applications. The evaluation results on real-world datasets show that CollabLearn consistently outperforms both the state-of-the-art AI-only and crowd-AI hybrid baselines in accurately assessing the damage of several world-renowned cultural heritage sites in recent disaster events.
Highlights
IntroductionC ULTURAL heritage sites (e.g., historical buildings, monuments, archeological sites, and landscapes) are precious and fragile resources that hold significant historical, esthetic, and social values in our society [1]
C ULTURAL heritage sites are precious and fragile resources that hold significant historical, esthetic, and social values in our society [1]
The results show that CollabLearn consistently outperforms the state-of-the-art AI-only and crowd-AI hybrid baselines in terms of cultural heritage damage assessment (CHDA) accuracy under various application scenarios
Summary
C ULTURAL heritage sites (e.g., historical buildings, monuments, archeological sites, and landscapes) are precious and fragile resources that hold significant historical, esthetic, and social values in our society [1]. This article focuses on an emerging application, cultural heritage damage assessment (CHDA), that aims to protect and conserve cultural heritage sites. The objective of CHDA applications is to accurately locate the damaged areas of a cultural heritage site by exploring the imagery data posted on social media during a disaster event. Unlike other infrastructure-based solutions (e.g., using surveillance cameras, drones, and satellites), the social media platforms provide an infrastructure-free solution that is more pervasive and scalable to acquire timely damage information of the cultural heritage sites during disaster events [3]–[5]. The assessment information can be leveraged by the government agencies and organizations to provide conservation and recovery actions to the sites and save them from further damages
Sign-in/Register to view highlights & summary Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
