3D Cultural Heritage Research Articles

Revolutionizing Depth Sensing: A Review Study of Apple LiDAR Sensor for as-built Scanning Applications

Incorporating the LiDAR sensor in the most recent Apple devices represents a substantial development in 3D mapping technology. Meanwhile, Apple's Lidar is still a new sensor. Therefore, this article reviews the potential uses of the Apple Lidar sensor in various fields, including engineering and construction, focusing on indoor and outdoor as-built 3D mapping and cultural heritage conservation. The affordable cost and shorter observation times compared to traditional surveying and other remote sensing techniques make the Apple Lidar an attractive choice among scholars and professionals. This article highlights the need for continued research on the Apple LiDAR sensor technology while discussing its specifications and limitations. A comprehensive review found that the Apple LiDAR sensor has shown promise in capturing 3D point clouds of small to medium-sized objects with exceptional detail. This technology offers a cost-effective and accessible option to scan areas faster and analyze data more quickly and automatically for 3D mapping and modelling in indoor and outdoor environments, particularly in areas with restricted access when using other traditional techniques. It also opens the door for more sophisticated applications in future studies, including cultural heritage conservation, archaeological investigations and feature detection, building health monitoring and many more.

METHODS, DATA AND TOOLS FOR FACILITATING A 3D CULTURAL HERITAGE SPACE

Abstract. In recent years, the cultural heritage (CH) sector has experienced a rapid evolution due to the introduction of increasingly powerful digital technologies and ICT (Information and Communication Technologies) solutions. As for many other domains, digital data, Artificial Intelligence (AI), and Extended Reality (XR) are opening up extraordinary opportunities for expanding heritage knowledge capabilities while boosting the research on innovative solutions for its valorisation and preservation. Being aware of the fundamental and strategic role of CH in the history and identity of the European countries, the European Commission has assumed a central role in fuelling the policy debate and putting together stakeholders to take a step forward in CH digitization and use, primarily through initiatives, programs, and recommendations. Within this framework, the ongoing European 5DCulture project (https://www.5dculture.eu/) has been funded to enrich the offer of 3D CH digital assets in the common European Data Space for Cultural Heritage by creating high-quality 3D contents and to foster their re-use in several sectors, from tourism to education. Through 8 re-use scenarios around historic buildings and cityscapes, archaeology, and fashion, the project aims to deliver a set of digital tools and increase the capacity of CH institutions to more effectively re-use their 3D digital assets.

Nanite as a Disruptive Technology for the Interactive Visualisation of Cultural Heritage 3D Models: A Case Study

The use of digital models of cultural heritage objects obtained from 3D scanning or photogrammetry requires the development of strategies in order to optimise computational resources and enhance the user experience when they are used in interactive applications or virtual museums. Through a case study, this work compares an original photogrammetric model with its optimised version using traditional remeshing techniques and an improved version using Nanite technology, developed by Epic Games. A self-contained executable is created in Unreal® 5.1 game engine to test the performance of the three models measured in frames per second (FPS). As a result, it was demonstrated that, although there is no substantial difference in the FPS rate, the Nanite technology avoids the need to perform the mesh and texture editing processes that lead to the construction of the optimised model. This saves considerable time and specialised effort, as the photogrammetric model can be converted to a Nanite object automatically. This would be a great advantage in the case of virtualisations of large collections of heritage objects, which is a common case in virtual museums.

Point Cloud-Based Historical Building Information Modeling (H-BIM) in Urban Heritage Documentation Studies

Integrating building information modeling (BIM) technology into historical and cultural heritage structures has resulted in historic building information modeling (H-BIM), an effective method for managing and documenting invaluable historical artifacts. H-BIM has the potential to aid with the design, restoration, and maintenance activities of historical buildings by properly integrating parametric data and deliberately assessing semantic information, thus progressing the 3D model’s performance. H-BIM also aims to conduct cultural heritage (CH) studies in a systematic manner by combining 3D modeling techniques and information management. The first step in this process is to conduct a suitable geomatic survey to build accurate 3D models. The most efficient modeling method is to employ 3D point cloud data that are acquired using the terrestrial laser scanning method. This study proposes a manual modeling strategy for urban historical buildings that falls within the scope of the H-BIM methodology. The proposed modeling strategy follows a workflow that inserts point cloud data into the BIM environment in order to create a 3D urban cultural heritage model using ontologies applied with the H-BIM methodology. With this objective in mind, the H-BIM model development platform Revit was used to model the object elements of the inheritance structure in 3D and save them in the BIM library. It is very important to model building elements separately in 3D modeling studies. The resulting BIM product sheds light on fundamental data for the future planning, design, operation, and maintenance processes of the building. Users can obtain data suitable for H-BIM by adding graphical and non-graphical data.

RESEARCH ON THE 3D DOCUMENTATION SYSTEM OF CLASSICAL CHINESE GARDENS IN SCENIC AREAS – TAKING KUNSHAN SUIYUAN AS AN EXAMPLE

Abstract. Classical Chinese gardens built in scenic areas have characteristics in rich scales, multi-type elements, and a complex development history. Due to the designed landscape characteristics of "hidden the garden in bigger landscapes", it is often difficult to obtain high-quality heritage information through traditional surveying approaches. The aim of this research is to establish a digital 3D documentation technology system for the classical gardens in scenic areas. Suiyuan Garden, a classical Chinese garden with a history of more than 300 years, was used as a case study. We have integrated UAV low-altitude photogrammetry, handheld laser scanning, ground-mounted laser scanning, macro structured light scanning and other technologies to obtain the 3D data, and to build a digital twin by developing the technical workflow and data standard. The innovation of the research lies in: 1) We used a variety of digital mapping methods to record the pattern between the garden and the surrounding landscape; 2) We adopted a combination of handheld and ground-mounted laser scanning systems to build spatial structures and better cover the ground spaces; 3) We established a workflow for the conversion from point cloud to mesh model for different garden features, which can improve the usability of 3D heritage information; 4) We used the 3D cultural heritage modelling approach to improve the standards of the digital conservation of historic plants. The outcomes of this research could be used as the basis for the construction of Historic Landscape Information Model (HLIM) in the near future.

PRODUCTION OF DIGITAL CONTENT (3D ASSET) SOURCE RESOURCES FOR CULTURAL HERITAGE

Abstract. In Korea, several 3D digital documentation projects have been carried out to preserve cultural heritage, but limitations in high-capacity/high-quality data have hindered their use in virtual restoration. To address this, organizations have produced low-capacity, high-quality 3D assets since 2020, but disseminating 3D assets with metadata containing cultural heritage information is important to reduce errors in historical research during virtual restoration. To overcome this problem, the Cultural Heritage Administration of Korea initiated the ‘Cultural Heritage Digital Content (3D Asset) Source Resource Production’ project in 2022. The project aims to convey and share the value of cultural heritage through 3D assets, which are complex due to the artistic values of units that must be contextually connected. Spatial and asset characteristics were classified into six categories of cultural heritage 3D assets, with the goal of turning tangible and intangible cultural heritage into 3D assets for comprehensive utilization. The cultural heritage digital source resources are a valuable resource for the creative industry.

5DMETEORA FRAMEWORK: MANAGEMENT AND WEB PUBLISHING OF CULTURAL HERITAGE DATA

Abstract. Cultural Heritage (CH) management software represents virtual information in various ways aiming either at usability and long-term preservation or interactivity and immersiveness. A single web-based framework that couples the organization of geospatial, multimedia and relational data with 4D visualization, Virtual Reality (VR) and Augmented Reality (AR) is presented in this paper (https://meteora.topo.auth.gr/5dmeteora.php). It comprises the 5dMeteora platform and the Content Management System (CMS) for uploading, processing, publishing and updating its content. The 5dMeteora platform integrates a responsive 3D viewer of high-resolution models in the basis of 3DHOP (3D Heritage Online Platform) and Nexus.js multi-resolution library. It offers data retrieval and interpretation mechanisms through navigation tools, clickable geometries in the 3D scene, named hotspots, and semantic organization of metadata. Its content and interactive services are differentiated, based on the scientific specialty or the field of interest of the users. To achieve the sense of spatial presence, VR and AR viewports are designed to give a clearer understanding of spatial bounds and context of 3D CH assets. The proposed CMS allows dynamic content management, automation of 3DHOP’s operations regarding 3D data uploading and hotspots defining, real-time preview of the 3D scene as well as extensibility at all levels (e.g., new data types). It is built upon a MySQL Database Management System and developed with PHP scripting, backend JavaScript and Ajax controllers as well as front-end web languages. The database maintains and manages the entities of every type of data supported by the platform, while encryption methods guarantee data confidentiality and integrity. The presented work is the first valid attempt of open-source software that automates the dissemination of 3D and 2D content for customized eXtented Reality (XR) experiences and reaches multiple levels of interactivity for different users (experts, non-experts). It can meet the needs of domain experts that own or manage multimodal heritage data.

3D 입체퍼즐을 활용한 문화유산 교육 프로그램 운영 방안 모색

Recently, as the level of public awareness about cultural heritage has improved, the need for more effective cultural heritage education programs is increasing. In this study, we analyzed all cases of cultural heritage 3D puzzle experiential education, which has recently been in the limelight in the face of the COVID-19 situation. Based on the results, a new way of undertaking cultural heritage education programs was attempted by operating a 3D puzzle experience and education program at Jeongeup Museong Seowon for in Jeongeup City elementary school students. The educational effects of the 3D puzzle experience are (1) a sense of achievement felt by participating students, (2) increased pride in the local cultural heritage felt by family members and the local community by owning the finished product. In addition, appropriate lectures and experiential education by professional instructors was appropriately arranged in consideration of the level of the students, and it was confirmed that the effect of the class is doubled when the students adopt the method of directly exploring the cultural heritage or watching related videos.

Harnessing Collective Differences in Crowdsourcing Behaviour for Mass Photogrammetry of 3D Cultural Heritage

Disorganised and self-organised crowdsourcing activities that harness collective behaviours to achieve a specific level of performance and task completeness are not well understood. Such phenomena become indistinct when highly varied environments are present, particularly for crowdsourcing photogrammetry-based 3D models. Mass photogrammetry can democratise traditional close-range photogrammetry procedures by outsourcing image acquisition tasks to a crowd of non-experts to capture geographically scattered 3D objects. To improve public engagement, we need to understand how individual behaviour in collective efforts work in traditional disorganised crowdsourcing and how it can be organised for better performance. This research aims to investigate the effectiveness of disorganised and self-organised collaborative crowdsourcing. It examines the collaborative dynamics among participants and the trends we could leverage if team structures were incorporated. Two scenarios were proposed and constructed: asynchronous crowdsourcing, which implicitly aggregates isolated contributions from disorganised individuals; and synchronous collaborative crowdsourcing, which assigns participants into a crowd-based self-organised team. Our experiment demonstrated that a self-organised team working in synchrony can effectively improve crowdsourcing photogrammetric 3D models in terms of model completeness and user experience. Through our study, we demonstrated that this crowdsourcing mechanism can provide a social context where participants can exchange information via implicit communication, and collectively build a shared mental model that pertains to their responsibilities and task goals. It stimulates participants’ prosocial motivation and reinforces their commitment. With more time and effort invested, their positive sense of ownership increases, fostering higher dedication and better contribution. Our findings shed further light on the potentials of adopting team structures to encourage effective collaborations in conventionally individual-based voluntary crowdsourcing settings, especially in the digital heritage domain.

Multi-Scale Upsampling GAN Based Hole-Filling Framework for High-Quality 3D Cultural Heritage Artifacts

With the rapid development of 3D scanners, the cultural heritage artifacts can be stored as a point cloud and displayed through the Internet. However, due to natural and human factors, many cultural relics had some surface damage when excavated. As a result, the holes caused by these damages still exist in the generated point cloud model. This work proposes a multi-scale upsampling GAN (MU-GAN) based framework for completing these holes. Firstly, a 3D mesh model based on the original point cloud is reconstructed, and the method of detecting holes is presented. Secondly, the point cloud patch contains hole regions and is extracted from the point cloud. Then the patch is input into the MU-GAN to generate a high-quality dense point cloud. Finally, the empty areas on the original point cloud are filled with the generated dense point cloud patches. A series of real-world experiments are conducted on real scan data to demonstrate that the proposed framework can fill the holes of 3D heritage models with grained details. We hope that our work can provide a useful tool for cultural heritage protection.

Benchmark Based Selection Of Point Cloud Registration Algorithms For Cultural Heritage 3D Digitization

Abstract This work aims to address the point cloud global registration algorithm selection problem by proposing a pipeline for testing and evaluating such algorithms. Moreover, this benchmark-based evaluation automates the algorithm selection process by assessing the point cloud registration algorithms under the same universal benchmark, using complete and diverse datasets. We demonstrate its performance by comparing two State-of-the-art global registration algorithms that are tested on a set of problems created by diverse, unstructured datasets, acquired by various types of sensors, and in different environmental conditions. Furthermore, a comprehensive metric is proposed, which serves as a qualitative metric, rather than an absolute result for the evaluation process. Finally, the different outcomes of the algorithms benchmarking performance are thoroughly presented and discussed.

Artifacsimile –3D printed preservation at Monticello

Monticello, Thomas Jefferson’s plantation in Virginia, experimented with displaying archaeological artifacts in a permanent outdoor exhibition on enslaved men, women, and children. Unfortunately, the artifacts began to deteriorate from exposure to environmental changes. How could cultural heritage artifacts be preserved, while still contributing to the exhibition? Curators and archaeologists collaborated with the University of Virginia Library to collect 3D data and produce facsimiles for display in outdoor exhibitions. By producing a printed object based on precision 3D data, an “artifacsimile” becomes a physical representation of the original with the resulting 3D data processed, archived, and accessible to scholars and educators. Artifacsimiles at Monticello are a case study for the benefits and efficacy of 3D technologies for preservation and exhibit, in support of the University of Virginia’s efforts to collect, process, archive and disseminate 3D data using methods of 3D Cultural Heritage Informatics.

3D reconstruction of Borobudur reliefs from 2D monocular photographs based on soft-edge enhanced deep learning

We propose a deep learning-based reconstruction method for three-dimensional (3D) cultural heritage objects, which are destroyed or disappear and thus are unavailable for 3D scanning to create digital archives. The proposed reconstruction method uses a single monocular photo and predicts its depth values based on a monocular depth estimation network. While most recent studies have focused on scenes with sharp (high-curvature) edges and large depth differences, such as indoor and outdoor scenes, the proposed method is designed to exert its performance on scenes with small depth differences and soft (low-curvature) edges. The key idea is to involve the soft edges as guidance information in the depth estimation network. We demonstrate the effectiveness of the proposed method by applying it to the hidden reliefs of the Borobudur temple, a UNESCO world heritage site. The hidden reliefs were buried behind stone walls and are no longer visible. The 3D information of the reliefs can only be estimated from their old monocular photos. We achieved 97% reconstruction accuracy for validation data. We also integrate the reconstructed hidden reliefs with 3D scanned data of the visible regions and execute see-through visualization for the integrated data. This visualization can simultaneously show us both visible regions and invisible regions of the temple.

A Web GIS-Based Integration of 3D Digital Models with Linked Open Data for Cultural Heritage Exploration

In recent years, considerable efforts have been made by cultural heritage institutions across the globe to digitise cultural heritage sites, artifacts, historical maps, etc. for digital preservation and online representation. On the other hand, ample research projects and studies have been published that demonstrate the great capabilities of web-geographic information systems (web-GIS) for the dissemination and online representation of cultural heritage data. However, cultural heritage data and the associated metadata produced by many cultural heritage institutions are heterogeneous. To make this heterogeneous data more interoperable and structured, an ever-growing number of cultural heritage institutions are adopting linked data principles. Although the cultural heritage domain has already started implementing linked open data concepts to the cultural heritage data, there are not many research articles that present an easy-to-implement, free, and open-source-based web-GIS architecture that integrates 3D digital cultural heritage models with cloud computing and linked open data. Furthermore, the integration of web-GIS technologies with 3D web-based visualisation and linked open data may offer new dimensions of interaction and exploration of digital cultural heritage. To demonstrate the high potential of integration of these technologies, this study presents a novel cloud architecture that attempts to enhance digital cultural heritage exploration by integrating 3D digital cultural heritage models with linked open data from DBpedia and GeoNames platforms using web-GIS technologies. More specifically, a digital interactive map, 3D digital cultural heritage models, and linked open data from DBpedia and GeoNames platforms were integrated into a cloud-based web-GIS architecture. Thus, the users of the architecture can easily interact with the digital map, visualise 3D digital cultural heritage models, and explore linked open data from GeoNames and DBpedia platforms, which offer additional information and context related to the selected cultural heritage site as well as external web resources. The architecture was validated by applying it to specific case studies of Australian cultural heritage and seeking expert feedback on the system, its benefits, and scope for improvement in the near future.

A Benchmark Dataset for Repetitive Pattern Recognition on Textured 3D Surfaces

AbstractIn digital archaeology, a large research area is concerned with the computer‐aided analysis of 3D captured ancient pottery objects. A key aspect thereby is the analysis of motifs and patterns that were painted on these objects' surfaces. In particular, the automatic identification and segmentation of repetitive patterns is an important task serving different applications such as documentation, analysis and retrieval. Such patterns typically contain distinctive geometric features and often appear in repetitive ornaments or friezes, thus exhibiting a significant amount of symmetry and structure. At the same time, they can occur at varying sizes, orientations and irregular placements, posing a particular challenge for the detection of similarities. A key prerequisite to develop and evaluate new detection approaches for such repetitive patterns is the availability of an expressive dataset of 3D models, defining ground truth sets of similar patterns occurring on their surfaces. Unfortunately, such a dataset has not been available so far for this particular problem. We present an annotated dataset of 82 different 3D models of painted ancient Peruvian vessels, exhibiting different levels of repetitiveness in their surface patterns. To serve the evaluation of detection techniques of similar patterns, our dataset was labeled by archaeologists who identified clearly definable pattern classes. Those given, we manually annotated their respective occurrences on the mesh surfaces. Along with the data, we introduce an evaluation benchmark that can rank different recognition techniques for repetitive patterns based on the mean average precision of correctly segmented 3D mesh faces. An evaluation of different incremental sampling‐based detection approaches, as well as a domain specific technique, demonstrates the applicability of our benchmark. With this benchmark we especially want to address the geometry processing community, and expect it will induce novel approaches for pattern analysis based on geometric reasoning like 2D shape and symmetry analysis. This can enable novel research approaches in the Digital Humanities and related fields, based on digitized 3D Cultural Heritage artifacts. Alongside the source code for our evaluation scripts we provide our annotation tools for the public to extend the benchmark and further increase its variety.

Metadata schema and ontology for capturing and processing of 3D cultural heritage objects

Motivated by the increased use of 3D acquisition of objects by cultural heritage institutions, we were investigating ontologies and metadata schemes for the acquisition process to provide details about the 3D capturing, which can be combined with preexisting ontologies describing an object. Therefore we divided the 3D capturing workflow into common steps starting with the object being placed in front of a 3D scanner to preparation and publication of the 3D datasets and/or derived images. While the proposed ontology is well defined on a coarse level of detail for very different techniques, e.g. Stucture from Motion and LiDAR we elaborated the metadata scheme in very fine detail for 3D scanners available at our institutions. This includes practical experiments with measurement data from past and current projects including datasets published at Zenodo as guiding examples and the source code for their computation. Additionally, the free and Open Source GigaMesh Software Framework’s analysis and processing methods have been extended to provide metadata about the 3D processing steps like mesh cleaning as well as 2D image generation. Finally, we discuss the current limitations and give an outlook about future extensions.

3D MODELLING USING AERIAL OBLIQUE IMAGES WITH CLOSE RANGE UAV BASED DATA FOR SINGLE OBJECTS

Abstract. The request for 3D Data for the use of 3D city-models is increasing rapidly. More and more tools are able to deal with data of several sensors, out of video-streams, oblique camera setups with huge overlaps as well as terrestrial data. To achieve high accuracy of the data and a fast processing pipeline, a smart workflow has to be defined and established. However, mixed data sources are still a challenge especially if different sensors with an extremely different GSD are used. This abstracts demonstrates such a workflow, the processing pipeline and the challenges in a mixed data processing. Special calibration and co-calibrating procedures have been applied to get model in model solution managed to solve the dual task of 3D city mapping and cultural heritage conservation. Especially the sensor setup directly influences the geometric accuracy of the product. To do missions for 2–5 cm GSD, metric systems are indispensable while for non-metric applications also simple and cheaper sensors do their job. Besides the different data-sources and sensors, the way of capturing and the related projection is a critical issue. While the classical oblique imaging is a standardized airborne application, captures with UAVs are more like close range photogrammetry on the facades. The combination requests specific pre-processing and definition and transformation steps.

The use of image and laser scanner survey archives for cultural heritage 3D modelling and change analysis

Cultural heritage studies often require the analysis of buildings that have undergone several changes and alterations during their lifetime. This often implies the loss of architectural elements or the construction of new elements, which both change the characteristics of the former buildings. The recovery of lost elements or structures through virtual reconstruction is of paramount importance in both scientific and cultural applications. Novel procedures in surveying and photogrammetric processing including historical photogrammetry and historical terrestrial laser scanning offer powerful tools that enable the extraction of geometric information from historical documentation such as archival images. This paper presents the integration of a metric 3D model with information present in archival surveys of lost architectural volumes. The methodology implies the availability of historical plans representing the survey object at scales consistent with UAV surveys and featuring shared elements. The methodology used to frame these plans in the reference system of the UAV survey for an open source GIS environment is also described as well as the accuracy checks. Finally, the procedure followed for the virtual reconstruction of the Fortezza in a BIM environment, which produced a model derived from the integration of historic and current data, is described.

Basic Research on 3D Cultural Heritage Packaging Technology Using Thermoplastic Polyurethane Elastomers

This study investigated mechanical property changes by measuring compression factors, resilience, and compressive strength according to packaging pattern and filling rate to identify the applicability of cultural heritage packaging using thermoplastic polyurethane elastomers (TPU). Research results indicate that the cross-shaped 3D pattern showed the best resilience when the internal filling rate was 20%, while the octet pattern was the best when the filling rate was either 40 and 60%. The octet pattern had the best mechanical properties and stability with resistance capacities of 20.79 kgf/cm2, 40.40 kgf/cm2, and 82.23 kgf/cm2 at 38%, 39%, and 40% recovery speeds, respectively, depending on the internal filling rate (20, 40, 60%). Based on these results, basic data on the applicability, stability, and reliability of 3D cultural heritage packaging materials using TPU materials were obtained.

Bridging the Gap between 3D Navigation and Semantic Search. The INCEPTION platform

The paper presents the main outcomes and future development of the INCEPTION project, “Inclusive Cultural Heritage in Europe through 3D semantic modelling”, funded by the European Commission under the Horizon 2020 Work Programme Europe in a changing world – inclusive, innovative and reflective Societies (Call Reflective-7-2014, Advanced 3D modelling for accessing and understanding European cultural assets) and completed in May 2019. In particular, the key-targeted project achievement will be presented: a specific cloud-based platform conceived to accomplish the main objectives of accessing, understanding and strengthening European Cultural Heritage by means of enriched 3D models. The need for digital data interpretation, in addition to documentation, guided the overall process of the cross-disciplinary work methodology, based on new methods and tools for 3D surveying and H-BIM modelling, new approaches and methodologies for Cultural Heritage 3D data inclusive access and exploitation by means of the Platform.