Abstract
Abstract. Creating three-dimensional as-built models from point clouds is still a challenging task in the Cultural Heritage environment. Nowadays, performing such task typically requires the quite time-consuming manual intervention of an expert operator, in particular to deal with the complexities and peculiarities of heritage buildings. Motivated by these considerations, the development of automatic or semi-automatic tools to ease the completion of such task has recently became a very hot topic in the research community. Among the tools that can be considered to such aim, the use of deep learning methods for the semantic segmentation and classification of 2D and 3D data seems to be one of the most promising approaches. Indeed, these kinds of methods have already been successfully applied in several applications enabling scene understanding and comprehension, and, in particular, to ease the process of geometrical and informative model creation. Nevertheless, their use in the specific case of heritage buildings is still quite limited, and the already published results not completely satisfactory. The quite limited availability of dedicated benchmarks for the considered task in the heritage context can also be one of the factors for the not so satisfying results in the literature.Hence, this paper aims at partially reducing the issues related to the limited availability of benchmarks in the heritage context by presenting a new dataset for semantic segmentation of heritage buildings. The dataset is composed by both images and point clouds of the considered buildings, in order to enable the implementation, validation and comparison of both point-based and multiview-based semantic segmentation approaches. Ground truth segmentation is provided, for both the images and point clouds related to each building, according to the class definition used in the ARCHdataset, hence potentially enabling also the integration and comparison of the results obtained on such dataset.
Highlights
In the Cultural Heritage (CH) environment, Heritage Building Information Modelling (H-BIM) has gained particular attention in recent years, due to the growing interest in protection, conservation, and restoration of historical buildings (López et al, 2018; Volk et al, 2014)
Several techniques have been successfully used in a wide of vision application: algorithmic approach (Murtiyoso and Grussenmeyer, 2020), machine learning (ML) (Croce et al, 2021), Neural Networks (NN), and especially Deep Learning (DL) with the introduction of Convolutional Neural Networks (CNN) for 2D image processing (Zhang et al, 2019; Long et al, 2015)
The main aim of the benchmark is to offer the possibility to implement and compare multi-view approaches on heritage building scenarios and leverage on the existing 2D segmentation architecture to ease the development of new classification machine learning and deep learning techniques
Summary
In the Cultural Heritage (CH) environment, Heritage Building Information Modelling (H-BIM) has gained particular attention in recent years, due to the growing interest in protection, conservation, and restoration of historical buildings (López et al, 2018; Volk et al, 2014). To the best of our knowledge, the use of multiview-based methods in this framework has not been experimented yet, probably due to the lack of a dedicated benchmark for 2D image semantic segmentation of cultural heritage buildings. Despite multiview-based approaches may introduce some information loss, due to the use of an intermediate data representation, dealing with 2D images could still be an effective strategy. On one hand, this allows to exploit the well-established NN-based 2D image semantic segmentation techniques, and, on the other hand, nowadays LiDAR and photogrammetric surveys are often integrated, their combination can represent a viable way to transfer the semantic knowledge extracted from images to the corresponding point cloud. The integration with point-based methods may lead to a hybrid method that may improve the performance of both such approaches
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Free) 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
