Abstract
Crowdsourced images hold information could potentially be used to remotely monitor heritage sites, and reduce human and capital resources devoted to on-site inspections. This article proposes a combination of semantic image segmentation and photogrammetry to monitor changes in built heritage sites. In particular, this article focuses on segmenting potentially damaging plants from the surrounding stone masonry and other image elements. The method compares different backend models and two model architectures: (i) a one-stage model that segments seven classes within the image, and (ii) a two-stage model that uses the results from the first stage to refine a binary segmentation for the plant class. The final selected model can achieve an overall IoU of 66.9% for seven classes (54.6% for one-stage plant, 56.2% for two-stage plant). Further, the segmentation output is combined with photogrammetry to build a 3D segmented model to measure the area of biological growth. Lastly, the main findings from this paper are: (i) With the help of transfer learning and proper choice of model architecture, image segmentation can be easily applied to analyze crowdsourcing data. (ii) Photogrammetry can be combined with image segmentation to alleviate image distortions for monitoring purpose. (iii) Beyond the measurement of plant area, this method has the potential to be easily transferred into other tasks, such as monitoring cracks and erosion, or as a masking tool in the photogrammetry workflow.
Highlights
Preservation of the authenticity and physical integrity of heritage sites requires regular monitoring and maintenance through on-site inspections
K j=1 TPj + FNj + FPj njj tj for K appear in the image
The second disadvantage of the model is the large number of false positives (FPs) in the predictions, which may lead to an overestimation of the plant area
Summary
Preservation of the authenticity and physical integrity of heritage sites requires regular monitoring and maintenance through on-site inspections. Historic Environment Scotland (HES) manages approximately 300 properties, with some of the sites being. Crowdsourcing has been widely used in the past to assist heritage preservation works [5]. Liu et al Heritage Science (2022) 10:27 preservation and reconstruction works include [6, 7] and [8]. Due to the diversity of camera angles of crowdsourced images, properties (e.g. areas of objects) cannot be directly compared between distinctive images. This article tests a combination of computer vision and photogrammetry to measure potentially damaging plant growth from crowdsourced images of built heritage sites
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have